CN103678552A - Remote-sensing image retrieving method and system based on salient regional features - Google Patents

Abstract

The invention discloses a remote sensing image retrieval method and system based on salient region features, comprising the steps of: using a visual attention model to obtain an image saliency map; using an adaptive threshold method to binarize the image saliency map; combining the original image with the corresponding binary image The value-based saliency map is used to perform "mask" operation to obtain the salient areas of the image; the salient points in the salient areas of the image are extracted, and the salient points are clustered based on the features of the salient points to obtain the feature vectors describing the features of the salient areas; finally, according to the preset Similarity metrics for image retrieval. While ensuring the feature extraction efficiency, the invention improves the retrieval precision rate, improves the retrieval results, and conforms to the visual characteristics of human eyes.

Description

Remote Sensing Image Retrieval Method and System Based on Significant Regional Features

technical field

The invention belongs to the technical field of remote sensing image processing and image retrieval, and relates to a remote sensing image retrieval method and system based on salient region features.

Background technique

With the development of remote sensing technology, the acquisition of multi-source remote sensing image data has become increasingly simple. Massive remote sensing image data not only provides more data options for scientific research, but also brings many problems that need to be solved urgently. On the one hand, the current processing and analysis capabilities of image data are limited, which makes the use of remote sensing image data inefficient; on the other hand, remote sensing image data has the characteristics of space, diversity, and complexity. The development of management, browsing and query lags far behind the growth rate of remote sensing image data, making it often impossible to quickly find the required remote sensing image for specific applications. The lack of effective retrieval methods for massive remote sensing image data has become a bottleneck restricting the application of remote sensing image data, and it is imperative to study efficient remote sensing image retrieval methods.

Traditional remote sensing image retrieval methods include keyword-based retrieval and content-based retrieval. The keyword-based image retrieval technology uses a series of keywords to describe the images in the database through manual annotation. When the keywords are entered during retrieval, the system will return matching images. Although the retrieval process of this method is simple and easy to understand , but manual labeling is inefficient and highly subjective. Content-based image retrieval technology retrieves by extracting low-level visual features (spectral features, texture features and shape features) of images. Although this technology improves the retrieval results to a certain extent, spectral features and texture features are global features of images. Considering the characteristics of the foreground and background of the image, the semantic information of the image cannot be well described; although the shape feature is related to the specific target on the image, the extraction of the shape feature often requires the segmentation of the image, and segmentation itself is a big problem in the field of computer vision. problem. It can be seen that due to the existence of the semantic gap, the low-level visual features cannot effectively reflect the essential content of the image.

According to the theory of human vision, what people pay attention to in an image is not the content of the whole image but the salient area of the image. For this reason, people introduce the visual attention model into the field of image retrieval, and study the use of the visual attention model to calculate the saliency map of the image and extract it. Color, texture, edge and other features are retrieved. Compared with low-level visual features, this method is more in line with people's query intention and can effectively bridge the semantic gap. However, the saliency map is a fuzzy grayscale image, and it is very difficult to directly extract the color, texture, edge and other features of the saliency map.

Contents of the invention

Aiming at the deficiencies in the prior art, the present invention provides a remote sensing image retrieval method and system based on salient region features that can better reflect the retrieval requirements. Extract salient areas, and realize remote sensing image retrieval by extracting salient area features.

Technical scheme of the present invention is as follows:

1. A remote sensing image retrieval method based on salient regional features, including steps:

Step 1, using the visual attention model to obtain the saliency map of the image;

Step 2, converting the saliency map obtained in step 1 into a corresponding binarized saliency map;

Step 3, based on the original image and the binarized saliency map obtained in step 2, the salient area of the image is obtained;

Step 4, extracting the salient points in the salient area of the image obtained in step 3, based on the salient point features, using a clustering method to cluster the salient points, and obtaining a feature vector describing the salient area features;

Step 5, based on the feature vector obtained in step 4, perform image retrieval according to a preset similarity measurement criterion.

Step 1 is specifically:

The brightness, color and direction features of the image are obtained by constructing Gaussian pyramids of different scales, and the saliency map with the same size as the original image is obtained by fusing the brightness, color and direction features of the image.

Step 2 uses the adaptive threshold method to transform the saliency map into the corresponding binarized saliency map, and the binarization threshold is determined by the Otsu method.

In step 3, the "mask" operation is performed on the original image and its corresponding binarized saliency map to obtain the salient region of the image.

The features described in step 4 are bag-of-words features, and the bag-of-words features are obtained according to the Bag of Words algorithm principle in text retrieval.

Step 5 further includes sub-steps:

5.1 Preset similarity measurement criteria;

5.2 Based on the eigenvectors of the salient areas of the image, use the similarity measurement criterion to obtain the similarity values of the image to be retrieved and the image in the image library one by one;

5.3 The images in the image library are sorted according to the similarity value and then output.

2. Remote sensing image retrieval system based on salient regional features, including:

A saliency map acquisition module, which is used to obtain a saliency map of an image using a visual attention model;

The binarized saliency map acquisition module is used to convert the saliency map into a corresponding binarized saliency map;

The salient area acquisition module is used to acquire the salient area of the image based on the original image and the binarized saliency map;

The feature vector acquisition module is used to extract the salient points of the salient areas of the image, based on the features of the salient points, the clustering method is used to cluster the salient points, and the feature vector describing the features of the salient areas is obtained;

The image retrieval module is used to perform image retrieval based on feature vectors and according to preset similarity measurement criteria.

Compared with the prior art, the present invention has the following characteristics and beneficial effects,

1. Use the visual attention model to obtain the salient area of the image. In order to make up for the shortcomings of the visual attention model, the original image and the binarized saliency map are subjected to a "mask" operation to obtain the salient area of the image.

2. Extracting the features of salient regions overcomes the difficulty of directly extracting features from salient images; image retrieval based on the extracted salient region features not only conforms to the characteristics of human vision but also better reflects the retrieval needs, reducing the low-level visual features The distance between high-level semantics and high-level semantics can effectively improve the precision and recall of remote sensing image retrieval.

3. Good scalability. The features of salient regions used for retrieval include but not limited to bag-of-words features, as long as they can describe the content of salient regions.

Description of drawings

Fig. 1 is the flowchart of the embodiment of the present invention;

Fig. 2 is a graph of the average precision rate under different numbers of returned images in the embodiment of the present invention.

Detailed ways

A kind of specific implementation method of the present invention is: utilize Itti visual attention model to calculate the saliency map of all images in the image storehouse, and adopt " Otsu method " self-adaptive determination threshold value to binarize the saliency map; Binarize the saliency map with The corresponding original image is subjected to "mask" operation to obtain the salient area of the image; the salient points of the salient area of the image are extracted by using the SIFT operator, and the visual word feature vector (visual words) describing the bag-of-words feature of the salient area is obtained through clustering; Retrieve the images in the image library using the set similarity measure criterion.

The specific embodiment of the present invention will be described in detail below in conjunction with Fig. 1, specifically include steps:

Step 1, obtain the saliency map of the image in the remote sensing image database.

First, build a remote sensing image database for retrieval.

The image data used in the retrieval image library in this embodiment comes from the aerial images of several major cities in the United States with a resolution of 30 cm. According to the Tiles non-overlapping block method, the image is divided into sub-blocks of 256*256 size to form sub-blocks containing aircraft, sparse The retrieved image library of four types of ground objects, including residential areas, buildings, and parking lots, where each type of ground features contains 100 images.

Then, obtain the saliency map of the image in the remote sensing image library.

In this embodiment, the effective and mature Itti visual attention model is used to obtain the saliency map of the image. By constructing Gaussian pyramids of different scales, the brightness, color, and direction of the image are obtained. The calculation of the three characteristics is shown in formulas (1) to (3).

I＝(r+g+b)/3 （1）

In formula (1), I is the brightness feature of the image, and r, g, and b are the three components of the color respectively.

$\left\{\begin{array}{c}\mathrm{<span\; class="notranslate">\; RG</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; \&Theta;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span>\\ \mathrm{<span\; class="notranslate">\; BY</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; Y</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; \&Theta;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; Y</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span>\end{array}\right.<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

In formula (2):

RG and BY represent the color difference between red and green and blue and yellow, respectively, that is, the color characteristics of the image;

R, G, B, and Y represent red, green, blue, and yellow color channels respectively, R=r-(g+b)/2, G=g-(r+b)/2, B=b-(r+ g)/2, Y=(r+g)/2-|r-g|/2-b, r, g, b are three components of color respectively;

c and s represent the central scale and peripheral scale, respectively;

Θ represents the "center-surround" scaling operator.

O(c,s,θ)＝|O(c,θ)ΘO(s,θ)| （3）

In formula (3),

O(c,s,θ) represents the direction characteristics of different scale images;

O(c,θ) and O(s,θ) represent the central scale image features and peripheral scale image features respectively;

θ is the direction of the Gabor filter;

Θ has the same meaning as formula (2).

Finally, the brightness feature, color feature and direction feature are fused by multi-feature fusion method to obtain a saliency map with the same size as the original image.

In specific implementation, multi-feature fusion can be performed by calculating the arithmetic mean of the three features or calculating the weighted average according to the importance of the three features to obtain the final saliency map.

The formula for multi-feature fusion by calculating the arithmetic mean of the three features is as follows:

$\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; Intensity</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; color</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; Orientation</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

In formula (4), S represents the final saliency map, Intensity represents the brightness feature, Color represents the color feature, and Orientation represents the direction feature.

The formula for calculating the weighted average value for multi-feature fusion according to the importance of the three features is as follows:

$\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; Intensity</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&beta;</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; color</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&gamma;</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; Orientation</span>}}{\mathrm{<span\; class="notranslate">\; Intensity</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; color</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; Orientation</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

In formula (5), S represents the final saliency map; Intensity represents the brightness feature, Color represents the color feature, and Orientation represents the orientation feature; α, β, and γ are the weights of the brightness feature, color feature, and orientation feature, respectively. The importance setting of , and α+β+γ=1.

Step 2, saliency map binarization.

In this embodiment, the "Otsu method (Otsu)" is adopted to adaptively determine the binarization threshold of the saliency map. The Otsu method is a relatively mature algorithm for determining the threshold value of image binarization segmentation. According to this algorithm, the threshold value at which the variance between classes of the foreground and background of the image reaches the maximum value is the appropriate threshold value for binarization. For the calculation of the variance between classes, see Formula (6):

v＝w ₀ w ₁ (μ ₀ -μ ₁ ) ² (6)

In formula (6): v represents the variance between classes; w ₀ represents the proportion of the foreground pixels of the saliency map to the pixels of the entire saliency map; w ₁ represents the proportion of the background pixels of the saliency map to the pixels of the entire saliency map; μ ₀ represents the significant The average gray level of the foreground pixels in the map; μ ₁ represents the average gray level of the background pixels in the salient map.

Step 3, get the salient area of the image.

From the perspective of digital image processing, the so-called "mask" operation is to cover some pixels in the image and retain the required pixels. When performing the mask operation, the two images are required to have the same size.

A digital image is represented by a matrix whose elements are gray values. The larger the gray value of an area, the brighter the area, and the darker the area. The gray value of the binarized saliency map is only 0 and 1. When performing mask operation with the original image, the matrix corresponding to the original image and the binarized saliency map can be multiplied by array. At this time, the original image The pixels corresponding to the positions with a gray value of 1 in the binarized saliency map are retained, and the gray values of other pixels become 0. The salient areas of the image can be separated from the original image through the "mask" operation.

Let F represent the original image, S represent the binarized saliency map corresponding to the original image F, and R represent the result of the "mask" operation. The mask operation is shown in formula (7):

R＝F*S （7）

in,

$\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{cccc}\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 0,0</span>}<span\; class="notranslate">\; )</span>& \mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 0,1</span>}<span\; class="notranslate">\; )</span>& <span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&CenterDot;</span>& \mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\\ \mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1,0</span>}<span\; class="notranslate">\; )</span>& \mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1,1</span>}<span\; class="notranslate">\; )</span>& <span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&CenterDot;</span>& \mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\\ <span\; class="notranslate">\; \&CenterDot;</span>& & & <span\; class="notranslate">\; \&CenterDot;</span>\\ <span\; class="notranslate">\; \&Center\; Dot;</span>& & & <span\; class="notranslate">\; \&CenterDot;</span>\\ <span\; class="notranslate">\; \&Center\; Dot;</span>& & & <span\; class="notranslate">\; \&CenterDot;</span>\\ \mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1,0</span>}<span\; class="notranslate">\; )</span>& \mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1,1</span>}<span\; class="notranslate">\; )</span>& <span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; \&Center\; Dot;</span>& \mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\end{array}\right]<span\; class="notranslate">\; ;</span>$

$\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{cccc}\mathrm{<span\; class="notranslate">\; Logical</span>}& \mathrm{<span\; class="notranslate">\; Logical</span>}& <span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&CenterDot;</span>& \mathrm{<span\; class="notranslate">\; Logical</span>}\\ \mathrm{<span\; class="notranslate">\; Logical</span>}& \mathrm{<span\; class="notranslate">\; Logical</span>}& <span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; \&CenterDot;</span>& \mathrm{<span\; class="notranslate">\; Logical</span>}\\ <span\; class="notranslate">\; \&CenterDot;</span>& & & <span\; class="notranslate">\; \&CenterDot;</span>\\ <span\; class="notranslate">\; \&CenterDot;</span>& & & <span\; class="notranslate">\; \&CenterDot;</span>\\ <span\; class="notranslate">\; \&CenterDot;</span>& & & <span\; class="notranslate">\; \&CenterDot;</span>\\ \mathrm{<span\; class="notranslate">\; Logical</span>}& \mathrm{<span\; class="notranslate">\; Logical</span>}& <span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; \&CenterDot;</span>& \mathrm{<span\; class="notranslate">\; Logical</span>}\end{array}\right]<span\; class="notranslate">\; ;</span>$

Each element in F is a gray value between 0 and 255, and Logical in S is a logical value of 0 or 1.

Step 4, extract the bag-of-words features of the salient regions of the image.

In this embodiment, the SIFT operator is used to extract the bag-of-words features of the salient regions of the image. For each salient area of the image, the SIFT operator can be used to extract a K×128-dimensional array, that is, the salient point features of the salient area of the image, where K is the number of salient points extracted, and each row of the K×128-dimensional array represents a salient point corresponding to The 128-dimensional feature vector of .

In order to obtain the visual word feature vector describing the feature of the bag of words, in this embodiment, the K-means clustering algorithm is used to cluster the salient points in the salient area of each image, and a 1*k-dimensional bag of word feature describing the salient area of the image is obtained. Visual word feature vector V, where k represents the number of cluster centers, visual word feature vector V is as follows:

V=[a ₁ a ₂ … a _k ] (8)

In formula (8), each element in V represents the number of significant points corresponding to each cluster center.

Step 5, image retrieval based on similarity criterion.

After step 4 is completed, the images in the remote sensing image library are described by visual word feature vectors, and image retrieval is performed according to the preset similarity measurement criteria. Commonly used similarity measurement methods include urban distance, Euclidean distance, histogram intersection method, quadratic distance, cosine distance, correlation coefficient, and KL divergence.

In this embodiment, the simpler Euclidean distance is used as the similarity measure criterion to measure the similarity between the images to be retrieved and the images in the image database. The calculation of the Euclidean distance is shown in formula (9). Calculate the similarity between the image to be retrieved and all the images in the image library according to formula (9), and return the retrieval results in a certain order. Usually, the similarity is sorted in descending order and then output, so that the higher the position in the output results An image of means that it has a higher similarity with the image to be retrieved.

$\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>}^{\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 9</span>}<span\; class="notranslate">\; )</span>$

In formula (9), I ₁ and I ₂ represent two images for similarity calculation; L(I ₁ , I ₂ ) represents the Euclidean distance between I ₁ and I ₂ ; a _i and b _i represent images I ₁ and The ith element in the visual word feature vector of I ₂ , n is the feature vector dimension.

When calculating the Euclidean distance, the feature vector dimensions of the images are required to be the same. At the same time, the value calculated by formula (9) strictly speaking does not indicate the similarity between the images but the gap between the images. Therefore, L(I ₁ , I ₂ ) The smaller the value, the greater the similarity between images I ₁ and I ₂ .

In summary, after the method of the present invention uses the Itti visual attention model to calculate the saliency map of the image, it binarizes the saliency map and performs a "mask" operation with the original image to obtain a salient area image that conforms to the visual characteristics of the human eye; The SIFT operator extracts the salient points of the salient regions of the image. At the same time, in order to obtain the feature vector of visual words describing the feature of bag-of-words in the salient region, the present invention uses the K-means clustering algorithm to cluster the salient points to obtain the feature vector of visual words. The method of the invention can effectively improve retrieval results and improve retrieval accuracy.

The beneficial effect of the present invention is verified by simulation experiment as follows:

In this experiment, a total of 400 remote sensing images of 4 categories were used to construct an image database for retrieval. Each type of remote sensing image contained 100 images, and the size was 256*256. In this experiment, two evaluation indexes, the average precision rate and the precision rate under different numbers of returned images, are used to evaluate the image retrieval effect.

The precision rate refers to the ratio of the number of similar images in the returned images to the number of all returned images. By setting the number of returned images, the precision rate of images under different numbers of returned images can be obtained, and then the average of each method can be calculated. Precision.

See Figure 2, where method 1 is a traditional image retrieval method based on color features, method 2 is a traditional image retrieval method based on texture features, and method 3 is to use the SIFT operator to directly extract word bag features from the original image for retrieval method, method 4 is the method of the present invention. It can be seen from Figure 2 that when the number of returned images is small, the above methods can maintain a high precision rate; but as the number of returned images increases, the retrieval precision rate of methods 1-3 drops rapidly, while this method The retrieval precision rate of the inventive method drops slowly, and can still maintain a high precision rate when the number of returned images is large.

The average precision of methods 1 to 4 for the four types of ground objects is shown in Table 1:

Table 1 Average precision rate statistics table

average precision method 1 0.4150 Method 2 0.4810 Method 3 0.3687 Method 4 0.5737

It can be seen from Table 1 that the method of the present invention has the best average precision rate, can well describe the salient content of various images, and has the best retrieval effect.

The above content is a further detailed description of the present invention in combination with the best embodiments, and it cannot be assumed that the specific implementation of the present invention is limited to these descriptions. It should be understood by those skilled in the art that various modifications can be made in the details without departing from the conditions defined by the appended claims, which should be regarded as the protection scope of the present invention.

Claims (7)

1. The remote sensing image retrieval method based on salient regional features, is characterized in that, comprises the steps: Step 1, using the visual attention model to obtain the saliency map of the image; Step 2, converting the saliency map obtained in step 1 into a corresponding binarized saliency map; Step 3, based on the original image and the binarized saliency map obtained in step 2, the salient area of the image is obtained; Step 4, extracting the salient points in the salient area of the image obtained in step 3, based on the salient point features, using a clustering method to cluster the salient points, and obtaining a feature vector describing the salient area features; Step 5, based on the feature vector obtained in step 4, perform image retrieval according to a preset similarity measurement criterion. 2. The remote sensing image retrieval method based on salient regional features as claimed in claim 1, characterized in that: Step 1 is specifically: The brightness, color and direction features of the image are obtained by constructing Gaussian pyramids of different scales, and the saliency map with the same size as the original image is obtained by fusing the brightness, color and direction features of the image. 3. The remote sensing image retrieval method based on salient regional features as claimed in claim 1, characterized in that: Step 2 uses the adaptive threshold method to transform the saliency map into the corresponding binarized saliency map, and the binarization threshold is determined by the Otsu method. 4. The remote sensing image retrieval method based on salient regional features as claimed in claim 1, characterized in that: In step 3, the "mask" operation is performed on the original image and its corresponding binarized saliency map to obtain the salient region of the image. 5. The remote sensing image retrieval method based on salient regional features as claimed in claim 1, characterized in that: The features described in step 4 are bag-of-words features. 6. The remote sensing image retrieval method based on salient regional features as claimed in claim 1, characterized in that: Step 5 further includes sub-steps: 5.1 Preset similarity measurement criteria; 5.2 Based on the feature vector of the salient area of the image, use the similarity measurement criterion to obtain the similarity value of the image to be retrieved and the image in the image library one by one; 5.3 After sorting the images in the image library according to the similarity value, output them. 7. A remote sensing image retrieval system based on salient regional features, characterized in that it includes: A saliency map acquisition module, which is used to obtain a saliency map of an image using a visual attention model; The binarized saliency map acquisition module is used to convert the saliency map into a corresponding binarized saliency map; The salient area acquisition module is used to acquire the salient area of the image based on the original image and the binarized saliency map; The feature vector acquisition module is used to extract the salient points of the salient areas of the image, based on the features of the salient points, the clustering method is used to cluster the salient points, and the feature vector describing the features of the salient areas is obtained; The image retrieval module is used to perform image retrieval based on feature vectors and according to preset similarity measurement criteria.

Images