CN103678552A - Remote-sensing image retrieving method and system based on salient regional features - Google Patents

Abstract

The invention discloses a remote-sensing image retrieving method and system based on salient regional features. The method comprises the steps of obtaining image saliency maps by utilizing a visual attention model, performing binaryzation on the image saliency maps through the adaptive threshold method, performing mask operation on original images and the corresponding binaryzation saliency maps to obtain salient regions of the images, extracting saliency points in the salient regions of the images, performing clustering on the saliency points based on saliency point features to obtain feature vectors for describing the salient regional features, and finally performing image retrieval according to preset similarity measure criteria. According to the remote-sensing image retrieving method and system based on the salient regional features, feature extraction efficiency is guaranteed, meanwhile, the precision ratio of retrieving is enhanced, and retrieved results are improved. Besides, the system and method conform to human vision features.

Description

Remote Sensing Image Retrieval method and system based on marking area feature

Technical field

The invention belongs to remote sensing image processing video search technical field, relate to a kind of Remote Sensing Image Retrieval method and system based on marking area feature.

Background technology

Along with development of remote sensing, obtaining of Methods on Multi-Sensors RS Image becomes day by day simple.Magnanimity remote sensing image data has also brought many difficult problems urgently to be resolved hurrily providing more data to select for scientific research when.On the one hand, present stage processing and the analysis ability of image data limited, make the service efficiency of remote sensing image data low; On the other hand, remote sensing image data has the features such as spatiality, diversity, complicacy, and at present the tissue of remote sensing image data, the development degree managing, browse and inquire about lag behind the growth rate of remote sensing image data far away, make often can not find fast required remote sensing image for application-specific.The shortage of effective search method of magnanimity remote sensing image data has become the bottleneck of restriction remote sensing image data application, studies efficient Remote Sensing Image Retrieval method imperative.

Traditional Remote Sensing Image Retrieval method comprises retrieval and the content-based retrieval based on key word.Video search technology based on key word is described the image in database with a series of key word by artificial mark mode, during retrieval, input key word, system will be returned to the image conforming to it, although the method retrieving simply and is easily understood, but manual mark efficiency is low, and subjectivity is strong.Content-based video search technology is retrieved by extracting the Low Level Vision feature (spectral signature, textural characteristics and shape facility) of image, although this technology has been improved result for retrieval to a certain extent, but spectral signature and textural characteristics are the global characteristics of image, do not consider the feature of image prospect and background, can not describe well the semantic information of image; Although shape facility is relevant with specific objective on image, the extraction of shape facility often needs image to cut apart, and cut apart, itself is exactly a great problem of computer vision field.Can find out, due to the existence of semantic gap, Low Level Vision feature can not effectively reflect the essential content of image.

Theoretical according to human eye vision, what for width image people, pay close attention to is not the content of view picture image but the marking area of image, people introduce video search field by visual attention model for this reason, and research and utilization visual attention model calculates the remarkable figure of image and extracts the features such as color, texture, edge and retrieve.Compare Low Level Vision feature, the method more meets people's query intention and can effectively make semantic gap up, yet significantly figure is fuzzy gray level image, and the features such as color, texture, edge of directly extracting remarkable figure are very difficult.

Summary of the invention

The deficiency existing for prior art, the invention provides and a kind ofly can better reflect Remote Sensing Image Retrieval method and system Search Requirement, based on marking area feature, the present invention is by analyzing human eye vision attention characteristic, from complicated remote sensing image, extract marking area, by extracting marking area feature, realize Remote Sensing Image Retrieval.

Technical scheme of the present invention is as follows:

One, the Remote Sensing Image Retrieval method based on marking area feature, comprises step:

Step 1, utilizes visual attention model to obtain the remarkable figure of image;

Step 2, is converted into corresponding binaryzation by the remarkable figure of step 1 gained and significantly schemes;

Step 3, obtains the marking area of image based on raw video and the remarkable figure of step 2 gained binaryzation;

Step 4, the significant point of the extraction step 3 image marking area that obtains, based on significant point feature, adopts clustering method to carry out cluster to significant point, obtains the proper vector of describing marking area feature;

Step 5, based on proper vector that step 4 obtains, carries out video search according to default similarity measurement criterion.

Step 1 is specially:

By the gaussian pyramid of structure different scale, obtain brightness, color characteristic and the direction character of image, merge brightness, color characteristic and the direction character acquisition of image and the remarkable figure of raw video consistent size.

Step 2 employing adaptive threshold method is converted into corresponding binaryzation by remarkable figure significantly to be schemed, and binary-state threshold adopts large Tianjin method to determine.

In step 3, by the remarkable figure of the raw video binaryzation corresponding with it is carried out to " mask " computing, obtain the marking area of image.

Described in step 4, be characterized as word bag feature, word bag feature obtains according to Bag of Words algorithm principle in text retrieval.

Step 5 further comprises sub-step:

5.1 default similarity measurement criterions;

5.2 proper vectors based on image marking area, adopt similarity measurement criterion to obtain one by one the similarity of image in image to be retrieved and Image Database;

5.3 export in Image Database, image sequence is rear according to similarity size.

Two, the Remote Sensing Image Retrieval system based on marking area feature, comprising:

Significantly figure acquisition module, is used for utilizing visual attention model to obtain the remarkable figure of image;

The remarkable figure acquisition module of binaryzation, is used for that remarkable figure is converted into corresponding binaryzation and significantly schemes;

Marking area acquisition module, is used for obtaining based on raw video and the remarkable figure of binaryzation the marking area of image;

Proper vector acquisition module, is used for extracting the significant point of image marking area, based on significant point feature, adopts clustering method to carry out cluster to significant point, obtains the proper vector of describing marking area feature;

Video search module, is used for based on proper vector, according to default similarity measurement criterion, carries out video search.

Compared with prior art, the present invention has following features and beneficial effect,

1, adopt visual attention model to obtain the marking area of image, in order to make up the shortcoming of visual attention model, raw video and the remarkable figure of binaryzation are carried out to the marking area that " mask " computing obtains image.

2, extract the feature of marking area, overcome and directly from remarkable figure, extracted the difficulty of feature; Marking area feature based on extracting is carried out video search, not only meets human eye vision feature and can reflect better Search Requirement, has dwindled the distance between Low Level Vision feature and high-level semantic, can effectively improve precision ratio and the recall ratio of Remote Sensing Image Retrieval.

3, favorable expandability, includes but not limited to word bag feature for the feature of the marking area retrieved, as long as can describe the feature of marking area content, all can.

Accompanying drawing explanation

Fig. 1 is the process flow diagram of the embodiment of the present invention;

Fig. 2 is that in the embodiment of the present invention, difference is returned to the average precision figure under image number.

Embodiment

A kind of specific implementation method of side of the present invention is: utilize Itti visual attention model to calculate the remarkable figure of all images in Image Database, and adopt " large Tianjin method " self-adaptation definite threshold by remarkable figure binaryzation; Binaryzation is significantly schemed to carry out with corresponding raw video the marking area that " mask " computing obtains image; Utilize the significant point of SIFT operator extraction image marking area, through cluster, obtain describing the vision word feature vector (visual words) of marking area word bag feature; According to image in default similarity measurement criterion retrieval Image Database.

Below in conjunction with Fig. 1, describe the specific embodiment of the present invention in detail, specifically comprise step:

Step 1, obtains the remarkable figure of image in Remote Sensing Image Database.

First, build retrieval Remote Sensing Image Database.

The image data adopting in the present embodiment retrieval Image Database comes from several metropolitan aviation images of the U.S. that resolution is 30cm, according to Tiles zero lap partitioned mode, it by Image Segmentation, is the sub-block of 256*256 size, form the retrieval Image Database that comprises aircraft, sparse residential block, buildings and parking lot 4 class atural objects, wherein every class atural object comprises 100 width images.

Then, obtain the remarkable figure of image in Remote Sensing Image Database.

The present embodiment adopts effect better and more ripe Itti visual attention model obtains the remarkable figure of image.By the gaussian pyramid of structure different scale, obtain three kinds of features of brightness, color, direction of image.Formula (1)～(3) are shown in the calculating of these three kinds of features.

I＝(r+g+b)/3 （1）

In formula (1), the brightness that I is image, r, g, b are respectively three components of color.

$\left\{\begin{array}{c}\mathrm{RG}(c,s)=\left|(R\left(c\right)-G\left(c\right))\mathrm{\Θ}(G\left(s\right)-R\left(s\right))\right|\\ \mathrm{BY}(c,s)=\left|(B\left(c\right)-Y\left(c\right))\mathrm{\Θ}(Y\left(s\right)-B\left(s\right))\right|\end{array}\right.---\left(2\right)$

In formula (2):

RG and BY represent respectively the color distortion between red green and blue Huang, the i.e. color characteristic of image;

R, G, B, Y represent respectively red, green, blue, yellow Color Channel, R=r-(g+b)/2, and G=g-(r+b)/2, B=b-(r+g)/2, Y=(r+g)/2-|r-g|/2-b, r, g, b are respectively three components of color;

C, s represent respectively central yardstick and periphery yardstick;

Θ represents " central authorities-periphery (center-surround) " yardstick operator.

O(c,s,θ)＝|O(c,θ)ΘO(s,θ)| （3）

In formula (3),

O (c, s, θ) represents the direction character of different scale image;

O (c, θ) and O (s, θ) represent respectively central yardstick image feature and periphery yardstick image feature;

θ is the direction of Gabor wave filter;

Θ implication cotype (2).

Finally by many Feature fusions, brightness, color characteristic and direction character are merged, obtain the remarkable figure consistent with former image size.

During concrete enforcement, can, by asking for the arithmetic mean of three kinds of features or asking for weighted mean value according to the importance of three kinds of features and carry out many Fusion Features, obtain final significantly figure.

By asking for the arithmetic mean of three kinds of features, to carry out the formula of many Fusion Features as follows:

$S=\frac{1}{3}(\mathrm{Intensity}+\mathrm{Color}+\mathrm{Orientation})---\left(4\right)$

In formula (4), the final significantly figure of S representative, Intensity represents brightness, and Color represents color characteristic, and Orientation represents direction character.

According to the importance of three kinds of features, asking for weighted mean value, to carry out the formula of many Fusion Features as follows:

$S=\frac{\mathrm{\α}*\mathrm{Intensity}+\mathrm{\β}*\mathrm{Color}+\mathrm{\γ}*\mathrm{Orietation}}{\mathrm{Intensity}+\mathrm{Color}+\mathrm{Orientation}}---\left(5\right)$

In formula (5), the final significantly figure of S representative; Intensity represents brightness, and Color represents color characteristic, and Orientation represents direction character; α, β, γ are respectively brightness, color characteristic and directional characteristic weights, according to the importance of three kinds of features, set, and alpha+beta+γ=1.

Step 2, significantly figure binaryzation.

The present embodiment adopts " large Tianjin method (Otsu) " to determine adaptively the binary-state threshold of remarkable figure.Large Tianjin method is a kind of algorithm of more ripe definite image binaryzation segmentation threshold, makes the inter-class variance of image prospect and background reach peaked threshold value be applicable binary-state threshold according to this algorithm, and formula (6) is shown in the calculating of inter-class variance:

v＝w ₀w ₁(μ ₀-μ ₁) ² （6）

In formula (6): v represents inter-class variance; w ₀represent that remarkable figure foreground pixel number accounts for the ratio of view picture specific image prime number; w ₁represent that remarkable figure background pixel number accounts for the ratio of view picture specific image prime number; μ ₀the average gray that represents remarkable figure foreground pixel; μ ₁the average gray that represents remarkable figure background pixel.

Step 3, obtains the marking area of image.

From the angle of Digital Image Processing, so-called " mask " computing is exactly to cover some pixel in image and retain required pixel, requires two width image sizes identical while carrying out mask computing.

Digital picture is to represent by take the matrix that gray-scale value is element, and this region of the larger expression of area grayscale value is brighter, otherwise represents that this region is darker.Binaryzation is significantly schemed its gray-scale value and is only had 0 and 1 two kind, when itself and raw video carry out mask computing, raw video and binaryzation are significantly schemed to corresponding matrix and carry out array multiplication computing, now on raw video with the remarkable figure of binaryzation in gray-scale value be 1 the corresponding pixel in position is retained, rest of pixels gray-scale value becomes 0.By " mask " computing, can isolate from raw video the marking area of image.

If F represents raw video, S represents that the binaryzation that raw video F is corresponding significantly schemes, and R represents " mask " operation result, and formula (7) is shown in mask computing:

R＝F*S （7）

Wherein,

$F=\left[\begin{array}{cccc}f\left(\mathrm{0,0}\right)& f\left(\mathrm{0,1}\right)& \·\·\·& f(0,n-1)\\ f\left(\mathrm{1,0}\right)& f\left(\mathrm{1,1}\right)& \·\·\·& f(1,n-1)\\ \·& & & \·\\ \·& & & \·\\ \·& & & \·\\ f(m-\mathrm{1,0})& f(m-\mathrm{1,1})& \·\·\·& f(m-1,n-1)\end{array}\right];$

$S=\left[\begin{array}{cccc}\mathrm{Logical}& \mathrm{Logical}& \·\·\·& \mathrm{Logical}\\ \mathrm{Logical}& \mathrm{Logical}& \·\·\·& \mathrm{Logical}\\ \·& & & \·\\ \·& & & \·\\ \·& & & \·\\ \mathrm{Logical}& \mathrm{Logical}& \·\·\·& \mathrm{Logical}\end{array}\right];$

In F, each element is the gray-scale value of 0～255, and in S, Logical is logical value 0 or 1.

Step 4, the word bag feature of extraction image marking area.

In the present embodiment, adopt the word bag feature of SIFT operator extraction image marking area.For each image marking area, adopt SIFT operator can extract the array that obtains K * 128 dimension, i.e. the significant point feature of image marking area, the significant point number of K for extracting wherein, 128 dimensional feature vectors corresponding to significant point of the every line display of K * 128 dimension group.

In order to obtain the vision word feature vector of descriptor bag feature, in the present embodiment, adopt K-means clustering algorithm to carry out cluster to the significant point of every width image marking area, obtain the vision word feature vector V of the description image marking area word bag feature of 1*k dimension, wherein, k represents the number of cluster centre, and V is as follows for vision word feature vector:

V=[a ₁ a ₂ … a _k] (8)

In formula (8), significant point number corresponding to each cluster centre of each element representation in V.

Step 5, the video search based on similarity criteria.

After completing steps 4, the image in Remote Sensing Image Database all adopts vision word feature vector to be described, and according to default similarity measurement criterion, carries out video search.Conventional method for measuring similarity comprises city distance, Euclidean distance, histogram intersection method, quadratic form distance, cosine distance, related coefficient and KL divergence etc.

In the present embodiment, adopt better simply Euclidean distance as similarity measurement criterion, to weigh the similarity size of image in image to be retrieved and Image Database, formula (9) is shown in the calculating of Euclidean distance.By formula (9), calculate respectively the similarity size of all images in image to be retrieved and Image Database, and return in certain sequence result for retrieval, conventionally similarity size is arranged to rear output by descending mode, in Output rusults, the more forward image in position represents that itself and image similarity to be retrieved are higher like this.

$L=(I_1,I_2)={\left(\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{(a_i-b_i)}^2\right)}^{\frac{1}{2}}---\left(9\right)$

In formula (9), I ₁and I ₂represent the two width images that carry out similarity calculating; L(I ₁, I ₂) expression I ₁and I ₂euclidean distance; a _i, b _irepresent respectively image I ₁and I ₂vision word feature vector in i element, n is proper vector dimension.

During compute euclidian distances, require the proper vector dimension of image identical, the numerical value that simultaneously formula (9) calculates do not represent strictly speaking the similarity size between image and means the gap size between image, therefore, and L (I ₁, I ₂) the less expression image I of value ₁and I ₂similarity is larger.

In sum, the inventive method, adopting Itti visual attention model to calculate after the remarkable figure of image, is carried out remarkable figure binaryzation and is carried out " mask " computing with raw video, obtains meeting the marking area image of human eye vision feature; Utilize the significant point of SIFT operator extraction image marking area.Meanwhile, in order to obtain describing the vision word feature vector of marking area image word bag feature, the present invention adopts K-means clustering algorithm that significant point cluster is obtained to vision word feature vector.The inventive method can effectively be improved result for retrieval, improves retrieval precision ratio.

By emulation experiment, verify beneficial effect of the present invention below:

This experiment adopts 4 classes totally 400 width remote sensing images structure retrieval Image Databases, and every class remote sensing image all comprises 100 width images, and size is 256*256.This experiment adopts average precision and different two evaluation indexes of precision ratio of returning under image number to evaluate video search effect.

Similar image number and the ratio of all image numbers that return in the image that precision ratio refers to return, the image number returning by setting can obtain returning in difference the precision ratio of the image under image number, and then calculates the average precision of each method.

See Fig. 2, wherein, method 1 is traditional method for retrieving image based on color characteristic, and method 2 is traditional method for retrieving image based on textural characteristics, method 3 is to utilize SIFT operator directly from raw video, to extract the method that word bag feature is retrieved, and method 4 is the inventive method.As can be seen from Figure 2, returning to image number when less, said method all can keep higher precision ratio; But increase along with returning to image number, the retrieval precision ratio of method 1～3 declines very fast, and that the retrieval precision ratio of the inventive method declines is slower, still can keep higher precision ratio when a lot of returning to image number.

The average precision of 1～4 pair of four class atural object of method is as shown in table 1:

Table 1 average precision statistical form

	Average precision
		Method 1	0.4150
Method 2	0.4810
		Method 3	0.3687
Method 4	0.5737

The inventive method has optimum average precision as can be seen from Table 1, can describe well the remarkable content of all kinds of images, and retrieval effectiveness is best.

Above content is the further description of the present invention being done in conjunction with optimum implementation, can not assert that specific embodiment of the invention is only limited to these explanations.Those skilled in the art will be understood that not departing from limited by appended claims in the situation that, can carry out in detail various modifications, all should be considered as protection scope of the present invention.

Claims (7)

1. the Remote Sensing Image Retrieval method based on marking area feature, is characterized in that, comprises step:

Step 1, utilizes visual attention model to obtain the remarkable figure of image;

Step 2, is converted into corresponding binaryzation by the remarkable figure of step 1 gained and significantly schemes;

Step 3, obtains the marking area of image based on raw video and the remarkable figure of step 2 gained binaryzation;

Step 4, the significant point of the extraction step 3 image marking area that obtains, based on significant point feature, adopts clustering method to carry out cluster to significant point, obtains the proper vector of describing marking area feature;

Step 5, based on proper vector that step 4 obtains, carries out video search according to default similarity measurement criterion.

2. the Remote Sensing Image Retrieval method based on marking area feature as claimed in claim 1, is characterized in that:

Step 1 is specially:

By the gaussian pyramid of structure different scale, obtain brightness, color characteristic and the direction character of image, merge brightness, color characteristic and the direction character acquisition of image and the remarkable figure of raw video consistent size.

3. the Remote Sensing Image Retrieval method based on marking area feature as claimed in claim 1, is characterized in that:

Step 2 employing adaptive threshold method is converted into corresponding binaryzation by remarkable figure significantly to be schemed, and binary-state threshold adopts large Tianjin method to determine.

4. the Remote Sensing Image Retrieval method based on marking area feature as claimed in claim 1, is characterized in that:

In step 3, by the remarkable figure of the raw video binaryzation corresponding with it is carried out to " mask " computing, obtain the marking area of image.

5. the Remote Sensing Image Retrieval method based on marking area feature as claimed in claim 1, is characterized in that:

Described in step 4, be characterized as word bag feature.

6. the Remote Sensing Image Retrieval method based on marking area feature as claimed in claim 1, is characterized in that:

Step 5 further comprises sub-step:

5.1 default similarity measurement criterions;

5.2 proper vectors based on image marking area, adopt similarity measurement criterion to obtain one by one the similarity of image in image to be retrieved and Image Database;

5.3 export in Image Database, image sequence is rear according to similarity size.

7. the Remote Sensing Image Retrieval system based on marking area feature, is characterized in that, comprising:

Significantly figure acquisition module, is used for utilizing visual attention model to obtain the remarkable figure of image;

The remarkable figure acquisition module of binaryzation, is used for that remarkable figure is converted into corresponding binaryzation and significantly schemes;

Marking area acquisition module, is used for obtaining based on raw video and the remarkable figure of binaryzation the marking area of image;

Proper vector acquisition module, is used for extracting the significant point of image marking area, based on significant point feature, adopts clustering method to carry out cluster to significant point, obtains the proper vector of describing marking area feature;

Video search module, is used for based on proper vector, according to default similarity measurement criterion, carries out video search.

Images