CN102073748A - Visual keyword based remote sensing image semantic searching method - Google Patents
Visual keyword based remote sensing image semantic searching method Download PDFInfo
- Publication number
- CN102073748A CN102073748A CN 201110054624 CN201110054624A CN102073748A CN 102073748 A CN102073748 A CN 102073748A CN 201110054624 CN201110054624 CN 201110054624 CN 201110054624 A CN201110054624 A CN 201110054624A CN 102073748 A CN102073748 A CN 102073748A
- Authority
- CN
- China
- Prior art keywords
- image
- keyword
- vision
- feature
- vision keyword
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention relates to a visual keyword based remote sensing image semanteme searching method. The method comprises the following steps: setting visual keywords which describe image contents in an image base; selecting a training image from the image base; extracting remarkable visual characteristics of each training image, wherein the remarkable visual characteristics include remarkable points, main dominant tone and texture; acquiring a key mode through a cluster center of a cluster algorithm; establishing a visual keyword hierarchical model by adopting a Gaussian mixture model; extracting the remarkable visual characteristics of all images in the image base, setting weight parameters, and constructing a visual keyword characteristic vector describing the image semanteme; and calculating the similarity between an image to be searched and all images according to the similarity criterion, and outputting a search result according to the high-low sequence of the similarity. The method can effectively improve the recall ratio and the precision ratio of image searching by establishing the correlation between low-layer remarkable visual characteristics and high-layer semantic information through the visual keywords, and the technical scheme provided by the invention has excellent expansibility.
Description
Technical field
The present invention relates to technical field of image processing, more particularly, relate to a kind of remote sensing image semantic retrieving method based on the vision keyword.
Background technology
Remote sensing image data is used the contradiction that is being faced with " data are not only much but also fewer ".On the one hand, develop rapidly along with Aero-Space and various kinds of sensors technology, computer networking technology, database technology etc., retrievable various remote sensing image data product, particularly the high spatial resolution remote sense image data are all increasing every day with surprising rapidity; On the other hand, in so immense remote sensing image data warehouse, people but generally feel and want to find fast interested target not a duck soup.This is because remote sensing image data itself has characteristics such as spatiality, diversity, complicacy and magnanimity, makes that shortage has hindered the application of remote sensing image data to effective search method of magnanimity remote sensing image data at present.The efficient retrieval of remote sensing image is to solve mass remote sensing data and people remotely-sensed data is used the key of the contradiction between the growing demand, is the difficult problem that present remote sensing application field needs to be resolved hurrily, and also is the forward position of subject research.
In every gordian technique that remote sensing image retrieval research institute relates to, the visualization feature that present research emphasis mainly concentrates on remote sensing image (comprises spectral signature, textural characteristics, shape facility and assemblage characteristic) extract and the similarity matching algorithm on, wherein to the research of textural characteristics be most widely used and go deep into, the description of target shape feature and extraction relative spectral feature, textural characteristics is a very complicated problems, so far the definite mathematical definition that does not also have " shape ", in the at present content-based video search, the shape of target adopts edge and provincial characteristics to describe usually, but it is still perfect not to the utmost to describe the research of operator and shape similarity coupling thereof for the edge of target and provincial characteristics; The difficulty that the based target shape facility is described and extracted, although people more and more recognize its significance in the remote sensing image retrieval, achievement in research is but very limited.Aspect the remote sensing image retrieval of assemblage characteristic, mainly contain retrieval based on color harmony texture-combined feature.The algorithm that data pre-service (blocking organization or pre-service automatically) and visualization feature combine also is based on textural characteristics.
Because the low layer visualization feature can not reflect the semantic information of image intuitively, there be not the down auxiliary of experts database or domain knowledge base, all can produce the result for retrieval of " the non-gained of asking " usually.Address this problem, improve recall precision and retrieval rate, on search method, must break through dependence visualization feature.Remote sensing image high-level semantic feature has comprised the understanding of people to presentation content, search method based on semanteme is not only taken visual signatures such as color, texture, shape into account, and emphasis is to the semantic description of presentation content, therefore semantic retrieval is more abundant more, accurate, intelligent higher than the content retrieval based on visual signature.Yet the remote sensing image retrieval based on semanteme at present still rests on the exploratory stage.
Summary of the invention
The objective of the invention is to shortcoming and defect at prior art, a kind of remote sensing image semantic retrieving method based on the vision keyword is provided, by meeting the image analysing computer method of human visual perception characteristic, the abstract vision keyword of remote sensing image feature with complexity for having semantic information, set up association between low-level image feature, middle level object and the high-layer semantic information by the vision keyword, the method that provides can be applicable to various dissimilar remote sensing image searching fields.
The technical solution adopted in the present invention is that a kind of remote sensing image based on the vision keyword is retrieved semantic method, may further comprise the steps:
Step 1, setting can be described the vision keyword of image type in the image storehouse, and select some width of cloth images that can reflect each vision keyword from the image storehouse respectively, as the training image;
Step 2 is extracted all kinds of remarkable visual signature that all train images;
Step 3 to all training images, adopts clustering algorithm to carry out cluster respectively all kinds of remarkable visual signature that obtains, and obtains the cluster centre that equates with vision keyword number, and each cluster centre is mapped as a critical mode; Adopt the arbitrary remarkable visual signature of gauss hybrid models match to belong to the probability density function of every class vision keyword, the gauss hybrid models parameter estimation is come the self-training image, approximating method adopts the expectation maximization method of estimation, thereby sets up vision key word hierarchy model;
Step 4 adopts the mode consistent with step 2 to extract all kinds of remarkable visual signature of all images in the image storehouse respectively;
Step 5, at each width of cloth image in the image storehouse, calculate the probability that remarkable visual signature belongs to every class vision keyword by step 3 gained probability density function, if belong to the probability maximum of certain class vision keyword, think that then remarkable visual signature belongs to such vision keyword, thereby realize of the mapping of remarkable visual signature to the vision keyword;
Step 6 at each width of cloth image in the image storehouse, according to default setting weight parameter, is added up the frequency that every class vision keyword occurs in this image, and then makes up the vision keyword feature vector of describing this image semanteme;
Step 7 adopts default similarity measurement criterion, by the similarity of all images in vision keyword feature vector calculation image to be retrieved and the image storehouse, result for retrieval is sorted from high to low according to similarity and exports.
And in step 2 and the step 4, the remarkable visual signature of extraction comprises dominant hue and the texture that significant point, object drive.
And the implementation of extracting remarkable visual signature is as follows,
(1) utilize the SIFT image local feature to describe the significant point of all training images of operator extraction, thereby obtain the significant point feature of image, each significant point is represented with one 128 dimensional feature vector;
(2) all training images are carried out over-segmentation based on Quick Shift algorithm, the over-segmentation result is carried out the zone to be merged, then the consistance subject area is adopted the HSV model, quantized result according to its tone passage extracts each regional dominant hue, thereby obtain the dominant hue feature of image, the dominant hue feature of each subject area is represented with a proper vector;
(3) all training images are carried out over-segmentation based on Quick Shift algorithm, the over-segmentation result is carried out the zone to be merged, then the consistance subject area is adopted wavelet transformation, obtain the average of each yardstick high fdrequency component and variance as texture descriptor, thereby obtain the textural characteristics of image, the textural characteristics of each subject area is represented with a proper vector.
And in the step 6, when weight parameter was set, the significant point feature was composed with the average weight, and dominant hue feature and textural characteristics are weight with the area of self subject area.
And in the step 3, the clustering algorithm that is adopted is K average or ISODATA algorithm.
And in the step 7, default similarity measurement criterion is the first approximation distance of KL divergence.
The beneficial effect of technical scheme provided by the invention is, set up related between the remarkable visual signature of low layer and the high-layer semantic information by the hierarchical model of vision keyword, dwindled " semantic wide gap " between remarkable visual signature of low layer and the high-level semantic, provide a new solution route for from magnanimity remote sensing image storehouse, locating fast and searching interesting target, can effectively improve the recall ratio and the precision ratio of video search.Technical scheme provided by the invention has good extendability simultaneously, the remarkable visual signature that is adopted includes but not limited to employed significant point, dominant hue and texture among the present invention, so long as meet the feature of human visual system, can both successfully include in the technical scheme provided by the invention.
Description of drawings
Fig. 1 is the process flow diagram of the embodiment of the invention.
Fig. 2 is the effect synoptic diagram of the embodiment of the invention.
Embodiment
The remote sensing image semantic retrieving method based on the vision keyword that the present invention proposes is provided with the vision keyword of reflection image storehouse content earlier, select the training image, extract remarkable visual signature, set up vision key word hierarchy model then, realize related between low layer visual signature and the high-level semantic, remote sensing image is carried out semantic modeling and description, adopt the similarity criterion that the image in the image storehouse is retrieved at last.Wherein mainly comprise training image remarkable Visual Feature Retrieval Process, set up vision key word hierarchy model, the semantic modeling of remote sensing image and based on four processes of video search of similarity criterion.
For describing embodiment in detail, referring to Fig. 1, the embodiment flow process is as follows:
Step S01 sets the vision keyword that is used for describing image storehouse presentation content.
The data that embodiment adopted come from the Zhengzhou area WorldView image that 2009-12-27 gathers, image spatial resolution is 0.5 meter, size is 8740*11644, and image is divided into the sub-piece of 320*320 size according to the Tiles partitioned mode, constitutes the retrieval image storehouse of 1036 width of cloth sub-images.Because the remote sensing image area coverage is big, ground species complexity, feature according to atural object is presented on image can be divided into the cover type on the face of land following eight classes: farmland, open ground, road, intensive residential district, sparse residential district, square, viaduct, greenery patches.Therefore, embodiment has set eight class vision keywords: farmland, open ground, road, intensive residential district, sparse residential district, square, viaduct, greenery patches.
Step S02 according to the vision keyword of setting, finds out the single image of atural object content that can reflect these keywords from the image storehouse, be used as the training image.
Embodiment respectively the pure image blocks of selection and farmland, open ground, road, intensive residential district, sparse residential district, square, viaduct, greenery patches respective type as training sample.
Step S03 adopts feature extraction algorithm, to each width of cloth training image, extracts remarkable visual signature.
Remote sensing image mulched ground species is various, and single feature space is difficult to form the effective differentiation to atural object, and the embodiment of the invention selects to represent the significant point of local feature, dominant hue and the remarkable visual signature of textural characteristics three major types that object drives.During concrete enforcement, feature such as selected shape as required.
For the sake of ease of implementation, below the extraction of the remarkable visual signature of this three class is described respectively:
(1) significant point feature extraction: for remote sensing image, angle point is expression and the key character of analyzing image, the characteristics of image that extracts from remarkable neighborhood of a point can reflect the local message of image effectively, and when people pay close attention to a width of cloth image, often attracted by significant part in the image easily, wherein the part visual focus is the angle point in the image.The present invention adopts the SIFT image local feature to describe operator extraction significant point feature.The SIFT proper vector changes rotation, scale, brightness and remains unchanged, and can reduce the influence to the significant point feature extraction of different spatial resolutions, different illumination conditions as far as possible.
(2) dominant hue feature extraction: adopt Quick Shift partitioning algorithm, utilize Space Consistency and colour consistency that image is carried out over-segmentation, then the resulting subject area of over-segmentation is merged, the subject area after obtaining merging,
Directly corresponding to the three elements of human eye color vision feature, three Color Channels are independent separately for the HSV model, can extract the dominant hue of image according to the quantized result of its tone passage.The present invention at first is quantified as the tone passage
Individual subarea merges each subject area of extracting the back with above-mentioned over-segmentation and represents with the dominant hue histogram after quantizing respectively,
Be a certain zone of image after cutting apart, then object
The dominant hue eigenvector can be expressed as follows shown in the formula, wherein
Be
The frequency that the class tone occurs,
Get 1 to
(3) texture feature extraction: adopt Quick Shift partitioning algorithm that image is carried out over-segmentation, adopt the subject area merging method described in the dominant hue feature extracting method to carry out the merging of subject area then, adopt multiple dimensioned multi-direction textural characteristics describing method to carry out texture feature extraction at last.The present invention adopts the average of each yardstick high fdrequency component behind the wavelet transformation and variance as texture descriptor, the proper vector dimension of this describing method gained is low, efficient is high and have certain representativeness, the present invention has simultaneously carried out normalized to wavelet conversion coefficient
Be a certain zone of image after cutting apart, object
The textural characteristics vector representation be shown below,
Be
The normalization average value of individual component and variance,
Get 1 to
,
Be the component sum, equal three times of yardstick quantity.
During concrete enforcement, can for over-segmentation process setting qualifications, cut apart quality with raising when realizing (2) and (3), following three conditions are satisfied in the zone that for example limits after these merging:
A. the inner difference of subject area should be as far as possible little;
B. difference should be bigger between contiguous object around object and its;
C. the subject area area should be greater than a certain threshold value.
The purpose of condition a is that the qualification object is pure end member, improves the accuracy of semantic assignment; The degree that condition b controlling object zone merges; The fundamental purpose of condition c is to reject the trifling zone of disturbing vision to judge, outstanding significant principal character, the efficient of raising algorithm.Suppose
Be a certain zone of image after cutting apart, zone
The inside difference
Be defined as:
Wherein
,
Be feature weight, satisfy
Be the standard deviation of intra-zone color,
Be the region shape index, shown in being defined as follows:
Step S04 to all training images, adopts clustering algorithm to carry out cluster respectively all kinds of remarkable visual signature that obtains, and obtains the cluster centre that equates with vision keyword number, and each cluster centre is mapped as a critical mode; Adopt the arbitrary remarkable visual signature of gauss hybrid models match to belong to the probability density function of every class vision keyword, the gauss hybrid models parameter estimation is come the self-training image, approximating method adopts the expectation maximization method of estimation, thereby sets up vision key word hierarchy model.
Embodiment sets up the specific implementation process of vision key word hierarchy model: the remarkable visual signature of three classes by width of cloth training image can extract comprises significant point, dominant hue and texture.For all proper vectors of the remarkable visual signature of each class, adopt K average or ISODATA clustering method to carry out cluster, obtain and the consistent cluster centre of vision keyword number, each cluster centre is mapped as a critical mode.Adopt the arbitrary proper vector of gauss hybrid models match to belong to the probability density function of every class vision keyword, model parameter estimation is come the self-training image, and method adopts the expectation maximization method of estimation, thereby sets up vision key word hierarchy model.Because among the embodiment, generalized Gaussian distribution component (GGD) quantity that comprises in each classification GMM model is 8.The center of the subspace by training sample image feature space also is a critical mode, obtain the Gaussian distribution of each critical mode, the independence combination of a plurality of critical modes also is that the merging of Gaussian distribution constitutes a keyword that contains semanteme, the view picture shadow table is shown the distribution histogram of all kinds of semantic keywords in the image, so far can finish and not have semantic visual signature to the modeling process that contains semantic keyword label.
For the sake of ease of implementation, this step provides related description as follows:
Remote sensing image can be expressed as from pixel to local notable feature or the hierarchical model of primitive, destination object and scene, the visual vocabulary that all comprises a series of description visual informations on each level of model is expressed being connected of semantic label and characteristics of image in the scene thereby form image.If a certain video vision vocabulary is defined as set
, wherein
Be the vocabulary type identification,
Be arbitrary visual vocabulary element,
Be vocabulary type sum,
Be whole possible lexical space set.
The polymerization of visual vocabulary can produce the image of any yardstick, the some of them polymerization belongs to overall polymerization, and the combination of these vocabulary is reducible to go out most information in the image, in the literary composition these vocabulary integrated modes is referred to as critical mode, based on following formula, a certain critical mode
Be defined as:
The critical mode set needs to satisfy approximate condition for completeness promptly
, wherein
Be the critical mode sum,
Be image feature space.Therefore, vision keyword model is the modeling to critical mode.The automatic cluster algorithm is sought automatically at feature space and is widely used aspect the cluster centre, and the present invention promptly seeks critical mode by K average or ISODATA automatic cluster algorithm commonly used from numerous and jumbled visual vocabulary table; The enough parameterized methods of gauss hybrid models energy are described the DATA DISTRIBUTION in the sample space, the parameter of gauss hybrid models had the simple and high-efficient advantage as the feature of image, suppose visual vocabulary distribution Gaussian distributed in the feature space, then critical mode distribute to be obeyed Gaussian Mixture distribution GMMs, the semantic keyword of each classification promptly by
The cluster centre of individual lexical space is formed, with characteristic distribution
Be example, it belongs to keyword
Probability density function can be expressed as:
Wherein
Be the characteristic distribution space,
Be the dimension of GMM model,
Be the vision keyword
The GMM model parameter,
,
For in the model
The mixing constant of individual gaussian variable,
Be
The average of individual gaussian variable,
The transposition of representing matrix,
Be corresponding covariance matrix,
Be the sample dimension.Model parameter estimation is a training data with selected image of all categories, and method adopts the expectation maximization method of estimation, and the unique GMM in the vision keyword character pair space of each classification distributes.
Below be characterized as example with significant point, describe in detail to adopt the arbitrary proper vector of gauss hybrid models match to belong to the process of the probability density function of every class vision keyword: at first to adopt clustering algorithm to be with the SIFT proper vector clusters of 128 dimensions
Sub spaces, the center of each subspace are represented a critical mode, suppose each SIFT feature critical mode Gaussian distributed
, then can utilize the GMM model fitting
The semantic keyword of a SIFT type is expressed in the combination of individual SIFT feature critical mode.
Suppose that the semantic keyword number of image is
, then train the back image
SIFT type vision key word hierarchy model
Be expressed as shown in the following formula:
Wherein
Be
The gauss hybrid models coefficient of individual critical mode correspondence,
For
Belong to
The probability distribution of individual keyword is a Gaussian distribution,
Be
Individual critical mode characteristic of correspondence vector,
The
The covariance matrix of sub spaces.Calculate the significant point descriptor that is extracted
Belong to
The posterior probability of individual keyword
, adopt maximum a posteriori probability sorter MAP (maximum a posteriori) that SIFT is described vector
Be labeled as
Keyword, wherein:
If every width of cloth image is regarded as " text " formed by the plurality of keywords in the crucial dictionary of vision, then the classical statistics method TF-IDF in the text retrieval technology (term frequent-Inverse document frequency) is entry frequency-arrange a text frequency, can be in order to the significance level of assessment words for certain part of file in a file set or the corpus.The number of times that the importance of words occurs hereof with it increase that is directly proportional, but the decline that can be inversely proportional to along with the frequency that it occurs in corpus simultaneously.The present invention is statistic unit with a single point, image
SIFT eigenvector based on the vision keyword can be expressed as follows shown in the formula:
The fit procedure of dominant hue and texture and the process of significant point are that similarly the present invention will not give unnecessary details.Through over-fitting, promptly set up vision key word hierarchy model, realized that the low layer visual signature is to the contact between the high-level semantic keyword.
Step S05 adopts feature extraction algorithm, and each width of cloth image in the image storehouse all extracts remarkable visual signature.The remarkable visual signature that embodiment extracts comprises significant point, dominant hue and texture, obtains the proper vector of a series of description image features, promptly corresponding significant point proper vector, dominant hue proper vector and texture feature vector.Specific implementation process gets final product with step S03 is consistent.
Step S06, according to the vision key word hierarchy model of setting up among the step S04, in the eigenvector substitution vision key word hierarchy model with three class visual signatures of each width of cloth image in the image storehouse, calculate the probability that all proper vectors in every class visual signature belong to the vision keyword, press
Principle proper vector is mapped as corresponding vision keyword, thereby set up the proper vector of all visual signatures and the corresponding relation of vision keyword.
Step S07, at each width of cloth image in the image storehouse, according to default setting weight parameter, add up the frequency that every class vision keyword occurs in this image, and then make up the vision keyword feature vector of describing this image semanteme, thereby realize the semantic modeling and the description of image based on the vision keyword.During concrete enforcement, adopt different weight parameter, the result for retrieval that obtains is different, can rule of thumb be set in advance by those skilled in the art.
Sum up the specific implementation process of the semantic modeling of remote sensing image among the embodiment: at step S05, all images in the image storehouse all according to top step S03, are extracted remarkable visual signature earlier, comprise significant point, dominant hue and texture; Then at step S06, under the support of vision key word hierarchy model, according to
Principle, the single eigenvector that every width of cloth image extracts all can be mapped as the keyword of limited number,
,
Number for unique point in the image or object.But the contribution rate of each keyword sign image is not duplicate, with the section object is example, in general image center zone or the contribution of image interpretation is greater than the less zone of corner area than large area region, the embodiment of the invention is in step S07, unique point is adopted even weight, the weight that is each some feature critical mode is all identical, adopt area factor as weight parameter to regional keyword (dominant hue, texture), then can count the frequency that each keyword occurs in image, with single category feature is example, image
The keyword modeling after eigenvector
Can be expressed as
Step S08 is provided with the weight of three class vision keyword feature vectors, is the similarity measurement mode with the first approximation distance of KL divergence, by the similarity of all images in vision keyword feature vector calculation image to be retrieved and the image storehouse.When concrete true, also can adopt other prior aries, for example cosine distance, KL distance, the first approximation distance of KL divergence, Euclidean distance, mahalanobis distance as the similarity measurement criterion.
Among the embodiment based on the specific implementation process of the video search of similarity criterion: adopt feature extraction algorithm, three types keyword be can obtain in the image and significant point, dominant hue and the textural characteristics of image described respectively, the TF-IDF describing method in the text representation method is used for reference in the description of keyword, therefore, every width of cloth image can be described as the eigenvector of a certain class or a few class vision keywords
Analyze from information-theoretical angle, the expressed implication of the eigenvector that extracts is the probability distribution relation that the vision keyword occurs in the width of cloth image, supposes between each vision keyword separately, and obeys probability density function
Distribution, then the distance between two width of cloth images also can be expressed as the Kullback-Leibler divergence, is shown below:
Subscript wherein
With
Difference expression two width of cloth images.
Between the vision keyword feature vector
The divergence computing formula is:
Subscript wherein
1 He
2 difference expressions, two width of cloth images,
Be the keyword sum,
Be the keyword sequence number, following formula comprises logarithm operation, and counting yield is low, selects its single order approximate distance
Can effectively reduce complexity, as follows:
One width of cloth image can adopt significant point, dominant hue, texture three category feature vectors to express, and the semantic similarity size that distributes of image adopts the Weighted distance of three category features to calculate, and is as follows:
Wherein
,
Represent significant point, dominant hue, texture respectively,
Distinctive mark two width of cloth images.Keyword feature vector expression mode is similar to the image histogram among the present invention, just its input is the semantic keyword of different characteristic type, rather than the gray-scale value of image, this expression way and image size are irrelevant, only need to set identical semantic dimension, the image of different sizes all can adopt following formula to carry out similarity measurement.
Step S09 sorts similarity according to from high to low order, the output result for retrieval.
In semantic keyword feature vector that above step S08 calculates the retrieval image and image storehouse, after the similarity of the semantic keyword feature vector of all images, export image as a result, be result for retrieval according to similarity order from high to low.
Technical solution of the present invention effect for convenience of explanation, adopt the average precision ratio of 8 class images of different weights, the situation that contrasts single textural characteristics, single dominant hue feature, single significant point feature and the used comprehensive characteristics of the embodiment of the invention is retrieved respectively, then the gained result is compared evaluation, as shown in Figure 2.Quantitative evaluation is taked average precision ratio, contains the ratio of similar image in preceding 16 width of cloth images that promptly return.The similarity measurement criterion has adopted the first approximation distance of KL divergence, and the weight of significant point, dominant hue, main texture three class vision keywords is set at 0.5,0.25,0.25 respectively in the comprehensive characteristics retrieval.As seen, the accuracy rate of technical solution of the present invention is higher.
Above content is in conjunction with optimum implementation the present invention to be said further describing of doing, and can not assert that concrete enforcement of the present invention is only limited to these explanations.It should be appreciated by those skilled in the art, do not breaking away under the situation about limiting, can carry out various modifications in detail, all should be considered as belonging to protection scope of the present invention by appended claims.
Claims (6)
1. remote sensing image semantic retrieving method based on the vision keyword is characterized in that may further comprise the steps:
Step 1, setting can be described the vision keyword of image type in the image storehouse, and select some width of cloth images that can reflect each vision keyword from the image storehouse respectively, as the training image;
Step 2 is extracted all kinds of remarkable visual signature that all train images;
Step 3 to all training images, adopts clustering algorithm to carry out cluster respectively all kinds of remarkable visual signature that obtains, and obtains the cluster centre that equates with vision keyword number, and each cluster centre is mapped as a critical mode; Adopt the arbitrary remarkable visual signature of gauss hybrid models match to belong to the probability density function of every class vision keyword, the gauss hybrid models parameter estimation is come the self-training image, approximating method adopts the expectation maximization method of estimation, thereby sets up vision key word hierarchy model;
Step 4 adopts the mode consistent with step 2 to extract all kinds of remarkable visual signature of all images in the image storehouse respectively;
Step 5, at each width of cloth image in the image storehouse, calculate the probability that remarkable visual signature belongs to every class vision keyword by step 3 gained probability density function, if belong to the probability maximum of certain class vision keyword, think that then remarkable visual signature belongs to such vision keyword, thereby realize of the mapping of remarkable visual signature to the vision keyword;
Step 6 at each width of cloth image in the image storehouse, according to default setting weight parameter, is added up the frequency that every class vision keyword occurs in this image, and then makes up the vision keyword feature vector of describing this image semanteme;
Step 7 adopts default similarity measurement criterion, by the similarity of all images in vision keyword feature vector calculation image to be retrieved and the image storehouse, result for retrieval is sorted from high to low according to similarity and exports.
2. according to the described remote sensing image semantic retrieving method based on the vision keyword of claim 1, it is characterized in that: in step 2 and the step 4, the remarkable visual signature of extraction comprises dominant hue and the texture that significant point, object drive.
3. according to the described remote sensing image semantic retrieving method based on the vision keyword of claim 2, it is characterized in that: the implementation of extracting remarkable visual signature is as follows,
(1) utilize the SIFT image local feature to describe the significant point of all training images of operator extraction, thereby obtain the significant point feature of image, each significant point is represented with one 128 dimensional feature vector;
(2) all training images are carried out over-segmentation based on Quick Shift algorithm, the over-segmentation result is carried out the zone to be merged, then the consistance subject area is adopted the HSV model, quantized result according to its tone passage extracts each regional dominant hue, thereby obtain the dominant hue feature of image, the dominant hue feature of each subject area is represented with a proper vector;
(3) all training images are carried out over-segmentation based on Quick Shift algorithm, the over-segmentation result is carried out the zone to be merged, then the consistance subject area is adopted wavelet transformation, obtain the average of each yardstick high fdrequency component and variance as texture descriptor, thereby obtain the textural characteristics of image, the textural characteristics of each subject area is represented with a proper vector.
4. according to the described remote sensing image semantic retrieving method of claim 3 based on the vision keyword, it is characterized in that: in the step 6, when weight parameter was set, the significant point feature was composed with the average weight, and dominant hue feature and textural characteristics are weight with the area of self subject area.
5. according to claim 1 or 2 or 3 or 4 described remote sensing image semantic retrieving methods based on the vision keyword, it is characterized in that: in the step 3, the clustering algorithm that is adopted is K average or ISODATA algorithm.
6. according to claim 1 or 2 or 3 or 4 described remote sensing image semantic retrieving methods based on the vision keyword, it is characterized in that: in the step 7, default similarity measurement criterion is the first approximation distance of KL divergence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011100546248A CN102073748B (en) | 2011-03-08 | 2011-03-08 | Visual keyword based remote sensing image semantic searching method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011100546248A CN102073748B (en) | 2011-03-08 | 2011-03-08 | Visual keyword based remote sensing image semantic searching method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102073748A true CN102073748A (en) | 2011-05-25 |
| CN102073748B CN102073748B (en) | 2012-07-25 |
Family
ID=44032287
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011100546248A Expired - Fee Related CN102073748B (en) | 2011-03-08 | 2011-03-08 | Visual keyword based remote sensing image semantic searching method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102073748B (en) |
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103020172A (en) * | 2012-11-28 | 2013-04-03 | 北京京东世纪贸易有限公司 | Method and device utilizing video information to search articles |
| CN103177105A (en) * | 2013-03-26 | 2013-06-26 | 新浪网技术(中国)有限公司 | Method and device of image search |
| CN103823843A (en) * | 2014-01-24 | 2014-05-28 | 北京理工大学 | Gauss mixture model tree and incremental clustering method thereof |
| CN104834757A (en) * | 2015-06-05 | 2015-08-12 | 昆山国显光电有限公司 | Image semantic retrieval method and system |
| CN105224619A (en) * | 2015-09-18 | 2016-01-06 | 中国科学院计算技术研究所 | A kind of spatial relationship matching process and system being applicable to video/image local feature |
| CN105243154A (en) * | 2015-10-27 | 2016-01-13 | 武汉大学 | Remote sensing image retrieval method and system based on significant point characteristics and spare self-encodings |
| CN105608673A (en) * | 2015-12-16 | 2016-05-25 | 清华大学 | Image color quantization and jittering method and system |
| CN106021250A (en) * | 2015-09-16 | 2016-10-12 | 展视网（北京）科技有限公司 | Image semantic information retrieval method based on keyword |
| CN106570125A (en) * | 2016-11-02 | 2017-04-19 | 中国科学院深圳先进技术研究院 | Remote sensing image retrieval method and remote sensing image retrieval device based on rotation/zooming/translation invariance |
| CN106570136A (en) * | 2016-11-02 | 2017-04-19 | 中国科学院深圳先进技术研究院 | Remote-sensing image semantic retrieval method and device based on pixel-level association rules |
| CN106777094A (en) * | 2016-12-14 | 2017-05-31 | 大连大学 | The medical science big data searching system of the Skyline that view-based access control model vocabulary is matched with multiple features |
| CN106777090A (en) * | 2016-12-14 | 2017-05-31 | 大连交通大学 | The medical science big data search method of the Skyline that view-based access control model vocabulary is matched with multiple features |
| CN107291855A (en) * | 2017-06-09 | 2017-10-24 | 中国电子科技集团公司第五十四研究所 | A kind of image search method and system based on notable object |
| CN107423695A (en) * | 2017-07-13 | 2017-12-01 | 苏州珂锐铁电气科技有限公司 | Dynamic texture identification method based on bipartite graph |
| CN107862238A (en) * | 2016-12-26 | 2018-03-30 | 北京理工雷科电子信息技术有限公司 | A kind of in-orbit aircraft candidate regions screening technique based on local grain density and divergence |
| CN107909561A (en) * | 2017-11-15 | 2018-04-13 | 浙江大学宁波理工学院 | A kind of optimal transmission color of image moving method based on right-value optimization |
| CN108230413A (en) * | 2018-01-23 | 2018-06-29 | 北京市商汤科技开发有限公司 | Image Description Methods and device, electronic equipment, computer storage media, program |
| CN108446616A (en) * | 2018-03-09 | 2018-08-24 | 西安电子科技大学 | Method for extracting roads based on full convolutional neural networks integrated study |
| WO2018166273A1 (en) * | 2017-03-17 | 2018-09-20 | 北京京东尚科信息技术有限公司 | Method and apparatus for matching high-dimensional image feature |
| CN110956213A (en) * | 2019-11-29 | 2020-04-03 | 珠海大横琴科技发展有限公司 | Method and device for generating remote sensing image feature library and method and device for retrieving remote sensing image |
| CN111291319A (en) * | 2020-03-24 | 2020-06-16 | 广东海洋大学深圳研究院 | Mobile robot state estimation method applied to non-Gaussian noise environment |
| WO2021042763A1 (en) * | 2019-09-03 | 2021-03-11 | Guangdong Oppo Mobile Telecommunications Corp., Ltd. | Image searches based on word vectors and image vectors |
| CN112836069A (en) * | 2021-04-22 | 2021-05-25 | 泰德网聚(北京)科技股份有限公司 | Self-recognition retrieval system based on image keywords |
| CN113139378A (en) * | 2021-03-18 | 2021-07-20 | 杭州电子科技大学 | Image description method based on visual embedding and condition normalization |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101241504A (en) * | 2008-01-23 | 2008-08-13 | 武汉大学 | Remote sense image data intelligent search method based on content |
| CN101315663A (en) * | 2008-06-25 | 2008-12-03 | 中国人民解放军国防科学技术大学 | Nature scene image classification method based on area dormant semantic characteristic |
| CN101692224A (en) * | 2009-07-08 | 2010-04-07 | 南京师范大学 | High-resolution remote sensing image search method fused with spatial relation semantics |
-
2011
- 2011-03-08 CN CN2011100546248A patent/CN102073748B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101241504A (en) * | 2008-01-23 | 2008-08-13 | 武汉大学 | Remote sense image data intelligent search method based on content |
| CN101315663A (en) * | 2008-06-25 | 2008-12-03 | 中国人民解放军国防科学技术大学 | Nature scene image classification method based on area dormant semantic characteristic |
| CN101692224A (en) * | 2009-07-08 | 2010-04-07 | 南京师范大学 | High-resolution remote sensing image search method fused with spatial relation semantics |
Cited By (41)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103020172B (en) * | 2012-11-28 | 2015-08-19 | 北京京东世纪贸易有限公司 | A kind of method and apparatus utilizing video information to search for article |
| CN103020172A (en) * | 2012-11-28 | 2013-04-03 | 北京京东世纪贸易有限公司 | Method and device utilizing video information to search articles |
| CN103177105A (en) * | 2013-03-26 | 2013-06-26 | 新浪网技术(中国)有限公司 | Method and device of image search |
| CN103177105B (en) * | 2013-03-26 | 2016-12-28 | 新浪网技术(中国)有限公司 | A kind of image search method and device |
| CN103823843B (en) * | 2014-01-24 | 2017-02-22 | 北京理工大学 | Gauss mixture model tree and incremental clustering method thereof |
| CN103823843A (en) * | 2014-01-24 | 2014-05-28 | 北京理工大学 | Gauss mixture model tree and incremental clustering method thereof |
| CN104834757A (en) * | 2015-06-05 | 2015-08-12 | 昆山国显光电有限公司 | Image semantic retrieval method and system |
| CN106021250A (en) * | 2015-09-16 | 2016-10-12 | 展视网（北京）科技有限公司 | Image semantic information retrieval method based on keyword |
| CN105224619A (en) * | 2015-09-18 | 2016-01-06 | 中国科学院计算技术研究所 | A kind of spatial relationship matching process and system being applicable to video/image local feature |
| CN105224619B (en) * | 2015-09-18 | 2018-06-05 | 中国科学院计算技术研究所 | A kind of spatial relationship matching process and system suitable for video/image local feature |
| CN105243154A (en) * | 2015-10-27 | 2016-01-13 | 武汉大学 | Remote sensing image retrieval method and system based on significant point characteristics and spare self-encodings |
| CN105243154B (en) * | 2015-10-27 | 2018-08-21 | 武汉大学 | Remote sensing image retrieval method based on notable point feature and sparse own coding and system |
| CN105608673A (en) * | 2015-12-16 | 2016-05-25 | 清华大学 | Image color quantization and jittering method and system |
| CN105608673B (en) * | 2015-12-16 | 2020-09-25 | 清华大学 | Image color quantization and dithering method and system |
| CN106570125A (en) * | 2016-11-02 | 2017-04-19 | 中国科学院深圳先进技术研究院 | Remote sensing image retrieval method and remote sensing image retrieval device based on rotation/zooming/translation invariance |
| CN106570125B (en) * | 2016-11-02 | 2020-08-25 | 中国科学院深圳先进技术研究院 | Remote sensing image retrieval method and device for rotational scaling and translation invariance |
| CN106570136B (en) * | 2016-11-02 | 2019-10-29 | 中国科学院深圳先进技术研究院 | A kind of remote sensing images semantic retrieving method and device based on Pixel-level correlation rule |
| CN106570136A (en) * | 2016-11-02 | 2017-04-19 | 中国科学院深圳先进技术研究院 | Remote-sensing image semantic retrieval method and device based on pixel-level association rules |
| CN106777090A (en) * | 2016-12-14 | 2017-05-31 | 大连交通大学 | The medical science big data search method of the Skyline that view-based access control model vocabulary is matched with multiple features |
| CN106777094A (en) * | 2016-12-14 | 2017-05-31 | 大连大学 | The medical science big data searching system of the Skyline that view-based access control model vocabulary is matched with multiple features |
| CN107862238B (en) * | 2016-12-26 | 2021-05-14 | 北京理工雷科电子信息技术有限公司 | On-orbit aircraft candidate area screening method based on local texture density and divergence |
| CN107862238A (en) * | 2016-12-26 | 2018-03-30 | 北京理工雷科电子信息技术有限公司 | A kind of in-orbit aircraft candidate regions screening technique based on local grain density and divergence |
| US11210555B2 (en) | 2017-03-17 | 2021-12-28 | Beijing Jingdong Shangke Information Technology Co., Ltd. | High-dimensional image feature matching method and device |
| WO2018166273A1 (en) * | 2017-03-17 | 2018-09-20 | 北京京东尚科信息技术有限公司 | Method and apparatus for matching high-dimensional image feature |
| CN107291855A (en) * | 2017-06-09 | 2017-10-24 | 中国电子科技集团公司第五十四研究所 | A kind of image search method and system based on notable object |
| CN107423695A (en) * | 2017-07-13 | 2017-12-01 | 苏州珂锐铁电气科技有限公司 | Dynamic texture identification method based on bipartite graph |
| CN107909561A (en) * | 2017-11-15 | 2018-04-13 | 浙江大学宁波理工学院 | A kind of optimal transmission color of image moving method based on right-value optimization |
| CN107909561B (en) * | 2017-11-15 | 2021-07-27 | 浙江大学宁波理工学院 | Optimal transmission image color migration method based on weight optimization |
| CN108230413A (en) * | 2018-01-23 | 2018-06-29 | 北京市商汤科技开发有限公司 | Image Description Methods and device, electronic equipment, computer storage media, program |
| CN108230413B (en) * | 2018-01-23 | 2021-07-06 | 北京市商汤科技开发有限公司 | Image description method and device, electronic equipment and computer storage medium |
| CN108446616A (en) * | 2018-03-09 | 2018-08-24 | 西安电子科技大学 | Method for extracting roads based on full convolutional neural networks integrated study |
| CN108446616B (en) * | 2018-03-09 | 2021-09-03 | 西安电子科技大学 | Road extraction method based on full convolution neural network ensemble learning |
| US11755641B2 (en) | 2019-09-03 | 2023-09-12 | Guangdong Oppo Mobile Telecommunications Corp., Ltd. | Image searches based on word vectors and image vectors |
| WO2021042763A1 (en) * | 2019-09-03 | 2021-03-11 | Guangdong Oppo Mobile Telecommunications Corp., Ltd. | Image searches based on word vectors and image vectors |
| CN110956213A (en) * | 2019-11-29 | 2020-04-03 | 珠海大横琴科技发展有限公司 | Method and device for generating remote sensing image feature library and method and device for retrieving remote sensing image |
| CN111291319A (en) * | 2020-03-24 | 2020-06-16 | 广东海洋大学深圳研究院 | Mobile robot state estimation method applied to non-Gaussian noise environment |
| CN111291319B (en) * | 2020-03-24 | 2023-07-07 | 广东海洋大学深圳研究院 | Mobile robot state estimation method applied to non-Gaussian noise environment |
| CN113139378A (en) * | 2021-03-18 | 2021-07-20 | 杭州电子科技大学 | Image description method based on visual embedding and condition normalization |
| CN113139378B (en) * | 2021-03-18 | 2022-02-18 | 杭州电子科技大学 | Image description method based on visual embedding and condition normalization |
| CN112836069A (en) * | 2021-04-22 | 2021-05-25 | 泰德网聚(北京)科技股份有限公司 | Self-recognition retrieval system based on image keywords |
| CN112836069B (en) * | 2021-04-22 | 2021-09-10 | 泰德网聚(北京)科技股份有限公司 | Self-recognition retrieval system based on image keywords |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102073748B (en) | 2012-07-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102073748B (en) | Visual keyword based remote sensing image semantic searching method | |
| CN102012939B (en) | Method for automatically tagging animation scenes for matching through comprehensively utilizing overall color feature and local invariant features | |
| Fauqueur et al. | Region-based image retrieval: Fast coarse segmentation and fine color description | |
| CN104376105B (en) | The Fusion Features system and method for image low-level visual feature and text description information in a kind of Social Media | |
| CN105844292B (en) | A kind of image scene mask method based on condition random field and secondary dictionary learning | |
| Tang et al. | SAR image content retrieval based on fuzzy similarity and relevance feedback | |
| Kim et al. | Color–texture segmentation using unsupervised graph cuts | |
| CN108647602B (en) | A kind of aerial remote sensing images scene classification method determined based on image complexity | |
| CN103186538A (en) | Image classification method, image classification device, image retrieval method and image retrieval device | |
| Zagoris et al. | Automatic image annotation and retrieval using the joint composite descriptor | |
| CN101789005A (en) | Image searching method based on region of interest (ROI) | |
| CN103377376A (en) | Method and system for image classification, and method and system for image retrieval | |
| Wang et al. | Remote-sensing image retrieval by combining image visual and semantic features | |
| Song et al. | Taking advantage of multi-regions-based diagonal texture structure descriptor for image retrieval | |
| Martinet et al. | A relational vector space model using an advanced weighting scheme for image retrieval | |
| Yan et al. | Adaptive fusion of color and spatial features for noise-robust retrieval of colored logo and trademark images | |
| CN103678552A (en) | Remote-sensing image retrieving method and system based on salient regional features | |
| Meng et al. | Merged region based image retrieval | |
| Guo et al. | Object discovery in high-resolution remote sensing images: a semantic perspective | |
| CN106570124B (en) | Remote sensing images semantic retrieving method and system based on object level correlation rule | |
| CN110162654A (en) | It is a kind of that image retrieval algorithm is surveyed based on fusion feature and showing for search result optimization | |
| Devesh et al. | Retrieval of monuments images through ACO optimization approach | |
| Kumar et al. | Automatic feature weight determination using indexing and pseudo-relevance feedback for multi-feature content-based image retrieval | |
| Yang et al. | Semantic categorization of digital home photo using photographic region templates | |
| Hao et al. | Color flag recognition based on HOG and color features in complex scene |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120725 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |