CN103324677A - Hierarchical fast image global positioning system (GPS) position estimation method - Google Patents

Hierarchical fast image global positioning system (GPS) position estimation method Download PDF

Info

Publication number
CN103324677A
CN103324677A CN201310199867XA CN201310199867A CN103324677A CN 103324677 A CN103324677 A CN 103324677A CN 201310199867X A CN201310199867X A CN 201310199867XA CN 201310199867 A CN201310199867 A CN 201310199867A CN 103324677 A CN103324677 A CN 103324677A
Authority
CN
China
Prior art keywords
image
cluster
level
center
picture
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201310199867XA
Other languages
Chinese (zh)
Other versions
CN103324677B (en
Inventor
李婧
钱学明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian Jiaotong University
Original Assignee
Xian Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian Jiaotong University filed Critical Xian Jiaotong University
Priority to CN201310199867.XA priority Critical patent/CN103324677B/en
Publication of CN103324677A publication Critical patent/CN103324677A/en
Application granted granted Critical
Publication of CN103324677B publication Critical patent/CN103324677B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Abstract

The invention discloses a hierarchical fast image global positioning system (GPS) position estimation method which includes: performing global feature clustering on images according to color texture global features for an on-line database, distributing the images obtained through clustering to second-stage small centers according to the shooting geological positions, and finally performing representative image selection on each clustering center. In a line system, a first-stage candidate class is selected according to the color texture features, the distance calculation is performed and a second candidate class is selected in the second-stage small centers contained in the first-stage candidate class according to the color texture features, and finally GPS position estimation is performed on eventually input images through a local feature confirmation method. By means of the hierarchical fast image GPS position estimation method, image retrieval speed can be effectively improved, and retrieval accuracy can be improved.

Description

A kind of gradable rapid image GPS location estimation method
Technical field
The invention belongs to the multi-media image processing technology field, relate to a kind of picture position method of estimation, especially a kind of gradable rapid image GPS location estimation method, the particularly interior location recognition that comprises scenery of image.
Background technology
Along with growth in the living standard, people begin to make earnest efforts tourism, and a large amount of pictures of shooting upload to picture sharing website and social networks in tourism process.Well-known picture sharing website such as Flickr, the picture sum of uploading reaches 5,000,000,000.The picture amount of uploading in the social networks is more surprising, and light Facebook one family has just reached 60,000,000,000.In China, everybody net of social network sites, happy net becomes the main approach of uploading with sharing.Meanwhile, people also collect the various pictures at various sight spots more and more easily.Perhaps people can meet the image that oneself enjoys a lot, but do not know that but these images are to clap somewhere.Therefore, for so extensive image multi-medium data, spot for photography how to effectively utilize the scenery that comprises in the automatic recognition image of multi-medium data help people of these huge sizes is necessary.
Some labels or review information that present image recognition generally all will comprise by image are classified and identification to image according to label and review information.Text-based image retrieval is also day by day ripe, as Google, Baidu etc., under the prerequisite of given input keyword, can all search out a series of image that has keyword label.But because there is noise in subsidiary label and the description of image, in addition, these incidental informations itself are that people add up, have some subjectivities unavoidably.So the identification that utilizes picture material to carry out image retrieval and image is necessary, on the other hand, the extensive image multimedia resource on the network also provides advantageous conditions for the identification of automatically carrying out image by picture material.
Along with the progress of science and technology, people's smart mobile phone and part digital camera have time and GPS writing function when taking pictures, and when uploading the picture that does not carry GPS information, Flickr provides the user can manually drag the energy that uploads images onto its relevant position.Therefore, be to obtain a large amount of images that has the geographic position label by the network multimedia means.This just estimates to provide condition to automatic image geographic position.
Summary of the invention
The object of the present invention is to provide a kind of gradable rapid image GPS location estimation method, this method is utilized two subsystems: off-line subsystem and online subsystem.Wherein the off-line subsystem is mainly to having the processing in marking image storehouse, geographic position on a large scale.And for online subsystem, be query image to be carried out feature describe, use global characteristics that the image class in all off-line picture libraries is screened, choose the candidate image group and use local feature to carry out characteristic quantification according to global characteristics afterwards and obtain visual vocabulary, at last in conjunction with the inverted index table of the representative picture library of each image class, according to distance, the result is sorted at last, and use the K-NN method that the image geographic position is estimated.In this process, used the inverted index structure to reach the purpose of rapid image retrieval, can accelerate finishing of image GPS location estimation by image retrieval fast.This method can improve the retrieval rate of image, improves retrieval precision.
The objective of the invention is to solve by the following technical programs:
This gradable rapid image GPS location estimation method may further comprise the steps:
(1) processing in offline image storehouse
1) image library pre-service
The mean flow rate of computed image and HWVP texture energy according to the mean picture brightness of calculating, are removed 1% the highest and 1% minimum image respectively; According to the HWVP texture energy of calculating, remove 1% the highest and 1% minimum image respectively;
2) image characteristics extraction
Extract the global characteristics in the image: 45-D color moment feature, 170-D HWVP feature also has local SIFT feature; Described 45-D color moment feature refers to image averaging is divided into four parts, and overlapping extraction picture centre zone uses nine dimension color moments to be described to five zones then again;
3) global characteristics cluster
According to global characteristics, adopt the K-means clustering method that all images in the image library is clustered into 32 first order clusters;
4) according to the geographic position refinement
Each class in 32 first order clusters that previous step is obtained is segmented according to the geographic position, obtains new second level cluster; Picture is taken in identical geographic position in described each second level cluster;
5) representative picture is chosen
Each second level cluster is carried out representative picture choose, obtain the presentation graphics group of each second level cluster;
6) set up file index structure fast
Use visual vocabulary bag model to be described to the presentation graphics group, and make up the inverted index structure of visual vocabulary and presentation graphics group;
(2) estimate in the line image geographic position
1) first order center is chosen
At first the global characteristics of input picture and each center of first order cluster are compared, nearest M the center of chosen distance is as the candidate center; Wherein comparison obtains by calculating distance, and distance calculating method is as follows,
D i=||LC i-L input||,(i=1,…,R),
L wherein InputThe 215 dimension global characteristics that the 45-D color moment feature of expression input picture and 170-D HWVP feature constitute, D iExpression input picture and i class C iGlobal characteristics center LC iDistance; || X|| is used for representing the norm of X; R=32;
Select M nearest first order cluster as candidate center S={S according to calculated distance 1..., S M, M≤R wherein;
2) center, the second level is selected
Selecting M first order cluster candidate center S={S 1..., S MAfterwards, further carry out the selection at meticulousr cluster candidate center, the second level; Use s={r 1..., r NRepresent selected S={S 1..., S MIn all second level cluster centres, r wherein i∈ { c J, k, j=1 ..., R; K=1 ..., N j V% is as the candidate center of selecting, the second level before selecting in all N second level cluster centre then, and V is the real number of 0-100; System of selection is identical with first order cluster candidate center, and computing method are as follows:
d i=||Lr i-L input||,i∈{1,…,N}
Wherein N is the number of the second level cluster centre that comprises below all first order cluster candidate centers, Lr iThe global characteristics that represents cluster candidate center, i the second level is described;
By all cluster candidate centers, the second level that this step obtains, note is SC={g 1..., g F, g wherein f∈ { r 1..., r N, wherein f ∈ l ..., F}, F=V * N/100;
3) use local feature to improve
Use BoW histogram or inverted index structure that the cluster candidate center of choosing, the second level is confirmed; Again according to confirming that the result who obtains uses the method for KNN that the geographic position of input picture is recommended.
Further, when using the BoW histogram that the cluster candidate center of choosing, the second level is confirmed to be taken in, carry out according to following:
In off-line system, after the BoW model description, generate the BoW histogram; At this, input picture is also generated the BoW histogram of corresponding dimension, and be expressed as h (k), k=1 ..., Q; Use four kinds of histogram distance metric modes to carry out distance metric, comprise that cosine distance C OS, city are apart from MAD, Euclidean distance MSD and histogram intersection HIST; Its distance calculating method is as follows respectively:
COS ( i , j ) = Σ k = 1 Q N H i , j ( k ) × h ( k ) [ Σ k = 1 Q ( NH i , j ( k ) ) 2 ] * [ Σ k = 1 Q ( h ( k ) ) 2 ] - - - ( 6 )
MAD ( i , j ) = Σ k = 1 Q | N H i , j ( k ) - h ( k ) | - - - ( 7 )
MSD ( i , j ) = Σ k = 1 Q ( NH i , j ( k ) - h ( k ) ) 2 - - - ( 8 )
HIST ( i , j ) = Σ k = 1 Q min ( NH i , j ( k ) , h ( k ) ) - - - ( 9 )
Wherein, NH is the BoW histogram description of presentation graphics group.
Further, when using the inverted index structure that the cluster candidate center of choosing, the second level is confirmed to be taken in, carry out according to following:
Use TF-IDF to calculate the method for weights, each image sets is calculated its corresponding score, the score computing method are as follows,
Score ( L ) = Σ x = 1 Q ω x * Freq L ( x ) Number L * Frequen x - - - ( 10 )
Wherein FreqL (x) is the frequency of x vocabulary, and NumberL is the number of all visual vocabularies in the image, and Frequenx is the number of times that vocabulary occurs in the entire image storehouse, ω xExpression be weight for input picture vocabulary; Weighing computation method is as follows,
ω x = Freq input ( x ) Number input - - - ( 11 )
Wherein the frequency and Numberinput input picture of Freqinput (x) expression BoW#x comprise the sum of visual vocabulary.
Compared with prior art, the invention has the advantages that:
1) but gradable rapid image GPS location estimation method of the present invention uses visual signature and geographical location information that large-scale image library has been made up the structure of layering, can effectively improve the retrieval rate of image;
2) the present invention proposes the improved GPS of hierarchical cluster and local feature geographic position method of estimation, by in conjunction with local feature, can effectively improve retrieval precision;
3) the present invention has quoted the inverted index structure and has accelerated retrieval rate and improve retrieval accuracy in the use of local feature.
Description of drawings
Fig. 1 is the general steps schematic block diagram of the inventive method;
Fig. 2 is the structural drawing of the inverted index table when adopting inverted index mechanism in the inventive method.
Embodiment
Below in conjunction with accompanying drawing the present invention is done and to describe in further detail:
Referring to Fig. 1: gradable rapid image GPS location estimation method of the present invention may further comprise the steps:
(1) processing in offline image storehouse
The purpose of off-line subsystem mainly is that the image library of extensive band geographical labels is carried out processed offline, can reach better application and estimate in online image geographic position, carries out the purpose that GPS estimates fast and effectively to reach.The off-line subsystem mainly comprises following six aspects: the 1) pre-service of image library, 2) to the overall situation of image in the image library and the description of local feature, 3) use global characteristics that image library kind image is carried out the K-means cluster; 4) class that the cluster of the first order is obtained according to the geographic position is segmented and is obtained partial image sets, 5) image sets that the 4th step was obtained carries out the selection of presentation graphics, 6) presentation graphics is set up upright inverted index table.Respectively these six parts are introduced below.
1) image library pre-service
The pretreated purpose of image library is to remove some noise images.Because in the process that image library makes up, retrieve in image library according to keyword and to download then.So just cause very heterogeneity of the picture quality that downloads to, it is high/low especially to have a lot of brightness of image, perhaps comprise many especially noises, these images estimate it is not effect for finishing GPS then, or even have counteractive, so just carried out preliminary pre-service earlier among the present invention, be intended to remove the relatively poor image of those mass ratioes.According to the mean picture brightness of calculating, remove the high and minimum image of Alpha% respectively, effect is best when finding that by the back experiment Alpha gets 1.For the removal that comprises many textures of noise complex image, what use among the present invention is the HWVP feature, has calculated texture energy, carries out the removal of noise then.By pre-service, guaranteed that the image that residue participates in using in the offline image storehouse is that some mass ratioes are schemed preferably.Therefore, in preferred plan of the present invention, this step is carried out according to following:
The mean flow rate of computed image and HWVP texture energy according to the mean picture brightness of calculating, are removed 1% the highest and 1% minimum image (be about to the mean picture brightness ordering, remove two head heights part and lower part respectively 1%) respectively; According to the HWVP texture energy of calculating, remove 1% the highest and 1% minimum image respectively;
2) image characteristics extraction
Extract the global characteristics in the image: 45-D color moment feature, 170-D HWVP feature also has local SIFT feature; Described 45-D color moment feature refers to image averaging is divided into four parts, and overlapping extraction picture centre zone uses nine dimension color moments to be described to five zones then again;
3) global characteristics cluster
Consider in the extensive image library image retrieval requirement to retrieval rate, propose to use the thought of the single image in the cluster centre alternative image storehouse among the present invention.Because compare picture number, middle calculation can lower a lot, and the time of required like this comparison has also just shortened.By the global characteristics cluster, the entire image storehouse just can be divided into the less image class of some relative scales, and each image class has similar global property.Why cluster is based on the consideration of lowering complexity.Also wish by the global characteristics cluster in addition, the image of taking in same place can be divided by different scenes, as the spring, summer, autumn and winter, daytime and night, terrestrial reference and non-landmark, classical architecture and modern architecture.
According to global characteristics, adopt the K-means clustering method that all images in the image library is clustered into 32 first order clusters;
Comprise color characteristic and textural characteristics for the feature of using in the global characteristics.What color characteristic used is the color moment of 45 dimensions, the HWVP feature of 170 dimensions that textural characteristics then is to use.Among the present invention two features are fused into the low-level features of one 215 dimension, images all in the image library is described respectively.By cluster, obtain R class C 1..., C R, and each class has a center LC i
4) according to the geographic position refinement
In this step, each class in 32 first order clusters that previous step is obtained is segmented according to the geographic position, obtains new second level cluster; Picture is taken in identical geographic position in described each second level cluster.Specific explanations is as follows:
Obtained the cluster r of the first order i∈ { c J, k, j=1 ..., R; K=1 ..., n; Afterwards, according to the spot for photography of image this R cluster is carried out meticulousr division.Suppose i center C iComprise below and take in N i(i=1 ..., the R) image at individual sight spot carries out the further segmentation according to the geographic position so, has obtained partial image clustering c I, j(i=1 ..., R; J=1 ..., N i).For each partial cluster, this paper has carried out the calculating at its global characteristics center, and uses Lc I, jExpression.Its computing formula is as follows,
Lc i , j = 1 n i , j Σ k = 1 n i , j L i , j , k , i=1,…,R;j=1,…,N i (1)
N wherein I, jBe c I, jThe middle number that comprises image, L I, j, kExpression c I, jIn k open image 215 the dimension the global characteristics vectors.Relation between second level picture number and the first order picture number is as follows,
z i = Σ j = 1 N i n i , j ; i=1,…,R;j=1,…,N i (2)
Therefore, in the image library total number of images be exactly in all first order intracardiac all images number add and, be expressed as follows:
Figure BDA00003246070500093
i=1,…,R;j=1,…,N i (3)
That is to say, by cluster with according to the GPS position, image library is divided, each image be assigned to the corresponding first order and partial in the heart, corresponding first order classification and second level classification have also just been arranged.
5) representative picture is chosen
Each second level cluster is carried out representative picture choose, obtain the presentation graphics group of each second level cluster.
Obtained second level cluster c I, jAfterwards, what can guarantee is that image in all classes all is to have similar global characteristics and take in similar place.Because the meticulous division at center, the second level is carried out according to the incidental label of image, and a lot of labels on the network all to be the handmarking comprise much noise, therefore, c I, jIn image can not guarantee it is to take in the place of institute's mark really.The another one aspect may have different shooting angle owing to take in the image at identical sight spot, also just causes having bigger difference.So only represent whole c by overall center I, jJust cause owing to got the hiding of internal diversity that product averages brings.Consider that local feature description's symbol can effectively extract the local message of image, can excavate the sight spot that comprises in the image, the present invention adopts the method for SIFT characteristic matching that each partial cluster is carried out choosing of presentation graphics.The specific algorithm of choosing is as follows, and this algorithm is to improve with reference to the way that the presentation graphics in people's articles such as Kennedy is selected to select.Algorithm 1 is the flow process that the presentation graphics that proposes of the present invention is selected:
Figure BDA00003246070500101
6) set up file index structure fast
Use visual vocabulary bag model to be described to the presentation graphics group, and make up the inverted index structure of visual vocabulary and presentation graphics group.
In image multimedia retrieval field, the descriptor of expression local visual feature or the vector in local feature zone just can be regarded visual vocabulary one by one as.Should have identical attribute such as the identical local feature point/zone that appears in the different images, and to describe the descriptor that identical local feature point faces territory or local feature zone should also be the same, so just we can say that corresponding two key frames of two images all have this visual vocabulary.The definition of analogy text inverted index so, local feature description's symbol that all images extracts in the image library just can be regarded the vocabulary one by one in the document as, and different images just can be regarded different documents as, inverted index based on visual vocabulary is exactly to be video name, the frame number of appearance even the coordinate position in this frame of each its appearance of visual vocabulary record in fact so.So for the foundation of the inverted index of visual vocabulary, most important is exactly that the visual vocabulary that quantity is unlimited is quantized on a limited number of visual vocabularies that can represent content, we also are referred to as to quantize visually.Just image can be used the BoW model indicate after quantizing.
Referring to Fig. 2, contrast traditional inverted list difference, the inverted list among the present invention is the corresponding relation of vocabulary between each presentation graphics group.What record in the inverted list is each visual vocabulary, with and the label of the presentation graphics group that occurred.
(2) estimate in the line image geographic position
Of the present inventionly estimate in online subsystem, to finish in the line image geographic position.The concrete module of online subsystem can be made of the online part among Fig. 1.Similar to the off-line subsystem, its each module also can be divided into feature extraction according to the difference that realizes function, and first order center is chosen, and center, the second level is chosen and local feature is confirmed three parts:
1) first order center is chosen
At first the global characteristics of input picture and each center of first order cluster are compared, nearest M the center of chosen distance is as the candidate center; Wherein comparison obtains by calculating distance, and distance calculating method is as follows,
D i=||LC i-L input||,(i=1,…,R),
L wherein InputThe 215 dimension global characteristics that the 45-D color moment feature of expression input picture and 170-D HWVP feature constitute, D iExpression input picture and i class C iGlobal characteristics center LC iDistance; || X|| is used for representing the norm of X; R=32;
So just can calculate input picture in the cluster at all first order centers, among the present invention the mode of all distances according to ascending order be sorted then.Because the distance between the image can be described visual similarity, distance is more near, shows that then similarity is more high above the vision.Select M nearest first order cluster as candidate center S={S according to calculated distance then 1..., S M, M≤R wherein; System of selection herein has two advantages, compares the distance of all images in calculating and the image library, and only Comparatively speaking the distance of computing center is just beaten and reduced calculated amount.And select M center but not a center is to consider that this step only is rough some candidate image collection of selection, and be not final meticulous selection, so the several clusters of multiselect are conducive to search out the image class of real representing input images.By the selection of the first step, obtained M first order candidate center, be designated as s={S 1..., S M.
2) center, the second level is chosen
Selecting M first order cluster candidate center S={S 1..., S MAfterwards, further carry out the selection at meticulousr cluster candidate center, the second level; Use s={r 1..., r NRepresent selected S={S 1..., S MIn all second level cluster centres, r wherein i∈ { c J, k, j=1 ..., R; K=1 ..., N j V% is as the candidate center of selecting, the second level before selecting in all N second level cluster centre then, and V is the real number of 0-100; System of selection is identical with first order cluster candidate center, and computing method are as follows:
d i=||Lr i-L input||,i∈{1,…,N}
Wherein N is the number of the second level cluster centre that comprises below all first order cluster candidate centers, Lr iThe global characteristics that represents cluster candidate center, i the second level is described;
By all cluster candidate centers, the second level that this step obtains, note is SC={g 1..., g F, g wherein f∈ { r 1..., r N, wherein f ∈ 1 ..., F}, F=V * N/100;
3) use local feature to improve
Above-mentioned what all consider in steps is the feature of the overall aspect of image, distinguishes for scene, and local features such as buildings identification have all embodied its validity always.Therefore, the present invention has done the local feature affirmation in this step, is intended to guarantee that the image sets of electing can represent input picture better, estimates purpose accurately to reach.In the similarity measurement of local feature, used the present invention to use two kinds of diverse ways, a kind of histogrammic similarity measurement of BoW that is based on, another is based on the similarity measurement of visual vocabulary inverted index structure.Comparison be similarity between the presentation graphics group that chooses in image and each off-line system, expectation is by selecting similar image sets, and the GPS positional information of use image sets is come the position, spot for photography of estimated image.
Based on the histogrammic similarity measurement of BoW
In off-line system, after the production yardage, we are described with the BoW histogram each presentation graphics group.At this, input picture is also generated the BoW histogram of corresponding dimension, and be expressed as h (k), k=1 ..., Q.Use four kinds of common histogram distance metric modes to carry out distance metric among the present invention, comprise cosine distance (being designated as COS), city distance (MAD), Euclidean distance (MSD) and histogram intersection (HIST).Its distance calculating method is as follows respectively,
COS ( i , j ) = Σ k = 1 Q N H i , j ( k ) × h ( k ) [ Σ k = 1 Q ( NH i , j ( k ) ) 2 ] * [ Σ k = 1 Q ( h ( k ) ) 2 ] - - - ( 6 )
MAD ( i , j ) = Σ k = 1 Q | N H i , j ( k ) - h ( k ) | - - - ( 7 )
MSD ( i , j ) = Σ k = 1 Q ( NH i , j ( k ) - h ( k ) ) 2 - - - ( 8 )
HIST ( i , j ) = Σ k = 1 Q min ( NH i , j ( k ) , h ( k ) ) - - - ( 9 )
Similarity measurement based on the inverted index structure
In being usually used in text retrieval, along with the continuous development of BoW in the image retrieval, inverted index had also obtained using widely the inverted index structure in the last few years.Used basic TF-IDF to calculate the method for weights among the present invention.Each image sets is calculated its corresponding score.The score computing method are as follows,
Score ( L ) = Σ x = 1 Q ω x * Freq L ( x ) Number L * Frequen x - - - ( 10 )
Wherein FreqL (x) is the frequency of x vocabulary, and NumberL is the number of all visual vocabularies in the image, and Frequenx is the number of times that vocabulary occurs in the entire image storehouse, ω xExpression be weight for input picture vocabulary.Weighing computation method is as follows,
ω x = Freq input ( x ) Number input - - - ( 11 )
Wherein the frequency and Numberinput input picture of Freqinput (x) expression BoW#x comprise the sum of visual vocabulary.
After more than using BoW histogram or inverted index structure that the cluster candidate center of choosing, the second level is confirmed; Again according to confirming that the result who obtains uses the method for KNN that the geographic position of input picture is recommended.
Experimental result and analysis
In order to verify the performance of proposition system, this paper has contrasted IM2GPS, the method for people's such as field space encoding (be designated as SC, comprise two kinds of 1-NN and K-NN), and the sorting technique (being designated as LC) based on the landmark thing of SVM of Li Dengren.
Performance Evaluation comprises two parts, and first is the crosscheck on GOLD storehouse and the GOLDEN storehouse, and second portion is accuracy rate and the time performance statistics of the image GPS location estimation of each image library inside.
Interpretational criteria
1) False Rate
In crosscheck, if the image in the GOLD storehouse is as input figure, the GOLDEN storehouse is as the words in offline image storehouse, the criterion of setting according to this paper should be to be judged as to estimate, if but system erroneous judgement can the GPS location estimation for image and has estimated wrong GPS position, so just think to judge by accident.Otherwise the image in the GOLDEN storehouse is done input picture, and when GOLD did the off-line storehouse, principle also was identical.Herein, ER represents False Rate, and with the picture number of FN representative generation erroneous judgement, TN represents the total number of images of the crosscheck that is useful on.The computing formula of False Rate is as follows:
ER = FN TN × 100 % - - - ( 3 - 12 )
2) accuracy rate of GPS estimation and time performance contrast
The accuracy rate of system is by choosing image as input from the offline image storehouse, utilize system to carry out the GPS location estimation, judges then with whether real position is consistent and carries out that accuracy rate measures.Computing formula is as follows:
AR = 1 G Σ i = 1 G A i - - - ( 3 - 13 )
In the formula:
The image GPS in AR---entire image storehouse estimates accuracy rate;
A i---the estimation accuracy rate at i sight spot;
The geographic position number of G---all tests.
Distribute at each sight spot and to calculate that formula is as follows accurately:
A i = N C i NA i × 100 % , i ∈ { 1 , . . . , G } - - - ( 3 - 14 )
In the formula:
NC i---the correct image number of estimating;
NA i---this geographic position goes out the sum of all test patterns.
Contrast with additive method
For the purpose of justice, in all control methodss, use all have only picture material.For method SC, 1-NN and K-NN contrast, and wherein to have chosen be that its performance reaches best value to K: 120.The scale of visual vocabulary is 60,000 in the LC method, and in the contrast, the basic setup parameter of the system of proposition is R=32, M=10, and K=50, V=100, simultaneously, the scale of visual vocabulary is 60,000.All participated in contrast based on histogrammic similarity measurement and based on the method for inverted index structure.Experimental result is shown in following table 3-1 and table 3-2.
Table 3-1SC (1-NN), SC (K-NN), IM2GPS, LC and this paper method be at image library COREL5000, the GPS accuracy of estimation (%) on OxBuild5000 and the GOLD
Figure BDA00003246070500171
The result shows that system performance in this paper obviously is better than additive method, not only on the GOLD storehouse, also shows on other two test library OxBuild5000 and the COREL5000 simultaneously.The mean accuracy of IM2GPS is 45.98%, 39.67%, 53.06% on three test data set.The mean accuracy of LC is 49.43%, 53.94%, 54.25%.SC (K-NN) is respectively 76.01%, 60.87% and 71.84% in the performance of three test sets, promotes to some extent than its corresponding SC (1-NN) performance.Our method is respectively 97%, 91% and 84.64% according to the COS result of measure on three data sets that give an example.MAD, MSD, our method is than IM2GPS under HIST and the IFS, and LC and SC are better.
Table 3-2.SC (1-NN), SC (K-NN), IM2GPS, LC and this paper method be at image library COREL5000, and the GPS on OxBuild5000 and the GOLD consumes (ms) estimated time
Figure BDA00003246070500172
Figure BDA00003246070500181
Found through experiments, effectively be conducive to image GPS location estimation in conjunction with global characteristics and local feature.Because IM2GPS has only utilized global characteristics, its AR is lower.Though SC utilizes local feature, really ignored global characteristics the effective information that can provide.Therefore, our method has embodied performance better.The lower reason of estimation accuracy rate of LC has two.One of them is that when the BoW that they use was histogrammic, the spatial information of local feature was uncared-for fully.In addition on the one hand, be that the influence that svm classifier is subjected to training set is very big, and generally speaking, can not guarantee that the image in the training set does not comprise noise image.
On three test sets, IM2GPS average computation cost is 60.46ms, 33.74ms, and 64927ms, and SC (K-NN) is 7.30ms, 5.51ms, 39.60ms, LC then are respectively 1.04ms, 1.34ms, 2.89ms.COS, MAD, MSD, HIST and IFS assess the cost and will be lower than SC, LC, and IM2GPS.For large-scale dataset, it is very obvious that the validity of IFS embodies, the average computation cost is 0.117ms, this is the time consuming 1.8 * 10-6% of IM2GPS, 0.25% SC (K-NN), 12.19% COS, 12.58% MAD, 11.36% MSD, and 11.82% HIST assesses the cost.

Claims (3)

1. a gradable rapid image GPS location estimation method is characterized in that, may further comprise the steps:
(1) processing in offline image storehouse
1) image library pre-service
The mean flow rate of computed image and HWVP texture energy according to the mean picture brightness of calculating, are removed 1% the highest and 1% minimum image respectively; According to the HWVP texture energy of calculating, remove 1% the highest and 1% minimum image respectively;
2) image characteristics extraction
Extract the global characteristics in the image: 45-D color moment feature, 170-D HWVP feature also has local SIFT feature; Described 45-D color moment feature refers to image averaging is divided into four parts, and overlapping extraction picture centre zone uses nine dimension color moments to be described to five zones then again;
3) global characteristics cluster
According to global characteristics, adopt the K-means clustering method that all images in the image library is clustered into 32 first order clusters;
4) according to the geographic position refinement
Each class in 32 first order clusters that previous step is obtained is segmented according to the geographic position, obtains new second level cluster; Picture is taken in identical geographic position in described each second level cluster;
5) representative picture is chosen
Each second level cluster is carried out representative picture choose, obtain the presentation graphics group of each second level cluster;
6) set up file index structure fast
Use visual vocabulary bag model to be described to the presentation graphics group, and make up the inverted index structure of visual vocabulary and presentation graphics group;
(2) estimate in the line image geographic position
1) first order center is chosen
At first the global characteristics of input picture and each center of first order cluster are compared, nearest M the center of chosen distance is as the candidate center; Wherein comparison obtains by calculating distance, and distance calculating method is as follows,
D i=||LC i-L input||,(i=1,…,R),
L wherein InputThe 215 dimension global characteristics that the 45-D color moment feature of expression input picture and 170-D HWVP feature constitute, D iExpression input picture and i class C iGlobal characteristics center LC iDistance; || X|| is used for representing the norm of X; R=32;
Select M nearest first order cluster as candidate center S={S according to calculated distance 1..., S M, M≤R wherein;
2) center, the second level is selected
Selecting M first order cluster candidate center S={S 1..., S MAfterwards, further carry out the selection at meticulousr cluster candidate center, the second level; Use s={r 1..., r NRepresent selected S={S 1..., S MIn all second level cluster centres, r wherein i∈ { c J, k, j=1 ..., R; K=1 ..., N j V% is as the candidate center of selecting, the second level before selecting in all N second level cluster centre then, and V is the real number of 0-100; System of selection is identical with first order cluster candidate center, and computing method are as follows:
d i=||Lr i-L input||,i∈{1,…,N}
Wherein N is the number of the second level cluster centre that comprises below all first order cluster candidate centers, Lr iThe global characteristics that represents cluster candidate center, i the second level is described;
By all cluster candidate centers, the second level that this step obtains, note is SC={g 1..., g F, g wherein f∈ { r 1..., r N, wherein f ∈ 1 ..., F}, F=V * N/100;
3) use local feature to improve
Use BoW histogram or inverted index structure that the cluster candidate center of choosing, the second level is confirmed; Again according to confirming that the result who obtains uses the method for KNN that the geographic position of input picture is recommended.
2. gradable rapid image GPS location estimation method according to claim 1 is characterized in that, when using the BoW histogram that the cluster candidate center of choosing, the second level is confirmed, carries out according to following:
In off-line system, after the BoW model description, generate the BoW histogram; At this, input picture is also generated the BoW histogram of corresponding dimension, and be expressed as h (k), k=1 ..., Q; Use four kinds of histogram distance metric modes to carry out distance metric, comprise that cosine distance C OS, city are apart from MAD, Euclidean distance MSD and histogram intersection HIST; Its distance calculating method is as follows respectively:
COS ( i , j ) = Σ k = 1 Q N H i , j ( k ) × h ( k ) [ Σ k = 1 Q ( NH i , j ( k ) ) 2 ] * [ Σ k = 1 Q ( h ( k ) ) 2 ] - - - ( 6 )
MAD ( i , j ) = Σ k = 1 Q | N H i , j ( k ) - h ( k ) | - - - ( 7 )
MSD ( i , j ) = Σ k = 1 Q ( NH i , j ( k ) - h ( k ) ) 2 - - - ( 8 )
HIST ( i , j ) = Σ k = 1 Q min ( NH i , j ( k ) , h ( k ) ) - - - ( 9 )
Wherein, NH is the BoW histogram description of presentation graphics group.
3. gradable rapid image GPS location estimation method according to claim 1 is characterized in that, when using the inverted index structure that the cluster candidate center of choosing, the second level is confirmed, carries out according to following:
Use TF-IDF to calculate the method for weights, each image sets is calculated its corresponding score, the score computing method are as follows,
Score ( L ) = Σ x = 1 Q ω x * Freq L ( x ) Number L * Frequen x - - - ( 10 )
Wherein FreqL (x) is the frequency of x vocabulary, and NumberL is the number of all visual vocabularies in the image, and Frequenx is the number of times that vocabulary occurs in the entire image storehouse, ω xExpression be weight for input picture vocabulary; Weighing computation method is as follows,
ω x = Freq input ( x ) Number input - - - ( 11 )
Wherein the frequency and Numberinput input picture of Freqinput (x) expression BoW#x comprise the sum of visual vocabulary.
CN201310199867.XA 2013-05-24 2013-05-24 Hierarchical fast image global positioning system (GPS) position estimation method Expired - Fee Related CN103324677B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310199867.XA CN103324677B (en) 2013-05-24 2013-05-24 Hierarchical fast image global positioning system (GPS) position estimation method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310199867.XA CN103324677B (en) 2013-05-24 2013-05-24 Hierarchical fast image global positioning system (GPS) position estimation method

Publications (2)

Publication Number Publication Date
CN103324677A true CN103324677A (en) 2013-09-25
CN103324677B CN103324677B (en) 2017-02-01

Family

ID=49193420

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310199867.XA Expired - Fee Related CN103324677B (en) 2013-05-24 2013-05-24 Hierarchical fast image global positioning system (GPS) position estimation method

Country Status (1)

Country Link
CN (1) CN103324677B (en)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104765764A (en) * 2015-02-06 2015-07-08 南京理工大学 Indexing method based on large-scale image
CN104778272A (en) * 2015-04-24 2015-07-15 西安交通大学 Method for estimating positions of images based on area mining and space encoding
CN104794219A (en) * 2015-04-28 2015-07-22 杭州电子科技大学 Scene retrieval method based on geographical position information
CN104820718A (en) * 2015-05-22 2015-08-05 哈尔滨工业大学 Image classification and searching method based on geographic position characteristics and overall situation vision characteristics
CN105183746A (en) * 2015-07-08 2015-12-23 西安交通大学 Method for realizing image retrieval by mining distinguishing features from multiple relevant pictures
CN105574536A (en) * 2014-10-31 2016-05-11 卡西欧计算机株式会社 Machine learning device, machine learning method, classification device, and classification method
CN106776849A (en) * 2016-11-28 2017-05-31 西安交通大学 A kind of method and guide system to scheme quick-searching sight spot
CN108399413A (en) * 2017-02-04 2018-08-14 清华大学 A kind of picture shooting region recognition and geographic positioning and device
CN108932273A (en) * 2017-05-27 2018-12-04 腾讯科技(深圳)有限公司 Picture screening technique and device
CN105095215B (en) * 2014-04-22 2019-05-10 富士通株式会社 Information acquisition device, method and server
CN110163982A (en) * 2019-04-11 2019-08-23 浙江大学 A kind of virtual fireworks analogy method of immersion based on Sketch Searching and controlled shape
CN112115901A (en) * 2020-09-25 2020-12-22 广州市派客朴食信息科技有限责任公司 High-accuracy food identification method

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101551823A (en) * 2009-04-20 2009-10-07 浙江师范大学 Comprehensive multi-feature image retrieval method
CN101702239A (en) * 2009-10-16 2010-05-05 西安交通大学 Method for describing texture of image with gradable wavelet packet transformation
CN101777059A (en) * 2009-12-16 2010-07-14 中国科学院自动化研究所 Method for extracting landmark scene abstract
CN102129451A (en) * 2011-02-17 2011-07-20 上海交通大学 Method for clustering data in image retrieval system
CN102341824A (en) * 2009-03-04 2012-02-01 公立大学法人大阪府立大学 Method of compiling image database, image database compilation program, image retrieval method

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102341824A (en) * 2009-03-04 2012-02-01 公立大学法人大阪府立大学 Method of compiling image database, image database compilation program, image retrieval method
CN101551823A (en) * 2009-04-20 2009-10-07 浙江师范大学 Comprehensive multi-feature image retrieval method
CN101702239A (en) * 2009-10-16 2010-05-05 西安交通大学 Method for describing texture of image with gradable wavelet packet transformation
CN101777059A (en) * 2009-12-16 2010-07-14 中国科学院自动化研究所 Method for extracting landmark scene abstract
CN102129451A (en) * 2011-02-17 2011-07-20 上海交通大学 Method for clustering data in image retrieval system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
JING LI ET AL: "GPS Estimation from Users’ Photos", 《ADVANCES IN MULTIMEDIA MODELING PART I》, vol. 7732, 9 January 2013 (2013-01-09), pages 118 - 129 *

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105095215B (en) * 2014-04-22 2019-05-10 富士通株式会社 Information acquisition device, method and server
CN105574536A (en) * 2014-10-31 2016-05-11 卡西欧计算机株式会社 Machine learning device, machine learning method, classification device, and classification method
CN105574536B (en) * 2014-10-31 2020-05-01 卡西欧计算机株式会社 Machine learning apparatus and method, classification apparatus and method, and recording medium
CN104765764A (en) * 2015-02-06 2015-07-08 南京理工大学 Indexing method based on large-scale image
CN104778272B (en) * 2015-04-24 2018-03-02 西安交通大学 A kind of picture position method of estimation excavated based on region with space encoding
CN104778272A (en) * 2015-04-24 2015-07-15 西安交通大学 Method for estimating positions of images based on area mining and space encoding
CN104794219A (en) * 2015-04-28 2015-07-22 杭州电子科技大学 Scene retrieval method based on geographical position information
CN104820718B (en) * 2015-05-22 2018-01-30 哈尔滨工业大学 Image classification and search method based on geographic location feature Yu overall Vision feature
CN104820718A (en) * 2015-05-22 2015-08-05 哈尔滨工业大学 Image classification and searching method based on geographic position characteristics and overall situation vision characteristics
CN105183746B (en) * 2015-07-08 2018-04-17 西安交通大学 The method that notable feature realizes image retrieval is excavated from more picture concerneds
CN105183746A (en) * 2015-07-08 2015-12-23 西安交通大学 Method for realizing image retrieval by mining distinguishing features from multiple relevant pictures
CN106776849B (en) * 2016-11-28 2020-01-10 西安交通大学 Method for quickly searching scenic spots by using pictures and tour guide system
CN106776849A (en) * 2016-11-28 2017-05-31 西安交通大学 A kind of method and guide system to scheme quick-searching sight spot
CN108399413A (en) * 2017-02-04 2018-08-14 清华大学 A kind of picture shooting region recognition and geographic positioning and device
CN108399413B (en) * 2017-02-04 2020-10-27 清华大学 Picture shooting area identification and geographical positioning method and device
CN108932273A (en) * 2017-05-27 2018-12-04 腾讯科技(深圳)有限公司 Picture screening technique and device
CN108932273B (en) * 2017-05-27 2022-03-04 腾讯科技(深圳)有限公司 Picture screening method and device
CN110163982A (en) * 2019-04-11 2019-08-23 浙江大学 A kind of virtual fireworks analogy method of immersion based on Sketch Searching and controlled shape
CN112115901A (en) * 2020-09-25 2020-12-22 广州市派客朴食信息科技有限责任公司 High-accuracy food identification method

Also Published As

Publication number Publication date
CN103324677B (en) 2017-02-01

Similar Documents

Publication Publication Date Title
CN103324677A (en) Hierarchical fast image global positioning system (GPS) position estimation method
Li et al. GPS estimation for places of interest from social users' uploaded photos
US11556743B2 (en) Learning highlights using event detection
CN111177446B (en) Method for searching footprint image
US10176196B2 (en) System, method and apparatus for scene recognition
CN101281540B (en) Apparatus, method and computer program for processing information
Zhang et al. Discovering place-informative scenes and objects using social media photos
US20150317511A1 (en) System, method and apparatus for performing facial recognition
CN104376003B (en) A kind of video retrieval method and device
CN104794219A (en) Scene retrieval method based on geographical position information
CN108427740B (en) Image emotion classification and retrieval algorithm based on depth metric learning
CN105631037B (en) A kind of image search method
CN106776849A (en) A kind of method and guide system to scheme quick-searching sight spot
CN103631932A (en) Method for detecting repeated video
CN104838420A (en) Rotation of image based on image content to correct image orientation
CN112395457B (en) Video to-be-retrieved positioning method applied to video copyright protection
CN104951562B (en) A kind of image search method based on VLAD dual adaptions
CN103064985A (en) Priori knowledge based image retrieval method
Li et al. GPS estimation from users’ photos
CN103853794A (en) Pedestrian retrieval method based on part association
CN108171976A (en) Based on the vehicular applications recognition methods in way data
CN104778272B (en) A kind of picture position method of estimation excavated based on region with space encoding
CN113705310A (en) Feature learning method, target object identification method and corresponding device
Kim et al. Classification and indexing scheme of large-scale image repository for spatio-temporal landmark recognition
Zhou et al. Modeling perspective effects in photographic composition

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20170201

Termination date: 20190524

CF01 Termination of patent right due to non-payment of annual fee