CN102368266B - Sorting method of unlabelled pictures for network search - Google Patents

Abstract

The invention discloses a sorting method of unlabelled pictures for network search, comprising the following steps of: (1) collecting several reference pictures according to query information; (2) calculating relevancy of each reference picture to the query information; (3) calculating similarity among the reference pictures; (4) amending the relevancy according to the similarity; and (5) sorting the unlabelled pictures according to the amended relevancy. In the method, artificial intelligence related technologies are used, web search and image search results are mined while querying the information, and the relevancy of the unlabelled pictures to the query information is estimated according to the similarity among the reference pictures so as to sort the unlabelled pictures accurately, so that users can search and get the unlabelled pictures, and the searching result is better.

Description

A kind of sort method without the mark picture for web search

Technical field

The invention belongs to web search ordering techniques field, be specifically related to a kind of sort method without the mark picture for web search.

Background technology

As far back as 20 century 70s, the scientific research personnel of the various countries managing image data that how effectively just to begin one's study, the technology that adopted at that time mainly is based on the image retrieval technologies (TBIR) of text, refer to utilize the mode of manual input text to input a series of key word for image, then contact will be set up between the store path of image and the image keyword, in fact image retrieval has become text retrieval, this method is simple, just can realize with traditional relational database, but also there are some shortcomings, workload such as manual input key word is excessive, the key word of sign mass image data is unpractical, and sign inevitably can be with a guy's subjectivity and uncertainty by hand, and different people may be not identical for the understanding of same width of cloth image after all.

In the beginning of this century, the automatic collection of info web and index have obtained deep research as the pith of search engine, and the search engines such as Google, Yahoo are released the picture searching function that adopts the TBIR technology one after another.The image identification that obvious this automatic indexing gathers is very coarse, and accuracy is not high, sometimes or even inaccurate, can retrieve a lot of irrelevant pictures out; Simultaneously, for the picture without text marking that much meets user search information, search engine is to carry out accurately sequencing display to these pictures.

In order to overcome the limitation of text based image retrieval technologies, since the nineties in 20th century, Content-based image retrieval (CBIR) has obtained great development.The CBIR technology mainly refers on the basis that image is processed, and utilizes the basic visual signatures such as spatial relationship of color, shape, texture, profile and the object of image to retrieve.Different from TBIR is the objective visual signature that it has utilized image itself to comprise, and can automatically realize the pick up and store of characteristics of image etc. by computing machine, has improved image processing speed, is conducive to realize that the robotization of image index and retrieval realizes.At present, the system's operation based on the CBIR technology of existing a lot of moulding is such as the Photo Book of MTT and the MARS of UIUC university etc.

But in actual applications, the user only has some about the subjective description of image to required image in advance usually, and what the user needed is the inquiry of image implication, rather than the features such as color, texture, shape.The implication of these images is exactly the high-level semantics features of image, and it has comprised the understanding of people to picture material.Therefore, the CBIR technology is only applicable under the subenvironment search such as the scientific research field database, and and is not suitable under the actual overall situation search such as the internet etc.

Summary of the invention

For the existing above-mentioned technological deficiency of prior art, the invention provides a kind of sort method without the mark picture for web search, realized according to the accurate ordering of Query Information to nothing mark picture, obtain so that the user can search for nothing mark picture, and search effect has been good.

A kind of sort method without the mark picture for web search comprises the steps:

(1) utilizes network search engines to carry out picture searching according to given Query Information, from Search Results, collect the front M pictures of ordering as the reference picture;

(2) calculate the degree of correlation of every reference picture and Query Information;

(3) calculate similarity between reference picture;

(4) according to the similarity between reference picture, revise the degree of correlation of every reference picture and Query Information, obtain the revised degree of correlation of every reference picture and Query Information;

(5) according to the revised degree of correlation of every reference picture and Query Information, have or not the mark picture to sort to institute corresponding to described Query Information.

In the described step (2), the process of the degree of correlation of calculating every reference picture and Query Information is as follows:

A. utilize network search engines to carry out Webpage search according to described Query Information, from Search Results, collect ordering top n webpage as the reference webpage, and be designated as D ₁～D _N

B. for individual with reference to webpage D at N ₁～D _NAny word that occurred is designated as w, and is individual with reference to webpage D at N according to following formula statistics w ₁～D _NIn total frequency of occurrences t (w), and then calculate TF-IDF (the contrary Wen Pin of word frequency) the coefficient ot (w) of w;

t(w)＝y ₁/m ₁+y ₂/m ₂+...+y _N/m _N (1)

ot(w)＝t(w)ln(1+N/n _w) (2)

Wherein: n _wFor N with reference to webpage D ₁～D _NIn contain the webpage number of word w; y _iFor w at reference webpage D _iIn occurrence number, m _iBe reference webpage D _iIn total word number, i=1,2 ..., N;

C. open reference picture G for M ₁～G _MIn arbitrary reference picture G _j, note reference picture G _jCorresponding picture webpage is GD _jFor at M picture webpage GD ₁～GD _MAny word that occurred is designated as gw, adds up gw at M picture webpage GD according to following formula ₁～GD _MIn total frequency of occurrences t _gAnd then calculate the TF-IDF coefficient ot of gw (gw), _g(gw);

t _g(gw)＝y _g，1/m _g，1+y _g，2/m _g，2+...+y _g，M/m _g，M (3)

ot _g(gw)＝t _g(gw)ln(1+M/n _g，gw) (4)

Wherein: n _{G, gw}Be M picture webpage GD ₁～GD _MIn contain the webpage number of word gw; y _{G, i}For gw at picture webpage GD _jIn occurrence number, m _{G, i}Be picture webpage GD _jIn total word number, j=1,2 ..., M;

D. individual with reference to webpage D for N ₁～D _NAny word w of middle appearance and M picture webpage GD ₁～GD _MAny word gw of middle appearance, by the semantic relevancy quantity algorithm, calculate semantic relevancy each other, and then obtain semantic relevancy matrix T H (Q), each semantic relevancy value corresponds to each element value among the semantic relevancy matrix T H (Q), semantic relevancy matrix T H (Q) is that U (Q) * V (Q) ties up matrix, and U (Q) is that N is individual with reference to webpage D ₁～D _NTotal number of middle word, V (Q) is M picture webpage GD ₁～GD _MTotal number of middle word, Q is Query Information;

E. according to semantic relevancy matrix T H (Q), by formula r (G _j, Q)=OT (Q) * TH (Q) * OT _G(Q) calculate every reference picture G _jAnd the degree of correlation r (G between the Query Information Q _j, Q); Wherein: OT (Q)=[ot (w ₁) ..., ot (w _U(Q))], OT _G(Q)=[(ot _g(gw ₁) ..., ot _g(gw _{V (Q)})].

In the described step (3), the process of the similarity between the computing reference picture is as follows:

A. open reference picture G for M ₁～G _MIn arbitrary reference picture G _j, by the Visual Feature Retrieval Process algorithm, extract G _jIn each local visual feature; Wherein, each local visual feature v is two tuples a: v=(C, Des), and C is that v is at picture G _jThe border circular areas of middle covering, Des are the proper vector of this border circular areas and are the vector of one 128 dimension;

B. add up G ₁～G _MIn the occurrence number of each different local visual feature, only keep that wherein occurrence number is greater than the local visual feature of first threshold, first threshold is generally 5～20; Wherein, if the Euclidean distance between two local visual features proper vector separately less than 0.01, then is considered as it two identical local visual features;

C. for the local visual feature that remains, judge local visual feature connectedness between any two: establish any two local visual features that remain and be respectively vi=(Ci, Desi) and vj=(Cj, Desj), if vi and vj are arranged in the local visual feature vk=(Ck that same picture and Ci and Cj intersect or exist another to retain, Desk) with vi and vj all is arranged in same picture and Ck intersects with Ci and Cj, then vi is communicated with vj, otherwise is not communicated with;

D. for the local visual feature that retains, according to connectedness statistics connected component, each connected component is designated as a visual signature bunch; With the equal disconnected single local visual feature of other local visual features, also be designated as a visual signature bunch;

E. add up G ₁～G _MIn the occurrence number of each different visual signature bunch, only keep occurrence number wherein greater than the visual signature of Second Threshold bunch, Second Threshold is generally 5～20; Wherein, if two visual signatures bunch contain duplicate local visual feature, then it is considered as two identical visual signatures bunch;

F. for G ₁～G _MIn any two reference picture G _i, G _j, calculate similarity s (G between them according to following formula _i, G _j);

$s(G_i,G_j)=\underset{\mathrm{vp}\∈G_i,G_j}{\mathrm{\Σ}}\frac{{\left|\right|\mathrm{VP}\left|\right|}^2}{1+\left|\right|\mathrm{VP}\left|\right|}/\mathrm{NVP}(G_i,G_j)---\left(5\right)$

Wherein: VP is a visual signature that retains bunch, and appears at simultaneously G _iAnd G _jIn; || VP|| represents the number of the local visual feature that contains among the VP; NVP (G _i, G _j) be G _iAnd G _jThe number of middle different visual signature bunch.

In the described step (4), the process of the degree of correlation of revising every reference picture and Query Information is as follows:

A. according to the similarity between reference picture, make up similarity matrix S (Q); Each similarity value corresponds to each element value among the similarity matrix S (Q), and similarity matrix S (Q) is that M * M ties up matrix;

B. according to the degree of correlation of every reference picture of following formula correction and Query Information;

$R^{\′}\left(Q\right)=(I+\mathrm{bS}\left(Q\right)+\frac{b^2{S}^2\left(Q\right)}{2!}+\frac{b^3{S}^3\left(Q\right)}{3!}+\frac{b^4{S}^4\left(Q\right)}{4!})R\left(Q\right)---\left(6\right)$

Wherein: I is that M * M ties up unit matrix, and b is correction factor and is generally 0.3, R (Q)=[r (G ₁, Q) ..., r (G _M, Q)], R ' is the revised matrix of R (Q) (Q), R ' each element value in (Q) is every reference picture and the revised relevance degree of Query Information.

In the described step (5), to Query Information corresponding have or not the process that sorts of mark picture as follows:

A. to G ₁～G _MIn arbitrary different visual signatures bunch vp, calculate it with respect to arbitrary reference picture G according to following formula _jTF-IDF coefficient ot _Vp(G _j, vp);

ot _vp(G _j，vp)＝(1+ln(t _j，vp))ln(1+M/m _vp) (7)

Wherein: t _{J, vp}For vp at G _jThe number of times of middle appearance, m _VpBe G ₁～G _MIn contain the picture number of vp;

B. to G ₁～G _MIn arbitrary different visual signatures bunch vp, calculate the degree of correlation rel (vp, Q) of vp and Query Information Q according to following formula;

$\mathrm{rel}(\mathrm{vp},Q)=\underset{j=1}{\overset{M}{\mathrm{\Σ}}}{r}^{\′}(G_j,Q)\frac{{\mathrm{ot}}_{\mathrm{vp}}(G_j,\mathrm{vp})}{{\mathrm{\Σ}}_{{\mathrm{vp}}^{\′}\∈G_j}{\mathrm{ot}}_{\mathrm{vp}}(G_j,{\mathrm{vp}}^{\′})}---\left(8\right)$

Wherein: r ' (G _j, Q) be the revised degree of correlation of every reference picture and Query Information;

C. mark picture P for arbitrary nothing without in the set of mark picture corresponding to Query Information Q _x, calculate P according to following formula _xDegree of correlation Rel (P with Query Information Q _x, Q);

$\mathrm{Rel}(P_x,Q)=\underset{\mathrm{vp}\∈P_x}{\mathrm{\Σ}}\mathrm{rel}(\mathrm{vp},Q)---\left(9\right)$

D. for without arbitrary in the mark picture set without the mark picture according to its degree of correlation with Query Information Q, from big to small ordering, as Query Information Q without mark picture searching demonstration result.

The present invention utilizes the artificial intelligence correlation technique, excavate simultaneously Webpage search and picture search result by Query Information, and according to the similarity between the reference picture, estimate without the degree of correlation between mark picture and the Query Information and then to accurately sorting without the mark picture, obtain so that the user can search for nothing mark picture, and search effect is good.

Description of drawings

Fig. 1 is the steps flow chart synoptic diagram of sort method of the present invention.

Fig. 2 is the present invention and LXTJ and the test data synoptic diagram of FSC method in the MIRFLICKR image data base.

Fig. 3 is the present invention and LXTJ and the test data curve map of FSC method in the CALTECH101 image data base.

Embodiment

In order more specifically to describe the present invention, be elaborated below in conjunction with the technical scheme of the drawings and the specific embodiments to sort method of the present invention.

As shown in Figure 1, a kind of sort method without the mark picture for web search comprises the steps:

(1) collects several reference picture according to Query Information.

The key word of the inquiry Q that the user is given is submitted on the third-party photographic search engine (such as Google's photographic search engine), collects front 100 pictures of ordering as the reference picture from Search Results, is designated as G ₁～G ₁₀₀

(2) degree of correlation of every reference picture of calculating and Query Information.

1. key word of the inquiry Q is submitted on the third-party web page search engine (such as Google's web page search engine), from Search Results, collects front 200 webpages of ordering as the reference webpage, be designated as D ₁～D ₂₀₀

For at 200 with reference to webpage D ₁～D ₂₀₀Any word that occurred is designated as w, according to following formula statistics w at 200 with reference to webpage D ₁～D ₂₀₀In total frequency of occurrences t (w), and then calculate the TF-IDF coefficient ot (w) of w.

t(w)＝y ₁/m ₁+y ₂/m ₂+...+y ₂₀₀/m ₂₀₀ (1)

ot(w)＝t(w)ln(1+200/n _w) (2)

Wherein: n _wBe 200 with reference to webpage D ₁～D ₂₀₀In contain the webpage number of word w; y _iFor w at reference webpage D _iIn occurrence number, m _iBe reference webpage D _iIn total word number, i=1,2 ..., 200.

3. for 100 reference picture G ₁～G ₁₀₀In arbitrary reference picture G _j, note reference picture G _jCorresponding picture webpage is GD _jFor at 100 picture webpage GD ₁～GD ₁₀₀Any word that occurred is designated as gw, adds up gw at 100 picture webpage GD according to following formula ₁～GD ₁₀₀In total frequency of occurrences t _gAnd then calculate the TF-IDF coefficient ot of gw (gw), _g(gw).

t _g(gw)＝y _g，1/m _g，1+y _g，2/m _g，2+...+y _g，100/m _g，100 (3)

ot _g(gw)＝t _g(gw)ln(1+100/n _g，gw)(4)

Wherein: n _{G, gw}Be 100 picture webpage GD ₁～GD ₁₀₀In contain the webpage number of word gw; y _{G, i}For gw at picture webpage GD _jIn occurrence number, m _{G, i}Be picture webpage GD _jIn total word number, j=1,2 ..., 100.

For 200 with reference to webpage D ₁～D ₂₀₀Any word w of middle appearance and 100 picture webpage GD ₁～GD ₁₀₀Any word gw of middle appearance, by the semantic relevancy quantity algorithm, calculate semantic relevancy each other, and then obtain semantic relevancy matrix T H (Q), each semantic relevancy value corresponds to each element value among the semantic relevancy matrix T H (Q), semantic relevancy matrix T H (Q) is that U (Q) * V (Q) ties up matrix, U (Q) be 200 with reference to webpage D ₁～D ₂₀₀Total number of middle word, V (Q) is 100 picture webpage GD ₁～GD ₁₀₀Total number of middle word, Q is key word of the inquiry.

5. according to semantic relevancy matrix T H (Q), by formula r (G _j, Q)=OT (Q) * TH (Q) * OT _G(Q) calculate every reference picture G _jAnd the degree of correlation r (G between the key word of the inquiry Q _j, Q); Wherein: OT (Q)=[ot (w ₁) ..., ot (w _{U (Q)})], OT _G(Q)=[(ot _g(gw ₁) ..., ot _g(gw _{V (Q)})].

(3) similarity between the computing reference picture.

1. for 100 reference picture G ₁～G ₁₀₀In arbitrary reference picture G _j, by the Visual Feature Retrieval Process algorithm, extract G _jIn each local visual feature; Wherein, each local visual feature v is two tuples a: v=(C, Des), and C is that v is at picture G _jThe border circular areas of middle covering, Des are the proper vector of this border circular areas and are the vector of one 128 dimension.

2. add up G ₁～G ₁₀₀In the occurrence number of each different local visual feature, only keep occurrence number wherein greater than 10 local visual feature; Wherein, if the Euclidean distance between two local visual features proper vector separately less than 0.01, then is considered as it two identical local visual features.

3. for the local visual feature that remains, judge local visual feature connectedness between any two: establish any two local visual features that remain and be respectively vi=(Ci, Desi) and vj=(Cj, Desj), if vi and vj are arranged in the local visual feature vk=(Ck that same picture and Ci and Cj intersect or exist another to retain, Desk) with vi and vj all is arranged in same picture and Ck intersects with Ci and Cj, then vi is communicated with vj, otherwise is not communicated with.

4. for the local visual feature that retains, according to connectedness statistics connected component, each connected component is designated as a visual signature bunch; With the equal disconnected single local visual feature of other local visual features, also be designated as a visual signature bunch.

5. add up G ₁～G ₁₀₀In the occurrence number of each different visual signature bunch, only keep occurrence number wherein greater than 10 visual signature bunch; Wherein, if two visual signatures bunch contain duplicate local visual feature, then it is considered as two identical visual signatures bunch.

6. for G ₁～G ₁₀₀In any two reference picture G _i, C _j, calculate similarity s (G between them according to following formula _i, G _j).

$s(G_i,G_j)=\underset{\mathrm{vp}\∈G_i,G_j}{\mathrm{\Σ}}\frac{{\left|\right|\mathrm{VP}\left|\right|}^2}{1+\left|\right|\mathrm{VP}\left|\right|}/\mathrm{NVP}(G_i,G_j)---\left(5\right)$

Wherein: VP is a visual signature that retains bunch, and appears at simultaneously G _iAnd G _jIn; || VP|| represents the number of the local visual feature that contains among the VP; NVP (G _i, G _j) be G _iAnd G _jThe number of middle different visual signature bunch.

(4) according to the similarity correction degree of correlation.

1. according to the similarity between reference picture, make up similarity matrix S (Q); Each similarity value corresponds to each element value among the similarity matrix S (Q), and similarity matrix S (Q) is 100 * 100 dimension matrixes.

2. according to the degree of correlation of every reference picture of following formula correction and key word of the inquiry.

$R^{\′}\left(Q\right)=(I+\mathrm{bS}\left(Q\right)+\frac{b^2{S}^2\left(Q\right)}{2!}+\frac{b^3{S}^3\left(Q\right)}{3!}+\frac{b^4{S}^4\left(Q\right)}{4!})R\left(Q\right)---\left(6\right)$

Wherein: I is 100 * 100 dimension unit matrixs, and b is 0.3, R (Q)=[r (G ₁, Q) ..., r (G ₁₀₀, Q)], R ' is the revised matrix of R (Q) (Q), R ' each element value in (Q) is every reference picture and the revised relevance degree of key word of the inquiry.

(5) according to the revised degree of correlation nothing mark picture is sorted.

1. to G ₁～G ₁₀₀In arbitrary different visual signatures bunch vp, calculate it with respect to arbitrary reference picture G according to following formula _jTF-IDF coefficient ot _Vp(G _j, vp).

ot _vp(G _j，vp)＝(1+ln(t _j，vp))ln(1+100/m _vp) (7)

Wherein: t _{J, vp}For vp at G _jThe number of times of middle appearance, m _VpBe G ₁～G ₁₀₀In contain the picture number of vp.

2. to G ₁～G ₁₀₀In arbitrary different visual signatures bunch vp, calculate the degree of correlation rel (vp, Q) of vp and key word of the inquiry Q according to following formula.

$\mathrm{rel}(\mathrm{vp},Q)=\underset{j=1}{\overset{100}{\mathrm{\Σ}}}{r}^{\′}(G_j,Q)\frac{{\mathrm{ot}}_{\mathrm{vp}}(G_j,\mathrm{vp})}{{\mathrm{\Σ}}_{{\mathrm{vp}}^{\′}\∈G_j}{\mathrm{ot}}_{\mathrm{vp}}(G_j,{\mathrm{vp}}^{\′})}---\left(8\right)$

Wherein: r ' (G _j, Q) be the revised degree of correlation of every reference picture and key word of the inquiry.

3. mark picture P for arbitrary nothing without in the set of mark picture corresponding to key word of the inquiry Q _x, calculate P according to following formula _xDegree of correlation Rel (P with key word of the inquiry Q _x, Q).

$\mathrm{Rel}(P_x,Q)=\underset{\mathrm{vp}\∈P_x}{\mathrm{\Σ}}\mathrm{rel}(\mathrm{vp},Q)---\left(9\right)$

For without arbitrary in the mark picture set without the mark picture according to its degree of correlation with key word of the inquiry Q, from big to small ordering, as Query Information Q without mark picture searching demonstration result.

With other two kinds of picture sort method LXTJ (method described in the paper of " the Textual query of personal photos facilitated by large-scale web data " by name that delivered in 2011) and FSC (method described in the paper of " the Abootstrapping framework for annotating and retrieving www images " by name that delivered in 2004) that are used for web search in present embodiment Ours and the prior art, be applied to respectively in MIRFLICKR and the CALTECH101 image data base, and search for respectively test, according to achievement data NDCG (Normalized Discounted Cumulative Gain) three kinds of methods are compared, concrete test data as shown in Figures 2 and 3; Wherein, the NDCG value is larger, and the expression search effect is just better, and the search effect of visible present embodiment has more advantage with respect to LXTJ and FSC.

Claims (1)

1. the sort method without the mark picture that is used for web search comprises the steps:

(1) utilizes network search engines to carry out picture searching according to given Query Information, from Search Results, collect the front M pictures of ordering as the reference picture;

(2) calculate the degree of correlation of every reference picture and Query Information, concrete mode is as follows:

A. utilize network search engines to carry out Webpage search according to described Query Information, from Search Results, collect ordering top n webpage as the reference webpage;

B. be designated as w for any word that occurred in reference to webpage at N, individual with reference to the total frequency of occurrences in the webpage at N according to following formula statistics w, and then the TF-IDF coefficient of calculating w;

t(w)=y ₁/m ₁+y ₂/m ₂+...+y _N/m _N （1）

ot(w)=t(w)ln(1+N/n _w)（2）

Wherein: n _wFor N with reference to webpage D ₁～D _NIn contain the webpage number of word w; y _iFor w at reference webpage D _iIn occurrence number, m _iBe reference webpage D _iIn total word number, t (w) is w ot (w) is the TF-IDF coefficient of w with reference to the total frequency of occurrences in the webpage at N, i=1,2 ..., N;

C. be designated as gw for any word that occurred at M picture webpage, described picture webpage is the corresponding webpage of reference picture, according to the total frequency of occurrences of following formula statistics gw in M picture webpage, and then the TF-IDF coefficient of calculating gw;

t _g(gw)=y _g,1/m _g，1+y _g,2/m _g,2+...+y _g，M/m _g,M （3）

ot _g(gw)=t _g(gw)ln(1+M/n _g，gw)（4）

Wherein: n _{G, gw}Be M picture webpage GD ₁～GD _MIn contain the webpage number of word gw; y _{G, i}For gw at picture webpage GD _jIn occurrence number, m _{G, i}Be picture webpage GD _jIn total word number, t _g(gw) be the total frequency of occurrences of gw in M picture webpage, ot _g(gw) be the TF-IDF coefficient of gw, j=1,2 ..., M;

D. for N with reference to any the word gw that occurred in any the word w that occurred in the webpage and M the picture webpage, by the semantic relevancy quantity algorithm, calculating semantic relevancy each other, and then obtain the semantic relevancy matrix;

E. according to the semantic relevancy matrix, by formula r (G _j, Q)=OT (Q) * TH (Q) * OT _G(Q), calculate the degree of correlation between every reference picture and the Query Information; Wherein: OT (Q)=[ot (w ₁) ..., ot (w _{U (Q)})], OT _G(Q)=[(ot _g(gw ₁) ..., ot _g(gw _{V (Q)})], TH (Q) is the semantic relevancy matrix, r (G _j, Q) be the degree of correlation between arbitrary reference picture and the Query Information; U (Q) is the individual total number with reference to word in the webpage of N, and V (Q) is total number of word in M the picture webpage;

(3) calculate similarity between reference picture, concrete mode is as follows:

A. open arbitrary reference picture in the reference picture for M, by the Visual Feature Retrieval Process algorithm, extract each local visual feature in arbitrary the reference picture;

B. add up M and open the occurrence number of each different local visual feature in the reference picture, only keep wherein occurrence number greater than the local visual feature of first threshold;

C. for the local visual feature that remains, judge local visual feature connectedness between any two;

D. for the local visual feature that retains, according to connectedness statistics connected component, each connected component is designated as a visual signature bunch; With the equal disconnected single local visual feature of other local visual features, also be designated as a visual signature bunch;

E. add up the occurrence number that M opens each different visual signature in the reference picture bunch, only keep occurrence number wherein greater than the visual signature of Second Threshold bunch;

F. open any two reference picture in the reference picture for M, calculate similarity between them according to following formula;

$s(G_i,G_j)=\underset{\mathrm{vp}\∈G_i,G_j}{\mathrm{\Σ}}\frac{{\left|\right|\mathrm{VP}\left|\right|}^2}{1+\left|\right|\mathrm{VP}\left|\right|}/\mathrm{NVP}(G_i,G_j)---\left(5\right)$

Wherein: s (G _i, G _j) be any two reference picture G _iAnd G _jBetween similarity, VP is a visual signature that retains bunch, and appears at simultaneously reference picture G _iAnd G _jIn; ‖ VP ‖ represents the number of the local visual feature that contains among the VP; NVP (G _i, G _j) be reference picture G _iAnd G _jThe number of middle different visual signature bunch;

(4) according to the similarity between reference picture, revise the degree of correlation of every reference picture and Query Information, obtain the revised degree of correlation of every reference picture and Query Information, concrete mode is as follows:

A. according to the similarity between reference picture, make up similarity matrix;

B. according to the degree of correlation of every reference picture of following formula correction and Query Information;

$R^{\′}\left(Q\right)=(I+\mathrm{bS}\left(Q\right)+\frac{b^2{S}^2\left(Q\right)}{2!}+\frac{b^3{S}^3\left(Q\right)}{3!}+\frac{b^4{S}^4\left(Q\right)}{4!})R\left(Q\right)---\left(6\right)$

Wherein: I is that M * M ties up unit matrix, and b is correction factor, and S (Q) is similarity matrix, R (Q)=[r (G ₁, Q) ..., r (G _M, Q)], R ' is the revised matrix of R (Q) (Q), R ' each element value in (Q) is every reference picture and the revised relevance degree of Query Information;

(5) according to the revised degree of correlation of every reference picture and Query Information, have or not the mark picture to sort to institute corresponding to described Query Information, concrete mode is as follows:

A. open in the reference picture arbitrary different visual signatures bunch for M, calculate it with respect to the TF-IDF coefficient of arbitrary reference picture according to following formula;

ot _vp(G _j,vp)=（1+ln(t _j,vp)）ln(1+M/m _vp)（7）

Wherein: vp is arbitrary different visual signature bunch, t _{J, vp}For vp at reference picture G _jThe number of times of middle appearance, m _VpFor M opens the picture number that contains vp in the reference picture, ot _Vp(G _j, be that vp is with respect to G vp) _jThe TF-IDF coefficient;

B. open in the reference picture arbitrary different visual signatures bunch for M, calculate the degree of correlation of itself and Query Information according to following formula;

$\mathrm{rel}(\mathrm{vp},Q)=\underset{j=1}{\overset{M}{\mathrm{\Σ}}}{r}^{\′}(G_j,Q)\frac{{\mathrm{ot}}_{\mathrm{vp}}(G_j,\mathrm{vp})}{{\mathrm{\Σ}}_{{\mathrm{vp}}^{\′}\∈G_j}{\mathrm{ot}}_{\mathrm{vp}}(G_j,{\mathrm{vp}}^{\′})}---\left(8\right)$

Wherein: r ' (G _j, Q) be the revised degree of correlation of every reference picture and Query Information; Q is Query Information, and rel (vp, Q) is the degree of correlation of vp and Q;

C. for Query Information corresponding without arbitrary in the mark picture set without the mark picture, calculate the degree of correlation of itself and Query Information according to following formula;

$\mathrm{Rel}(P_x,Q)=\underset{\mathrm{vp}\∈P_x}{\mathrm{\Σ}}\mathrm{rel}(\mathrm{vp},Q)---\left(9\right)$

Wherein: P _xBe arbitrary nothing mark picture, Rel (P _x, Q) be P _xThe degree of correlation with Q;

D. for without arbitrary in the mark picture set without the mark picture according to its degree of correlation with Query Information, sort from big to small.

Images