A kind of image search method based on strengthening microstructure and context similarity
Technical field
The invention belongs to Computer Image Processing and information retrieval field.Relate to a kind of image search method based on strengthening microstructure and context similarity.
Background technology
Along with the development of multimedia and Internet technology, increasing image information is produced.How effectively to retrieve them and just become a difficult problem.Image search method can be divided three classes: text based retrieval (TBIR), content-based retrieval (CBIR) and the retrieval based on semantic (SBIR).Text based retrieval needs manually image to be marked in a large number, and subjectivity is strong, and cost is very high, is not substantially used now.Use the method for machine learning to mark image based on semantic retrieval, owing to being subject to technical limitation, also very immature at present.What therefore application was at present more is CBIR.
The inquiry illustration that CBIR provides according to user is searched for it maximally related image and is returned to user in database.Common CBIR method can be divided into semi-supervised method and unsupervised method two classes.The low-level feature of unsupervised method abstract image also directly uses these feature calculation similarities; And semi-supervised method is generally carried out cluster by the local visual feature to test pattern and is obtained vision code book, then with quantizing vector representation image and calculating similarity so that image is sorted.The method that IEEE image is processed transactions (IEEE Transactions on Image Processing15 (6) (2006) 1443 – 1453) announcement is used unsupervised method exactly, and European computer vision international accounting (In Proceedings of9
theuropean Conferenceon Computer Vision (ECCV) 2012) propose method be a kind of semi-supervised method.Unsupervised method is generally extracted the features such as the color, texture, shape of image and is retrieved.In image retrieval, single feature generally cannot obtain gratifying effect, carries out image retrieval and exists weight to be difficult to the problem of determining and the time efficiency that causes sharply declines because will process various features and combine multiple features.On the other hand, semi-supervised method is owing to relating to problem concerning study, and do not have image that sufficient amount and content are suitable in most application scenarios as training/study image, so its scope of application is very limited.
Pattern-recognition (Pattern Recognition2011:2123-2133) proposes the descriptor based on image microstructure, and this descriptor carrys out the various features of integrated image by microstructure, and algorithm time efficiency is higher.But this algorithm distance metric is unreasonable, and do not make full use of contextual information, therefore retrieval rate is not high.
Summary of the invention
In order to overcome the deficiency of conventional images retrieval technique, the object of this invention is to provide a kind of image search method based on strengthening microstructure and context similarity, the method adopts simple mode to extract the descriptor of the multiple visual signature of a kind of integrated image, use in addition contextual information to introduce the supervision message that non-user participates in, in not reducing algorithm time efficiency, can improve image retrieval accuracy rate.
Object of the present invention is achieved through the following technical solutions:
Based on the image search method that strengthens microstructure and context similarity, it is characterized in that its step is as follows:
1) by image from RGB color space conversion to hsv color space and quantize, obtain color quantizing mapping graph;
2) corresponding gray-scale map is carried out to local mode quantification, local mode based on after quantizing shines upon the microstructure mapping graph that is enhanced, color quantizing mapping graph is filtered with this mapping graph, the mapping graph after filtering is adopted to the symbiosis Description Image based on color quantizing value;
3) gauged distance of the image conjunctive search image in storehouse is sorted, the method that recycling context similarity is propagated is resequenced to image in all reference sets;
Described by image from RGB color space conversion to hsv color space and quantize, obtaining color quantizing mapping graph step is: the coloured image g (x that is M × N for a width size, y), first by it from RGB color notation conversion space to hsv color space.Then by H, S, V component is quantized into respectively 8,3,3 grades.Therefore color histogram has 72 handles.By quantize after color image be expressed as C (x, y), wherein c (x, y) ∈ 1,2 ... 72}.
Described carries out local mode quantification to corresponding gray-scale map, local mode based on after quantizing shines upon the microstructure mapping graph that is enhanced, color quantizing mapping graph is filtered with this mapping graph, adopts the symbiosis Description Image step based on color quantizing value to be to the mapping graph after filtering:
(1) local mode quantizes
First coloured image converts gray-scale map to, and then gray-scale map is quantized into L handle, herein L=32.Quantize image representation be G (x, y) wherein G (x, y)=l ∈ 0,1 ... L-1}.Definition local mode mapping lpm (local pattern map), its computing method as shown in the formula:
Wherein
Also be that arbitrary pixel place mapping value is set to quantize the number of pixels that gray-scale value is less than or equal to current pixel gray-scale value in its 3 × 3 neighborhood.Lpm span from 0 to 8.
(2) strengthening microstructure mapping extracts
Represent to strengthen microstructure mapping graph with alphabetical M, in M, first all elements is initialized as 0, and then in M, the value of any point p is obtained by following formula:
Nei in above formula
8(p) 8 neighborhoods of indication p are also 3 × 3 neighborhoods.Mapping graph lpm is pressed from top to bottom, sequential scanning from left to right, for the some p of current scanning, if having at least the value of any and its value to equate in its 8 neighborhood, the value of local microstructure mapping graph M mid point p corresponding position is made as 1, otherwise is 0.
(3) iamge description
Set up color quantizing microstructure figure f (x, y) with following formula.
f(x,y)=M(x,y)·C(x,y) 4
Then f is expressed as to the vectorial H of one 72 dimension (the same with the dimension of C), as the descriptor of former figure.H is determined by following formula:
Wherein p
0image f current scan point, | p
i-p
0|=1 represents p
ip
0a point in 3 × 3 neighborhoods.N{*} represent the to satisfy condition number of point of *.N{f (p
0)=w
0represent that mapping graph f intermediate value is w
0the number of point, and N{f (p
0)=w
0∧ f (p
i)=w
0|| p
i-p
0|=1} is illustrated in mapping graph f and meets value for w
0and it is also w that 3 × 3 neighborhoods of this point have a point value at least
0the number of point.
The described gauged distance to the image conjunctive search image in storehouse sorts, and the method that recycling context similarity is propagated to the step of resequencing of image in all reference sets is:
(1) gauged distance calculates
The descriptor P that two images are corresponding, Q spacing is calculated as follows:
Wherein P
i, Q
ibe respectively P, i the component that Q is corresponding.
Use the distance measure of above formula, the distance span between all random images pair, between 0 to 1, is therefore called gauged distance.
(2) reference set based on shortest path calculates
A given group objects X={x
1... x
n, a distance function d=X × X → R
2with a similarity function sim (x, y)=X × X → R
2, x
1query image and { x
2... x
none group of known object in image library, x, y ∈ X.SP (x
1, x
t) represent from x
1to x
ta shortest path, piece image in each ode table diagram image set on this path, corresponding R
sP(x
t) be called shortest path reference object collection, be expressed as follows:
R
SP(X
t)=SP(x
1,x
t) 7
Make d
sP(x
i, x
j) expression node x
iand x
jbetween shortest path distance.Node x
iand nodal set
between shortest path distance definition as follows:
Reference set X
r sP(x
t) be defined as follows:
X
r SP(x
t)={x
i|d
SP(x
i,R
SP(x
t))≤q
r} 9
Q is set
r=r (r=0 ... R).
Then " r-expansion shortest path references object collection " be defined as follows:
R
rESP(x
t)=X
r SP(x
t) 10
(3) image sequence based on context similarity reorders
If query image is x
1, iterations T value is 50, the front N width image of getting initial collating sequence reorders, N=100 herein, another Distance matrix D
ij=distance (x
i, x
j) i, j=1 ... N+1 is x wherein
2, x
3... x
n+1it is the front N width image of initial collating sequence.The step reordering is as follows:
1) use following formula conversion distance matrix to weight matrix
w
ij=exp(-d
2 ij/σ
2 ij) 11
Its center size σ
ijbe defined as follows:
σ
ij=α·mean(dis(x
i,knnd(x
i)) 12
Convert weight matrix to probability matrix again:
2)For t=1 to T do
For i=2 to N+1 do
End
End
3) according to N width image before new similarity Re-rank, and export the sequence x ' after Re-rank
2, x '
3... x '
n+1
Mean in 12 formulas (X) refers to gather the average of all elements in X, and knnd (x) refers to the set of K nearest-neighbors composition of element x, and K value is 2, and α value is 0.2.In 13 formulas, r value is 4, and primary iteration value simi in 14 formulas
0(x
1, x
1)=1, simi
0(x
1, x
j)=0, j=2,3 ... N+1.
Compared with prior art, the present invention has following beneficial effect:
1) the enhancing microstructure descriptor in the present invention is integrated color, texture, shape facility, but it is not to combine integrated by weights, but obtains by the enhancing microstructure based on local mode.Thereby avoid the shortcoming that classic method speed is slow, intrinsic dimensionality is high, with the feature of the relatively low dimension of VELOCITY EXTRACTION faster;
2) the present invention is normalized distance metric, relatively in the past more reasonable with its distance metric method of class methods;
3) the present invention can solve to a certain extent in frame of video and repeat the too much problem of captions, can prevent that again some captions is missed simultaneously, has obtained good effect on continuous sequence of frames of video.
The present invention can overcome the microstructure that generally method based on microstructure exists and visual cues is inconsistent, distance metric is unreasonable and do not consider context environmental shortcoming, by introducing the shortest path transmission method of local mode mapping and distance, good image retrieval effect be can obtain, retrieval precision ratio and recall ratio improved to a certain extent.
Brief description of the drawings
Fig. 1 is image retrieval frame diagram;
Embodiment
Technical scheme for a better understanding of the present invention, below in conjunction with accompanying drawing 1, the invention will be further described.Accompanying drawing 1 has been described the frame diagram of image search method of the present invention.
The step of the image search method based on enhancing microstructure and context similarity is as follows:
1) by image from RGB color space conversion to hsv color space and quantize, obtain color quantizing mapping graph; Step is:
Be the coloured image g (x, y) of M × N for a width size, first by it from RGB color notation conversion space to hsv color space.Then by H, S, V component is quantized into respectively 8,3,3 grades.Therefore color histogram has 72 handles.By quantize after color image be expressed as C (x, y), wherein c (x, y) ∈ 1,2 ... 72}.
2) corresponding gray-scale map is carried out to local mode quantification, local mode based on after quantizing shines upon the microstructure mapping graph that is enhanced, color quantizing mapping graph is filtered with this mapping graph, the mapping graph after filtering is adopted to the symbiosis Description Image based on color quantizing value; Step is:
(1) local mode quantizes
First coloured image converts gray-scale map to, and then gray-scale map is quantized into L handle, herein L=32.Quantize image representation be G (x, y) wherein G (x, y)=l ∈ 0,1 ... L-1}.Definition local mode mapping lpm (local pattern map), its computing method as shown in the formula:
Wherein
Also be that arbitrary pixel place mapping value is set to quantize the number of pixels that gray-scale value is less than or equal to current pixel gray-scale value in its 3 × 3 neighborhood.Lpm span from 0 to 8.
(2) strengthening microstructure mapping extracts
Represent to strengthen microstructure mapping graph with alphabetical M, in M, first all elements is initialized as 0, and then in M, the value of any point p is obtained by following formula:
Nei in above formula
8(p) 8 neighborhoods of indication p are also 3 × 3 neighborhoods.Mapping graph lpm is pressed from top to bottom, sequential scanning from left to right, for the some p of current scanning, if having at least the value of any and its value to equate in its 8 neighborhood, the value of local microstructure mapping graph M mid point p corresponding position is made as 1, otherwise is 0.
(3) iamge description
Set up color quantizing microstructure figure f (x, y) with following formula.
f(x,y)=M(x,y)·C(x,y) 4
Then f is expressed as to the vectorial H of one 72 dimension (the same with the dimension of C), as the descriptor of former figure.H is determined by following formula:
Wherein p
0image f current scan point, | p
i-p
0|=1 represents p
ip
0a point in 3 × 3 neighborhoods.N{*} represent the to satisfy condition number of point of *.N{f (p
0)=w
0represent that mapping graph f intermediate value is w
0the number of point, and N{f (p
0)=w
0∧ f (p
i)=w
0|| p
i-p
0|=1} is illustrated in mapping graph f and meets value for w
0and it is also w that 3 × 3 neighborhoods of this point have a point value at least
0the number of point.
3) gauged distance of the image conjunctive search image in storehouse is sorted, the method that recycling context similarity is propagated is resequenced to image in all reference sets; Step is:
(1) gauged distance calculates
The descriptor P that two images are corresponding, Q spacing is calculated as follows:
Wherein P
i, Q
ibe respectively P, i the component that Q is corresponding.
Use the distance measure of above formula, the distance span between all random images pair, between 0 to 1, is therefore called gauged distance.
(2) reference set based on shortest path calculates
A given group objects X={x
1... x
n, a distance function d=X × X → R
2with a similarity function sim (x, y)=X × X → R
2, x
1query image and { x
2... x
none group of known object in image library, x, y ∈ X.SP (x
1, x
t) represent from x
1to x
ta shortest path, piece image in each ode table diagram image set on this path, corresponding R
sP(x
t) be called shortest path reference object collection, be expressed as follows:
R
SP(X
t)=SP(x
1,x
t) 7
Make d
sP(x
i, x
j) expression node x
iand x
jbetween shortest path distance.Node x
iand nodal set
between shortest path distance definition as follows:
Reference set X
r sP(x
t) be defined as follows:
X
r SP(x
t)={x
i|d
SP(x
i,R
SP(x
t))≤q
r} 9
Q is set
r=r (r=0 ... R).
Then " r-expansion shortest path references object collection " be defined as follows:
R
rESP(x
t)=X
r SP(x
t) 10
(3) image sequence based on context similarity reorders
If query image is x
1, iterations T value is 50, the front N width image of getting initial collating sequence reorders, N=100 herein, another Distance matrix D
ij=distance (x
i, x
j) i, j=1 ... N+1 is x wherein
2, x
3... x
n+1it is the front N width image of initial collating sequence.The step reordering is as follows:
1) use following formula conversion distance matrix to weight matrix
w
ij=exp(-d
2 ij/σ
2 ij) 11
Its center size σ
ijbe defined as follows:
σ
ij=α·mean(dis(x
i,knnd(x
i)) 12
Convert weight matrix to probability matrix again:
2)For t=1 to T do
For i=2 to N+1 do
End
End
3) according to N width image before new similarity Re-rank, and export the sequence x ' after Re-rank
2, x '
3... x '
n+1
Mean in 12 formulas (X) refers to gather the average of all elements in X, and knnd (x) refers to the set of K nearest-neighbors composition of element x, and K value is 2, and α value is 0.2.In 13 formulas, r value is 4, and primary iteration value simi in 14 formulas
0(x
1, x
1)=1, simi
0(x
1, x
j)=0, j=2,3 ... N+1.
Concrete application example:
For certain piece image, provide the flow instance to extracting the mapping of enhancing microstructure wherein and microstructure descriptor.Describe below in conjunction with method of the present invention the concrete steps that this example is implemented in detail, as follows:
(1) for certain piece image, adopt hsv color quantization method first by image from RGB color space conversion to hsv color space and quantize, wherein H element quantization becomes 8 grades, S and V component are respectively quantified as 3 grades.Obtain color quantizing mapping graph.
(2) use local mode quantization method that image is carried out to gray processing for this figure, and obtain local mode mapping (converting 8 grades of gray-scale maps to).Obtain image and local mode map figure after gray processing;
(3) the local mode mapping graph that above step obtains is for inputting, and employing strengthens microstructure mapping abstracting method and extracts enhancing microstructure, obtains the image after the mapping of enhancing microstructure.Wherein stain represents that this some place mapping value is 0, otherwise is 1;
(4) above step obtains enhancing microstructure and color quantizing mapping graph are input, adopt Image Description Methods to obtain the vector description of image;
(5) image vector upper step being obtained is described, and uses gauged distance computing method to obtain the preliminary distance between image, the line ordering of going forward side by side;
(6) for the result of preliminary sequence, use based on the reference set of shortest path and calculate and image sequence method for reordering computing reference collection based on context similarity utilize reference set and context recalculates the similarity between image, and re-start sequence.
This method can find out preferably with retrieve the similar image of illustration and by they come result for retrieval sequence before, obtain higher retrieval rate.