CN103473288A - Image search method based on hybrid micro-structure descriptor - Google Patents

Abstract

The invention relates to an image search method based on hybrid micro-structure descriptor (Hybrid MSD, HMSD). The method includes: firstly, using a HMSD to generate a image feature library corresponding to an image library, secondly, using the HMSD to extract content features of a to-be-searched image, thirdly, using similarity measuring criterions for measurement, and finally sorting the similarity measuring results and displaying corresponding images to users. Due to the fact that the HMSD has two common direction and color micro-structures under a color spatial model, high color distinguishing capability on color, texture, shape features and color distribution information is achieved, and attributes of human vision systems is reflected to a certain degree, and search performance is improved greatly.

Description

A kind of image search method based on mixing the microstructure descriptor

Technical field

The present invention relates to a kind of image search method based on mixing the microstructure descriptor.

Background technology

CBIR (Content Based Image Retrieval, CBIR) basic thought is that visually-perceptible content and the semantic content comprised according to retrieving images set up eigenvector, and then mates retrieval according to the similarity of eigenvector.Vision content feature commonly used in CBIR mainly comprises color of image, shape and texture etc.Color, shape, texture be the picture engraving content from different perspectives.In order to improve retrieval performance, need more rational Description Image content, therefore effectively comprehensive dissimilar feature is the effective ways that promote retrieval performance.Document " Image retrieval based on micro-structure descriptor; Pattern Recognition; 2011,44 (9): 2123-2133. " discloses a kind of descriptor of the microstructure for feature extraction (Micro-Structure Descriptor, MSD).The method utilizes microstructure to carry out effective comprehensive description to color, texture, shape and the distributed intelligence thereof of image.At first will input natural image and be transformed into the hsv color space, then carry out the extraction of edge direction at the image obtained, then on the edge direction image, extract microstructure figure, finally, on hsv color spatial image according to microstructure figure after quantification, shone upon, obtain final microstructure image, and microstructure image is carried out to co-occurrence matrix and histogrammic description.Although the characteristics of image such as the method color combining, texture and shape, but because microstructure image is produced by the computing that sought common ground of edge direction image and quantized image, so edge directional information and colouring information descriptive power deficiency, and microstructure analysis process complexity in the method.

The image content informations such as the color that in sum, existing feature extracting method can not effectively comprehensive characterization image, texture, edge direction.

Summary of the invention

The technical matters solved

For fear of the deficiencies in the prior art part, the present invention proposes a kind of image search method based on mixing the microstructure descriptor, overcomes the shortcoming of feature descriptive power deficiency in the conventional images search method, improves the precision of image retrieval.

Technical scheme

A kind of image search method based on mixing the microstructure descriptor is characterized in that step is as follows:

Step 1: to all images in image library, utilize and mix the feature description that the microstructure descriptor carries out picture material, generate corresponding characteristics of image storehouse;

Step 2: utilize and mix the content characteristic that the microstructure descriptor extracts retrieving images;

Step 3: the proper vector in retrieving images feature and characteristics of image storehouse is carried out to similarity measurement;

Step 4: the image that the top n result of similarity measurement is corresponding returns, and the similarity quantization method adopts city piece distance;

The mixing microstructure descriptor step of described step 1 and step 2 is as follows:

Step a: input picture is converted to respectively to the data under RGB and hsv color spatial model;

Step b: the view data under every kind of color space model is carried out respectively to edge direction extraction and color quantizing, obtain color quantizing image and edge direction image;

Step c: respectively each color quantizing image and edge direction image are carried out to the microstructure analysis column hisgram of going forward side by side and describe;

Steps d: use co-occurrence matrix and histogram to be described the microstructure of two same types under the different colours model;

Step e: the edge direction microstructure histogram of the quantized image microstructure histogram of mixing and mixing is carried out comprehensively.

Edge direction extraction step in described step b is as follows:

Step (a): use the Sobel operator to obtain the direction gradient of each passage of image in x and y direction in three-dimensional color space;

Step (b): according to formula $\left|a\right|=\sqrt{{\left(X_h\right)}^2+{\left(Y_h\right)}^2+{\left(Z_h\right)}^2},$ $\left|b\right|=\sqrt{{\left(X_v\right)}^2+{\left(Y_v\right)}^2+{\left(Z_v\right)}^2}$ Obtain formula ab=X _hx _v+ Y _hy _v+ Z _hz _v, then according to angle formulae between vector

calculate the image edge direction image;

Wherein, a (X _h, Y _h, Z _h) and b (X _v, Y _v, Z _v) mean respectively the gradient of horizontal and vertical direction, X _hmean X passage gradient in the horizontal direction, X _vmean X passage gradient in the vertical direction.Y _h, Y _vand Z _h, Z _vit is respectively Y, the Z passage gradient at both direction;

Step (c): the edge direction θ unified quantization of each pixel is become to the m lattice, wherein m ∈ 6,12 ..., 36}.θ (x, y) means edge orientation map, wherein

in HMSD, unification turns to m=24 by side vector, and step-length is 7.5 °.

In described step b, the color quantizing method is as follows: the method that to adopt hsv color space and RGB unified quantization in the quantification of the color space model of HMSD be 120 looks.Wherein the H of two models and R partly be quantified as 0 to 5, S and G partly be quantified as 0 to 3, V and B partly be quantized 0 to 4, integrate, two kinds of models are all to produce 120 looks.

Microstructure analysis method in described step c is as follows: in image, from top to bottom, from left to right travel through each point, when this point and (0 ° of the point of its three neighborhood, consecutive point on 45 ° and 90 ° of directions) in, there is the value of one or more points and this point to equate, this point carries out mark so, otherwise does not carry out mark.After this operation, just completed the microstructure analysis of image when each point of image.

Microstructure histogram describing method in described step c is as follows: unified means the f for image (x, y) of quantification, and the f for value (x, y) of image=w means; To each the some P in image ₀=(x ₀, y ₀), f (P is arranged ₀)=w ₀; At 0 °, spend on direction, use P for 45 ° and 90 ° _i=(x _i, y _i) expression P ₀three neighborhoods and f (P _i)=w _i, i=1 wherein, 2,3; w ₀and w _ico-occurrence time with N, mean,

mean w ₀occurrence number; From top to bottom, from left to right travel through each point on the image quantized, add up point of proximity and its correlation information of this point according to following formula:

Use following formula respectively the microstructure histogram of two edge directions and the microstructure histogram of quantized image to be merged, H (i)=Max{H _a(i), H _b(i) }; H wherein _a(i) be that under RGB color space model, microstructure is described, H _b(i) be that under the hsv color spatial model, microstructure is described.H (i) is the results of two kinds of microstructures after comprehensive.

It is as follows that microstructure under described different colours model is described the integrated approach concrete steps:

Obtain the microstructure histogram of the quantized image of the edge direction microstructure histogram that mixes and mixing, H is arranged respectively _θand H _c.Then by H _θappend at H _cback obtains final descriptor.H _θand H _cthe merging method as shown in the formula.

$H\left(i\right)=\left\{\begin{array}{cc}{H}_C\left(i\right)& i<N\\ \mathrm{\&beta;}\&CenterDot;H_{\mathrm{\&theta;}}(i-N)& i\&GreaterEqual;N\end{array}\right.$

Wherein H is that the histogram of final HMSD is described, H _θthe histogram that edge direction is mixed the microstructure descriptor, H _cbe the histogram that quantized image mixes microstructure, β is H _cand H _θbetween related coefficient, N is H _cdimension.Because H _cand H _θdescription Image different content information, the degree of contributing when retrieving images is also different, revises H so introduce β, in order to improve the feature descriptive power of H.

Beneficial effect

A kind of image search method based on mixing the microstructure descriptor that the present invention proposes, at first utilize and mix microstructure descriptor (Hybrid MSD, HMSD) characteristics of image storehouse corresponding to synthetic image storehouse, then utilize HMSD to extract the content characteristic of retrieving images, then utilize the similarity measurement criterion to be measured, finally by the similarity measurement sort result, and correspond to image and present to the user.

Contain two kinds of directions under color space model commonly used and the microstructure of color owing to mixing the microstructure descriptor in the present invention, therefore on color, texture, shape facility and color distribution information, very strong resolution characteristic is arranged, reflecting to a certain extent human visual system's attribute.Therefore the image retrieval performance of the image search method based on mixing the microstructure descriptor has larger lifting.

The accompanying drawing explanation

Fig. 1: the basic flow sheet of the inventive method;

Fig. 2: the process flow diagram that the microstructure descriptor extracts;

Fig. 3: the process flow diagram of microstructure analysis;

Embodiment

Now in conjunction with the embodiments, the invention will be further described for accompanying drawing:

1,, to all images in image library, the feature of utilizing HMSD to carry out picture material is described, the characteristics of image storehouse that off-line synthetic image storehouse is corresponding;

2, utilize and mix the content characteristic that the microstructure descriptor extracts retrieving images;

3, the retrieving images feature and the proper vector in the characteristics of image storehouse that obtain are used to L ₁distance is carried out similarity measurement one by one.And sorted from small to large according to distance, obtained the minimum front N width result of distance;

4, sorted from small to large according to distance, obtained the minimum front N width result of distance.Image corresponding to this N result returned.

Mix microstructure descriptor leaching process in the present invention as follows:

(1) original image of input is converted to respectively to the data under RGB and hsv color spatial model, the data of acquiescence reading images are the data under RGB color space model, and the computing formula of utilizing RGB to be converted to the hsv color space obtains the data of hsv color spatial model.

The computing formula that RGB is converted to the hsv color space is:

$H=\left\{\begin{array}{cc}\mathrm{\&theta;}& G\&GreaterEqual;B\\ 2\mathrm{\&pi;}-\mathrm{\&theta;}& G<B\end{array}\right\}$

Wherein $\mathrm{\&theta;}={\cos }^{-1}\left[\frac{\frac{1}{2}[(R-G)+(R-B)]}{\sqrt{{(R-G)}^2+(R-B)(G-B)}}\right].$

$S=1-\frac{\sqrt{3}}{V}\min (R,G,B)$

$V=\frac{1}{\sqrt{3}}[R+G+B]$

(2) view data under every kind of color space model is carried out respectively to edge direction extraction and color quantizing, obtain color quantizing image C and edge direction image θ;

Described image edge direction extracting method is as follows:

(a) gradient of the x in color space and y direction can be expressed as a (X _h, Y _h, Z _h) and b (X _v, Y _v, Z _v), X wherein _hmean X passage gradient in the horizontal direction.Their norm and dot product can be defined as follows simultaneously:

$\left|a\right|=\sqrt{{\left(X_h\right)}^2+{\left(Y_h\right)}^2+{\left(Z_h\right)}^2}$

$\left|b\right|=\sqrt{{\left(X_v\right)}^2+{\left(Y_v\right)}^2+{\left(Z_v\right)}^2}$

a·b＝X _hX _v+Y _hY _v+Z _hZ _v

(b) angle of a and b can be expressed as:

$\cos (a,b)=\frac{a\&CenterDot;b}{\left|a\right|\&CenterDot;\left|b\right|}$

$\mathrm{\&theta;}=\arccos \left(\frac{a\&CenterDot;b}{\left|a\right|\&CenterDot;\left|b\right|}\right)$

(c) the edge direction θ unified quantization of each pixel that will calculate becomes the m lattice, wherein m ∈ 6,12 ..., 36}.θ (x, y) means edge orientation map,

in HMSD, unification turns to m=24 by side vector, and step-length is 7.5 °.

Described color quantizing method is: the method that is 120 looks by hsv color space and RGB unified quantization.Wherein the H of two models and R partly be quantified as 0 to 5, S and G partly be quantified as 0 to 3, V and B partly be quantized 0 to 4, so integrate, two kinds of models are all to produce 120 looks.

Concrete for the hsv color spatial model: at first by H by [0,360) be quantified as [0,179], S and V by [0,1) be quantified as [0,255], then H is quantified as [0,1 ..., 5], S is quantified as [0,1,2,3], and V is quantified as [0,1,2,3,4].Finally calculate final quantized result by following formula again.

C＝H*4*5+S*5+V

For the quantization method broadly similar of RGB color space model and HSV, when just quantizing for the first time, three passages all are quantized into [0,255]

Traditional color quantizing method is all to be 8,3,3 dimensions by hsv color space unified quantization, and the H passage is quantified as 8, S and V is quantified as 3.And three passage constants of RGB color space turn to even size, such as 4,4,4.Found through experiments color quantizing method in the present invention characteristics of image more useful for image retrieval.

(3) respectively each color quantizing image and edge direction image being carried out to the microstructure analysis column hisgram of going forward side by side describes;

Described microstructure analysis method is: on image, from top to bottom, from left to right travel through each point, when (0 ° of the point of three neighborhoods, consecutive point on 45 ° and 90 ° of directions) in, have the value of one or more points and this point to equate, this point carries out mark so, otherwise does not carry out mark.After this operation, just obtained the microstructure image M of image when each point of image.Because under every kind of color color quantizing image C and edge direction image θ arranged respectively, obtain altogether four kinds of microstructure images.Be expressed as respectively MC ₁, MC ₂, M θ ₁, M θ ₂.

Former MSD method is that certain constraint condition of utilizing on the edge direction image is screened and obtained microstructure.Then utilize and just now obtained microstructure as constraint condition, at the enterprising row filter of coloured image quantized, obtain microstructure image.Then being added up texture and colouring information on microstructure image.Be actually the common factor of having got two kinds of image informations, omitted a large amount of image content informations.In HMSD, again the microstructure descriptor is combined, the form of employing union has effectively combined two kinds of characteristic informations of image, has carried out the feature description under two kinds of color space models simultaneously.So HMSD is better than original MSD method to the descriptive power of picture material.

Described histogram describing method is: unified means f (x, y) for microstructure image, and the f for value (x, y) of image=w means.To each the some P in image ₀=(x ₀, y ₀), f (P is arranged ₀)=w ₀.At 0 °, spend on direction, use P for 45 ° and 90 ° _i=(x _i, y _i) expression P ₀three neighborhoods and f (P _i)=w _i, i=1 wherein, 2,3.W ₀and w _ico-occurrence time with N, mean,

mean w ₀occurrence number.From left to right travel through from top to bottom each point on the image quantized, and add up closing on and its correlation information of it according to following formula.

Wherein, w ₀=w _i, i ∈ 1,2,3}, and α has meaned that edge direction Microstructure Information and quantized image Microstructure Information are to characterizing the weight of picture material, α carries out approximate evaluation by great many of experiments.

(4) use co-occurrence matrix and histogram to be described the microstructure of two same types under the different colours model.Respectively to { MC ₁, MC ₂and { M θ ₁, M θ ₂carry out comprehensive description.

The method that feature is described is: the histogram of describing the grain direction Microstructure Information in the histogram due to four microstructures is 24 dimensions, and the histogram of describing the quantized image Microstructure Information is 120 dimensions.If these four histograms are directly combined, total histogram just has 288 dimensions.Dimension is excessive, and to cause feature extraction and similarity comparison time and resource cost be unacceptable.Therefore need a kind of effective method to reduce histogrammic dimension.Use following formula respectively the microstructure histogram of two edge directions and the microstructure histogram of quantized image to be merged,

H(i)＝Max{H ₁(i),H ₂(i)}

H wherein ₁(i) be that under RGB color space model, microstructure is described (MC ₁, M θ ₁), H ₂(i) be that under the hsv color spatial model, microstructure is described (MC ₂, M θ ₂).H (i) is the results of two kinds of microstructures after comprehensive, obtains the microstructure histogram of the quantized image of the edge direction microstructure histogram that mixes and mixing, and H is arranged respectively _θand H _c.

(5) the edge direction microstructure histogram of the quantized image microstructure histogram of mixing and mixing is carried out to comprehensive description.Describing method is:

H is arranged respectively _θand H _c.Then by H _θappend at H _cback obtains final descriptor.H _θand H _cthe merging method as shown in the formula.

$H\left(i\right)=\left\{\begin{array}{cc}{H}_C\left(i\right)& i<N\\ \mathrm{\&beta;}\&CenterDot;H_{\mathrm{\&theta;}}(i-N)& i\&GreaterEqual;N\end{array}\right.$

Wherein H is that the histogram of final HMSD is described, H _θthe histogram that edge direction is mixed the microstructure descriptor, H _cbe the histogram that quantized image mixes microstructure, β is H _cand H _θbetween related coefficient, N is H _cdimension.Because H _cand H _θthe quantification dimension and feature from the difference of effect in retrieval, revise H so introduce β, reach the effect that the feature of H descriptive power is provided.

The present invention uses L ₁distance is as similarity measurement, and formula is as follows:

$L_1(A,B)=\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}|a_i-b_i|$

Wherein A, B are two width image characteristic of correspondence vectors, a _i, b _ithe representative feature component.

Claims (5)

1. the image search method based on mixing the microstructure descriptor is characterized in that step is as follows:

Step 1: to all images in image library, utilize and mix the feature description that the microstructure descriptor carries out picture material, generate corresponding characteristics of image storehouse;

Step 2: utilize and mix the content characteristic that the microstructure descriptor extracts retrieving images;

Step 3: the proper vector in retrieving images feature and characteristics of image storehouse is carried out to similarity measurement;

Step 4: the image that the top n result of similarity measurement is corresponding returns, and the similarity quantization method adopts city piece distance;

The mixing microstructure descriptor step of described step 1 and step 2 is as follows:

Step a: input picture is converted to respectively to the data under RGB and hsv color spatial model;

Step b: the view data under every kind of color space model is carried out respectively to edge direction extraction and color quantizing, obtain color quantizing image and edge direction image;

Step c: respectively each color quantizing image and edge direction image are carried out to the microstructure analysis column hisgram of going forward side by side and describe;

Steps d: use co-occurrence matrix and histogram to be described the microstructure of two same types under the different colours model;

Step e: the edge direction microstructure histogram of the quantized image microstructure histogram of mixing and mixing is carried out comprehensively.

2. the image search method based on mixing the microstructure descriptor according to claim 1, is characterized in that: described

Edge direction extraction step in step b is as follows:

Step (a): use the Sobel operator to obtain the direction gradient of each passage of image in x and y direction in three-dimensional color space;

Step (b): according to formula $\left|a\right|=\sqrt{{\left(X_h\right)}^2+{\left(Y_h\right)}^2+{\left(Z_h\right)}^2},$ $\left|b\right|=\sqrt{{\left(X_v\right)}^2+{\left(Y_v\right)}^2+{\left(Z_v\right)}^2}$ Obtain formula ab=X _hx _v+ Y _hy _v+ Z _hz _v, then according to angle formulae between vector calculate the image edge direction image;

Wherein, a (X _h, Y _h, Z _h) and b (X _v, Y _v, Z _v) mean respectively the gradient of horizontal and vertical direction, X _hmean X passage gradient in the horizontal direction, X _vmean X passage gradient in the vertical direction.Y _h, Y _vand Z _h, Z _vit is respectively Y, the Z passage gradient at both direction;

Step (c): the edge direction θ unified quantization of each pixel is become to the m lattice, wherein m ∈ 6,12 ..., 36}.θ (x, y) means edge orientation map, wherein

in HMSD, unification turns to m=24 by side vector, and step-length is 7.5 °.

3. the image search method based on mixing the microstructure descriptor according to claim 1, it is characterized in that: in described step b, the color quantizing method is as follows: the method that to adopt hsv color space and RGB unified quantization in the quantification of the color space model of HMSD be 120 looks.Wherein the H of two models and R partly be quantified as 0 to 5, S and G partly be quantified as 0 to 3, V and B partly be quantized 0 to 4, integrate, two kinds of models are all to produce 120 looks.

4. the image search method based on mixing the microstructure descriptor according to claim 1, it is characterized in that: the microstructure analysis method in described step c is as follows: in image, from top to bottom, from left to right travel through each point, in the point of this point and its three neighborhood, there is the value of one or more points and this point to equate, this point carries out mark so, otherwise does not carry out mark.After this operation, just completed the microstructure analysis of image when each point of image; It is 0 ° that one or more points are arranged in the point of described three neighborhoods, the consecutive point on 45 ° and 90 ° of directions.

5. the image search method based on mixing the microstructure descriptor according to claim 1, it is characterized in that: the microstructure histogram describing method in described step c is as follows: unified by the f (x for image quantized, y) mean, the f for value (x, y) of image=w means; To each the some P in image ₀=(x ₀, y ₀), f (P is arranged ₀)=w ₀; At 0 °, spend on direction, use P for 45 ° and 90 ° _i=(x _i, y _i) expression P ₀three neighborhoods and f (P _i)=w _i, i=1 wherein, 2,3; w ₀and w _ico-occurrence time with N, mean, mean w ₀occurrence number; From top to bottom, from left to right travel through each point on the image quantized, add up point of proximity and its correlation information of this point according to following formula:

Use following formula respectively the microstructure histogram of two edge directions and the microstructure histogram of quantized image to be merged, H (i)=Max{H _a(i), H _b(i) }; H wherein _a(i) be that under RGB color space model, microstructure is described, H _b(i) be that under the hsv color spatial model, microstructure is described.H (i) is the results of two kinds of microstructures after comprehensive.

Images