CN1165859C - Muctilevel image grid data structure and image search method using the same - Google Patents

Abstract

The present invention relates to an image search method capable of expressing one color feature related to a spatial color feature of a still image based on a multilevel image grid and similarity-based searching images using the thusly expressed multilevel image grid. In the present invention, hierarchical grids of different levels are generated with respect to one feature for thereby obtaining a data structure in which each cell corresponding to the grid is expressed based on a reliability on a region representative color and the region representative color, so that it is possible to fast and accurately search images with respect to a user's content-based query based on a cell matching of the same level as two image grids and different levels or a color local matching of the grid match.

Description

Use the visual searching method of multi-level image data structure

Technical field

The present invention relates to use the visual searching method of multi-level image data structure, be specifically related to have the multi-level image raster data structure of different classification lattice level structures with respect to a kind of color characteristic relevant with the spatial color character of still image, and the visual searching method that is used to use multi-level image raster data structure search image.

Background technology

In the visual searching method of routine, be to represent such as features such as color, shape, textures with the visual raster data structure of one-level, and the similarity between the different pictorial data of the visual raster data search same structure of use one-level, thereby search image.

According to the visual searching method search image of routine the time, according to the characteristic of the image that will search for, the importance of each feature is different.And even only for a kind of feature, the importance of each unit also is different in the conventional visual raster data structure.For example, in the visual searching method that uses the color histogram that forms by n dimension structure,, the weighted value that reflects the importance of each element can be defined as different values for each element that forms this n dimension structure.

That is, in the conventional visual searching method of the map data structure that uses one-level, come importance between the representation feature based on the grid of correspondence.But, in this case, do not consider the importance of each element of certain feature.In order to address this problem, another kind of conventional visual searching method has adopted the method for calculating the average importance of each element in certain feature.

But, in the visual searching method of above-mentioned routine, because the importance of each element is influenced by target image or reference picture, therefore the average importance of each element of certain feature is useless in the image search, and the mean value that promptly pre-determines each element of certain feature is useless.

And because conventional visual raster data structure only forms one-level, therefore the search to the target (or target image) that comprises in the image is coarse in the conventional visual searching method.

Summary of the invention

Therefore, an object of the present invention is to provide a kind of multipole map data structure, wherein by representing a feature based on the multi-level image grid and representing the regional representativeness color of each unit and, represent each level by the unit of multilevel hierarchy not at the same level with respect to the reliability of regional representativeness color.

Another object of the present invention provides a kind of visual searching method, can between the not at the same level and color region of the unit of the same one-level of two visual grids, grid, mate, with for carrying out the color similarity retrieval with the corresponding multi-level image grid of different images.

To achieve these goals, according to the invention provides a kind of multi-level image data structure, wherein to have the spatial color feature of representing an image more than two hierarchical image lattice structures not at the same level.

To achieve these goals, according to the invention provides a kind of visual searching method that uses the multi-level image data structure, this data structure is represented based on the visual grid with at least two different split poles, this method may further comprise the steps: determine the color similarity between reference picture and the target image, wherein reference picture and target image are all represented by the lattice level of classification; With come search image according to the content-based inquiry of user.

Preferably, described determining step comprises: the unit that the unit that the grid of match reference image comprises in extremely and the grid of target image comprise in extremely.

Preferably, described determining step comprises: each grid utmost point of the grid utmost point of match reference image and target image; And the grid utmost point of the grid utmost point of cross-matched reference picture and target image.

Preferably, described determining step comprises: the regional representativeness color value between the grid utmost point of match reference image and target image.

Preferably, the similarity between the unit that comprises in extremely of the split pole grid of determining reference picture and target image according to the following step: will and the unit that comprises in extremely of the grid of reference picture and target image between the corresponding color similarity of similarity of regional representativeness color on duty with one first flexible strategy; The similarity of the similarity of the reliability between the unit that described color similarity value and are comprised in extremely by the grid of representing reference picture and target image multiply by the value addition that one second flexible strategy obtain; Unit similarity with the normalization gained.

Preferably, when having compared two grids and having calculated similarity, based on by according to the shift amount that obtains by width and difference in height between the grid utmost point in the total value that level and vertical direction displacement are added up to, determine the similarity between the same stages grid in reference picture and the target image.

Preferably, based on by the width between the grid utmost point and difference in height in the value that level and vertical direction displacement are added up to, determine the color similarity between the grid in reference picture and the target image.

Preferably, carrying out under the situation of search by the match zone, the unit similarity between the grid utmost point in use reference picture and the target image is searched for same position and the diverse location between the identical grid utmost point.

Preferably, when the color similarity between the different grid utmost points of search, the color region matching operation between the grid utmost point of reference picture and target image is used to search at the same position and the diverse location of the different grid utmost points.

The advantage that the present invention adds, purpose and feature can easily obtain from following explanation.

Description of drawings

Can more completely understand the present invention by following detailed description and accompanying drawing, the example that provides in the accompanying drawing only is illustrative, is not limitation of the present invention therefore, wherein:

Fig. 1 is the synoptic diagram according to the embodiment of multi-level image raster data structure of the present invention and 3 grades of visual raster data structures;

Fig. 2 is the synoptic diagram according to the visual searching method of matching structure between use multi-level image raster data structure of the present invention and 3 grades of visual raster data structures;

Fig. 3 is according to the synoptic diagram of the embodiment of the visual searching method of matching structure between the peer in use multi-level image raster data structure of the present invention and the 3 grades of visual raster data structures;

Fig. 4 is the synoptic diagram according to the embodiment of the visual searching method of matching structure between use multi-level image raster data structure of the present invention and 3 grades of visual raster data structures not at the same level;

Fig. 5 A and 5B are the synoptic diagram according to the embodiment of the visual searching method of use multi-level image raster data structure of the present invention, wherein Fig. 5 A is the synoptic diagram of two identical visual raster data structures, and Fig. 5 B is the synoptic diagram of the matching process of two visual raster data structures.

Embodiment

The present invention relates to the visual searching method of multi-level image raster data structure and this data structure of use.To describe the method that is used to produce the multi-level image raster data structure according to the present invention below.

Under the situation of square image, it is divided by height and width equably, under the situation of non-square image, divide equably on one side according to the aspect ratio of image width and height, and divide another side equably by unit on one side.Promptly, have the level of equal length and the regular square structure of vertical edges according to the same units division, and under the situation of the rectangular configuration of level with different length and vertical edges, on one side (for example, longer sides) evenly divided, another side (for example, than minor face) is then divided by this dividing unit on one side.

Therefore to above-mentioned similar, in a map data structure, the spatial color feature is divided into multistage grid not at the same level, thus the structure of expression multi-level image grid.

At this moment, each visual grid is a multilevel hierarchy not at the same level, and the resolution of each grade is divided by classification.Distribute two values for the unit of each grid, these two values are regional representativeness color (RRC) and the reliability score value (S) relevant with the precision of regional representativeness color.

Fig. 1 represents the embodiment according to multi-level image raster data structure of the present invention and 3 grades of visual raster data structures.That is, image is represented as the visual lattice level of the first order, the second level and the third level.

In the resolution of 3 grades of visual raster data structures, according to the level of being divided, first order image raster resolution is minimum, and second level image grid is an intergrade, and third level image raster resolution is than the visual grid height in the second level.

First order image grid is divided into the image region that comprises with the proportional M1 * N1 of the aspect ratio local unit of vertical edges M and horizontal sides N.Each unit is represented as represents each regional regional representativeness color (RRC) and corresponding to the reliability score value (S) of the precision of representative color value.

And according to the division state, second level image grid and third level image grid are divided into the image region that comprises M2 * N2 and M3 * N3 local unit, and each unit has regional representativeness color (RRC) and reliability score value (S).

For example, when the maximum perpendicular length M of first order image grid and horizontal length N are the individual local unit in 8 (=8 * 8), the maximum perpendicular length M 2 and the horizontal length N2 of second level image grid are the individual local unit in 16 (=16 * 16), and the maximum horizontal length M 3 and the vertical length N3 of third level image grid are the individual local unit in 32 (=32 * 32).

Wherein, (i j) is represented as regional representativeness color and reliability score value C to certain unit Cell of third level image grid ³ _Ij, S ³ _Ij

At this moment, the division number of each image level of the first order, the second level and the third level is to determine according to the aspect ratio of image, thereby accurately represents to be included in the position of the object in the image.That is, under the situation of longer sides, divide longer sides equably, and divide than minor face with the dividing unit of longer sides.

In the another kind of method that is used for producing visual grid,, can vertically be set to identical with horizontal length in order to improve the approximate location information of the object that comprises in processing speed and the consideration image.

Below the visual searching method that uses the multi-level image raster data structure is described.

The different images that are divided into the multi-level image grid are represented as the representative area color (RRC) of representing the zone and the reliability score value of representing the precision of representative colors, a pair of representative area color and reliability score value and another are to coupling, and come the computing unit similarity, thereby carry out the image search according to the content-based inquiry of user.

By relatively being included in the unit in every grade the visual grid and represent the field color (RRC) of each unit, use the color similarity between two images of multi-level image raster data structure calculating.That is, use color similarity Color_Sim (RRC_C1, RRC_C2) color similarity between two unit of calculating of the similarity of regional representativeness color value between representative unit C1 and the unit C2.

First flexible strategy (α) multiply by color similarity Color_Sim (RRC_C1, RRC_C2), with color similarity Color_Sim (RRC_C1, RRC_C2) and second flexible strategy (β) and with respect to the product of the similarity I of reliability between two unit and the result of product addition of the color similarity and first flexible strategy.Resulting additive value divided by first flexible strategy and second flexible strategy then by normalization, thereby obtain two unit C1, the unit similarity Cell_Sim of C2 (C1, C2).Above-mentioned computing can be expressed as follows.

$\mathrm{Cell}\_\mathrm{Sim}(\mathrm{<figure-callout\; id="1"\; label="C"\; filenames="C0080011000161.png"\; state="\{\{state\}\}">C</figure-callout>}1,C2)=\frac{(\mathrm{\&alpha;}+\mathrm{\&beta;}\&times;I)\&times;\mathrm{Color}\_\mathrm{Sim}(\mathrm{PRC}\_\mathrm{<figure-callout\; id="1"\; label="C"\; filenames="C0080011000161.png"\; state="\{\{state\}\}">C</figure-callout>}1,\mathrm{PRC}\_C2)}{(\mathrm{\&alpha;}+\mathrm{\&beta;})}......\left(1\right)$

Wherein, (S1, similarity I S2) obtains according to I=1-|S1-S2| the reliability between two unit.

Therefore, with respect to the unit similarity between the part between the same stages of multi-level image and two different multi-level image grids of coupling not at the same level, and the feature between the comparison picture.

Fig. 2 represents to search for and have according to the image of use multi-level image raster data structure of the present invention two visual I of 3 grades of visual raster data structures ₁And I ₂Grid between the embodiment based on the search of similarity.

Two visual I ₁And I ₂Comprise first order image grid G _{1_1st}, G _{2_1st}, second level image grid G _{1_2nd}, G _{2_2nd}And third level image grid G _{1_3rd}, G _{2_3rd}

To the similarity Grid_Sim (G between two lattice level that comprise in two images ₁, G ₂) carry out inter-stage relatively.Above-mentioned computing can be expressed as follows.

Grid_Sim(G ₁，G ₂)＝w ₁×Sim_of_the_Exact _{G1_1st_and_G2_1st}

+w ₂×Sim_of_the_Exact _{G1_2nd_and_G2_2nd}

+w ₃×Sim_of_the_Exact _{G1_3rd_and_G2_3rd}

+w ₄×Sim_of_the_Inter _{G1_1st_and_G2_2nd}

+w ₅×Sim_of_the_Inter _{G1_2nd_and_G2_3rd} -----(2)

+w ₆×Sim_of_the_Inter _{G1_3rd_and_G2_1st}

+w ₇×Sim_of_the_Inter _{G1_1st_and_G2_3rd}

+w ₈×Sim_of_the_Inter _{G1_2nd_and_G2_1st}

+w ₉×Sim_of_the_Inter _{G1_3rd_and_G2_2nd}

Wherein for the flexible strategy of color similarity separately, Sim_of_the_Exact represents with respect to two visual I w1 to w9 representative ₁, I ₂Similarity between identical visual lattice level, Sim_of_the_Inter represents with respect to two visual I ₁, I ₂Similarity between the different visual lattice level.

That is, obtain two visual I of difference according to as shown in Figure 3 coupling ₁And I ₂In similarity Sim_of_the_Exact between the identical visual lattice level that comprises.And, obtain two visual I of difference according to as shown in Figure 4 coupling ₁And I ₂In similarity Sim_of_the_Inter between the different visual lattice level that comprises.

With reference to Fig. 5 A and 5B aforesaid operations is elaborated below.

Will be corresponding to the similarity addition of two unit of the same stages of two different images, and by at level and vertical direction by the aspect ratio displacement, the similarity of two unit is added to the aggregate values of gained.

At this moment, by the absolute value of the difference of certain grade aspect ratio of two images is added one, calculate the number of the coupling of two grids.

For example, shown in Fig. 5 A, suppose visual I ₁The number of grid of aspect ratio be M * N, visual I ₂The number of grid of aspect ratio be O * P, the coupling sum between two grids be (| M-O|+1) * (| N-P|+1).

By according to the aspect ratio of two grids based on two grids of different shift amounts couplings, calculate corresponding to identical lattice level Max (M, N)=Max (O, the similarity between two unit P).

At this moment, according to following formula 3-1,3-2 obtains based on two visual I ₁And I ₂Same stages between the similarity Sim_of_the_Exact of coupling.

Sim_of_the_Exact＝Max(Sim_bet_two_levels_given_cell_corresS(i，j))

Vi，0≤i≤|M-O|

Vj，0≤i≤|N-P| ------(3-1)

Sim_bet_two_levels_given_cell_corres?S(i，j)

$=\frac{\underset{y=o}{\overset{\mathrm{Min}|N-P|-1}{\mathrm{\&Sigma;}}}(\underset{x=o}{\overset{\mathrm{Min}|M-O|-1}{\mathrm{\&Sigma;}}}\mathrm{Sim}\_\mathrm{of}\_\mathrm{corres}\_\mathrm{two}\_\mathrm{cells}(x,y,i,j)}{\mathrm{Min}(N,P)\&times;\mathrm{Min}(M,O)}.....(3-2)$

When the similarity (Sim_of_the_Exact) between the coupling same stages, above-mentioned formula

$=\underset{y=o}{\overset{\mathrm{Min}|N-P|-1}{\mathrm{\&Sigma;}}}(\underset{x=o}{\overset{\mathrm{Min}|M-O|-1}{\mathrm{\&Sigma;}}}\mathrm{Sim}\_\mathrm{of}\_\mathrm{corres}\_\mathrm{two}\_\mathrm{cells}(x,y,i,j)$ Representative is with respect to the summation of the coupling of the level of two corresponding units and vertical edges.

By based on aspect ratio M: N, O: P is suitable for formula 404 to formula 4-1, can obtain two similarity Sim_of_corres_two_cells between the unit.

Sim(Cell ^G1(x+i，y+j)，cell ^G2(x，y))， if (Min(N，P)＝P) ⌒(Min(M，O)＝O)…(4-1)

Sim(Cell ^G1(x+i，y)， cell ^G2(x，y+j))， if (Min(N，P)＝N) ⌒(Min(M，O)＝O)…(4-2)

Sim(Cell ^G1(x，y+i)， cell ^G2(x+i，y))， if (Min(N，P)＝P) ⌒(Min(M，O)＝M)…(4-3)

Sim(Cell ^G1(x，y)， cell ^G2(x+i，y+j))，if (Min(N，P)＝N) ⌒(Min(M，O)＝M)…(4-4)

Wherein, when P less than N and M application of formula 4-1 during less than O, work as grid G ₁Length N less than grid G ₂Length P and grid G ₂Width O less than grid G ₁Width M the time application of formula 4-2.And, work as grid G ₂Vertical length P less than grid G ₁N and grid G ₁Horizontal length M less than G ₂O the time application of formula 4-3, work as G ₁N less than G ₂Its M of P application of formula 4-4 during less than O.

At this moment, will with respect to the length difference between the length of grid G 1 and grid G 2 (| M-O|, | shift amount N-P|) (i, j) be added to the unit coordinate (x, y), and each starting point (x y) becomes 0 for i, i.

By mate two visual lattice level of difference calculate different lattice level (Max (and M, N) ≠ Max (O, P) the similarity Sim_of_the_Inter between.This operation is similar to the search of lattice level similarity Sim_of_the_Exact.

In addition, based on | M-O|+1) * (| N-P|+1) obtain the number of the coupling of the visual grid between the different visual lattice level.

Carry out the color region matching operation, with the similar zone of representative color value between the search multi-level image grid.This search is carried out based on following method, a kind of method be between the lattice level (accurately ratio coupling) that is used for from identical size shift one's position and relative position is searched for color similarity, a kind of method is to be used for from shifting one's position and relative position search color similarity between the lattice level (mating between ratio) of different sizes.

That is, carry out color region matching operation between the visual lattice level (accurately ratio coupling) of identical size from the method for the color region of target image search same stages based on a kind of being used for.The identical visual lattice level of based target image is calculated the similarity of color region then with this position and relative position coupling, and the shifting one's position of same stages of this position and target image mated, thereby calculates the similarity of color region.

Based on a kind of be used for search between target image not the method for color region at the same level carry out color region matching operation between the different visual lattice level (mating between ratio), and in the different visual lattice level of target image, calculate the similarity of the color region of same stages.

In the color region matching process between the visual lattice level of difference, by the position is mated with the same position in the different visual lattice level of target image, calculate the similarity of color region, and, calculate the similarity of color region by the shifting one's position of another grade of position and target image mated.

As mentioned above, in the present invention, a visual raster data structure is divided into multistage raster data structure.Therefore, when content-based visual of the multistage lattice structure search of use dividing, might respond user's subjectivity inquiry efficiently.In addition, under certain condition, visual search speed is fast, precision is high.

Though, it will be understood by those skilled in the art that under the condition of the scope and spirit that do not depart from claims for the illustration purpose discloses the preferred embodiments of the present invention, can carry out various improvement, add and replacement.

Claims (9)

1. visual searching method that uses the multi-level image data structure, this data structure represents that based on the visual grid with at least two different classifications this method may further comprise the steps:

Determine the color similarity between reference picture and the target image, wherein reference picture and target image are all represented by the lattice level of classification; With

The inquiry content-based according to the user comes search image.

2. according to the process of claim 1 wherein that described determining step comprises: the unit that comprises in the unit that comprises in the lattice level of match reference image and the lattice level of target image.

3. according to the process of claim 1 wherein that described determining step comprises: each lattice level of the lattice level of match reference image and target image; And the lattice level of the lattice level of cross-matched reference picture and target image.

4. according to the process of claim 1 wherein that described determining step comprises: the regional representativeness color value between the lattice level of match reference image and target image.

5. according to the process of claim 1 wherein that in described each visual lattice level each all is divided into a plurality of unit, the step of the color similarity between described definite reference picture and the target image comprises the steps:

Will and each lattice level of reference picture and target image in the corresponding color similarity of similarity of regional representativeness color between the unit that comprises on duty with one first flexible strategy;

Described color similarity is on duty with a similarity and one second flexible strategy of representing the similarity of the reliability between the described unit; With

Calculate weighted mean value with normalization unit similarity.

6. according to the method for claim 5, wherein also comprise: when having compared two grids and having calculated similarity, based on by according to the shift amount that obtains by width between the lattice level and difference in height in the total value that level and vertical direction displacement are added up to, determine the similarity between the same stages grid in reference picture and the target image.

7. also comprise according to the process of claim 1 wherein: based on by the width between the lattice level and difference in height in the value that level and vertical direction displacement are added up to, determine the color similarity between the grid in reference picture and the target image.

8. according to the method for claim 4, wherein carrying out by matching area representative colors similarity under the situation of search, the unit similarity between the lattice level in use reference picture and the target image is searched for same position and the diverse location between the identical lattice level.

9. according to the method for claim 4, wherein, the operation of carrying out regional representativeness color value coupling between the lattice level of reference picture and target image is same position and the enterprising line search of diverse location in different lattice level.

Images