MULTILEVEL IMAGE GRID DATA STRUCTURE AND IMAGE SEARCH METHOD USING THE SAME
BACKGROUND OF THE INVENTION
1 Field of the Invention
The present invention relates to an image grid data structure and an image search method using the same, and in particular to a multilevel image grid data structure having a structure of different hierarchical grid levels with respect to one color feature related to a spatial color property of a still image and an image search method for searching an image using a multilevel image grid data structure
2 Description of the Background Art In a conventional image search method, a feature such as a color, shape, texture, etc is expressed in an image grid data structure of one level, and a similarity between different image data of the same structure is searched using an image grid data of one level for thereby searching the image
When searching an image in accordance with a conventional image search method, the importance of each feature is different in accordance with the characteristics of an image which will be searched In addition, even with respect to only one feature, the importance is different for each cell in the conventional image grid data structure For example, in the image search method using a color histogram, which is formed in a n-dimensional structure, a weight value reflecting the importance of each element can be determined as a different value for each element forming the n-dimensional structure
Namely, in the conventional image search method using an image
data stucture of one level, the importance between features is expressed based on the corresponding grid. In this case, however, the importance for each element of a certain feature is not considered. In order to resolve this problem, another conventional image search method adopts a method for computing an average importance of the elements in a certain feature
However, in the above-described conventional image search method, the average importance for elements of a certain feature is not useful, i.e , a predetermination of an average value for elements of a certain feature is not useful in image search since the importance of each element caries by a reference image of target image
In addition, since the conventional image grid data structure is formed of only one level, the destination contained in an image (or target image) is not accurately searched in the conventional image search method
SUMMARY OF THE INVENTION
Accordingly, it is an object of the present invention to provide a data structure in which each level is expressed by the cells of a hierarchical structure of different levels by expressing one feature based on a multilevel image grid, and expressing a region representative color of each cell and a reliability with respect to the region representative color
It is another object of the present invention to provide an image search method capable of matching between cells of the same level of two image grids, different levels of grids, and color regions to perform a color similarity retrieval with respect to multilevel image grids corresponding to different images
To achieve the above objects, there is provided a multilevel image data structure according to the present invention in which a spatial color feature
of one image is expressed in a hierarchical image grid structure having more than two different levels
To achieve the above objects, there is provided an image search method using a multilevel image data structure according to the present invention in which the color similarities of a spatial color feature of a reference image divided into different hierarchical image grid levels and a target image are matched, so that an image is searched in accordance with user's content-based query
Additional advantages, objects and features of the invention will become more apparent from the description which follows
BRIEF DESCRIPTION OF THE DRAWINGS
The present invention will become more fully understood from the detailed description given hereinbelow and the accompanying drawings which are given by way of illustration only, and thus are not limitative of the present invention, and wherein
Figure 1 is a view illustrating an embodiment of a multilevel image grid data structure and a 3-level image grid data structure according to the present invention, Figure 2 is a view illustrating an image search method using a multilevel image grid data structure and the construction of a match between 3-level image grid data structures according to the present invention,
Figure 3 is a view illustrating an embodiment of an image search method using a multilevel image grid data structure and the construction of a match between the same levels in a 3-level image grid data structure according to the present invention,
Figure 4 is a view illustrating an embodiment of an image search
method using a multilevel image grid data structure and the construction of a match between different levels of a 3-level image grid data structure according to the present invention, and
Figures 5A and 5B are views illustrating an embodiment of an image search method using a multilevel image grid data structure according to the present invention, of which Figure 5A is a view illustrating two same image grid data structures, and Figure 5B is a view illustrating a process of a match of two image grid data structures
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
The present invention relates to a multilevel image grid data structure and an image search method using the same The method for generating a multilevel image grid data structure according to the present invention will be explained In the case of square image, it is uniformly divided by height and width, and in the case of a non-square image, one side is uniformly divided in accordance with an aspect ratio of a width and height of an image, and the other side is uniformly divided by the unit of one side Namely, a regular square structure having the same length of horizontal and vertical sides is divided by the same unit, and in the case of a rectangular structure having different lengths of horizontal and vertical sides, one sιde(for example, a lengthy side) is uniformly divided, and the other sιde(for example, a shorter side) is divided by the dividing
Therefore similarly as above, in one image data structure, the spatial color feature is divided into hierarchical grids of different levels for thereby expressing a structure of a multilevel image grid
At this time, each image grid is a hierarchical structure of different
levels, and the resolution of each level is hierarchically divided The cell of each grid is assigned with two values which are a regional representative color (RRC) and a reliability score (S) relating to an accuracy of the regional representative color Figure 1 illustrates an embodiment of a multilevel image grid data structure and a 3-level image grid data structure according to the present invention Namely, one image is expressed in an image grid level of a first level, second level, and third level
In the resolution of the 3-level image grid data structure, the first level image grid is the lowest, the second level image grid is an intermediate level and the third level image grid is higher than the second level image grid in accordance with the divided levels
The first level image grid is divided into the image region including a
M1xN1 number of local cells in proportion to the aspect ratio of a vertical side M and a horizontal side N Each cell is expressed as a region representative color(RRC) which represents each region, and a reliability score(S) which corresponds to the accuracy of the representative color value
In addition, the second level image grid and the third level image grid are divided into the image regions including a M2xN2 number and M3xN3 number of local cells in accordance with the dividing state, and each cell has a region representative color(RRC) and a reliability score(S)
For example, when the maximum vertical length M of the first level image grid and the horizontal length N are 8(=8x8), the maximum vertical length M2 of the second level image grid and the horizontal length N2 are 16(=16x16), and the maximum horizontal length M3 of the third level image grid and the vertical length N3 are 32(=32x32) of the local cells
Here, a certain cell Cell(ι ) of the third level image grid is expressed
as a region representative color and a reliability score C^., S^,,
At this time, the number of divisions of each of the image levels of 1 st level, second level and third level is determined based on an aspect ratio of the image for accurately expressing the position of the object included in the image Namely, in the case of the lengthy side, the lengthy side is uniformly divided, and the short side is divided by the divided unit of the lengthy side
In another method for generating the grid of the image, to increase processing speed and to consider approximate positional information of the object included in the image, the vertical and horizontal lengths may set identically
The image search method using the multilevel image grid data structure will be explained
Different images divided into the multilevel image grids are expressed as a representative region color(RRC) which represents the region and a reliability score which expresses an accuracy of the representative color, and a pair of representative region color and reliability are matched to another one, and a cell similarity is computed in accordance with the content-based query of a user for thereby performing an image search
The color similarity between two images is computed using the multilevel image grid data structure by comparing the cells included in an image grid of each level and the region color(RRC) representing each cell Namely, the color similarity between two cells is computed using the color similarities Color_Sιm(RRC_C1 , RRC_C2) which represent the similarity of a region representative color value between the cell C1 and Cell C2 The first weight (α) is multiplied by the color similarities
Color_Sιm(RRC_C1 , RRC_C2), and a result of the multiplication of the color similarities Color_Sιm(RRC_C1 , RRC_C2) and the second weight (β)
and the similarity I with respect to a reliability between two cells is summed by the result obtained by multiplying the color similarity and the first weight The thusly summed value is divided by the first weight and second weight and then is normalized, so that the cell similarity Cell_Sιm(C1 , C2) of two cells C1 , C2 is obtained The above-described operation may be expressed as follows
Here, the similarity I of the relιabιlιty(S1 , S2) between two cells is obtained based on 1=1 -|S1-S2|
Therefore, the cell similarities between two different multilevel image grid are matched with respect to the portions between the same levels of the multilevel image and the different levels, and a feature between the images is compared Figure 2 illustrates an embodiment of the image search using a multilevel image grid data structure according to the present invention and a similarity- based search between the grids of two images ^ and I2 having a 3-level image grid data structure
Two images l-| and I2 include first level image grids G1 ι sr G2 -) st second level image grids G^ 2nd- ^2 2nd' and tnιrd level ιmage gπds G-| 3rcj,
G2_3rd
The similarities Grιd_Sιm(G<| , G2) between grid levels included in two images are compared between the levels The above-descπbe operation may be expressed as follows
Grid _ Sim (G, , G2 ) = w] x Szm _ of _ the _ Exact G] , and C2 t + w2 x Sim _ of the _ Exact G] 2nd and G2_2nd + w3 x Sim _of _ the _ Exact G] 3rd and_G2 ird + w4xSιm_of the _ Inter G]_lst and G2 2nd + w5x Sim _of the _ Inter G]_2nd and G2 3rd + w6x Sun _ of _ the Inter GX ird_and G2 st + w7x Sim of _ the __ Inter G1 l and G2 2rd + wixSιm _of _the _Inter G1 2nd and G2 l + w9x Sim _of _ the _ Inter G1 3rd and G2 2nd
(2)
where w1 through w9 represent weights with respect to the respective color similarity, and Sιm_of_the_Exact represents a similarity between the same image grid levels with respect to two images l-|, I2, and Sιm_of_the_lnter represents a similarity between different image grid levels with respect to two images l-|, I2
Namely, the similarity Sιm_of_the_Exact between the same image grid levels included in two different images 11 and 12 is obtained based on the match as shown in Figure 3 In addition, the similarity Sιm_of_the_lnter between different image grid levels included in two different images 11 and 12 is obtained based on the match as shown in Figure 4
The above-described operation will be explained in more detail with reference to Figures 5A and 5B
The similarities of two cells corresponding to the same levels of two different images are summed, and the similarities of two cells are summed to the thusly summed value by shifting in the horizontal and vertical directions by the aspect ratio
At this time, the number of the matches of two grids is computed by adding 1 to the absolute value of the difference of the aspect ratio of a certain level of two images.
For example, as shown in Figure 5A, assuming that the number of the grids of the aspect ratio of the image l-j is MxN, and the number of the grids of the aspect ratio of the image I2 is OxP, the total number of matches between two grids is (|M-0|)+1 )x(|N-P|+1 ).
The similarity between two cells corresponding to the same grid levels Max(M,N)=Max(O,P) is calculated by matching two grids based on different shift amount in accordance with the aspect ratio of two grids.
At this time, the similarity Sim_of_the_Exact based on the match between the same levels of two images 11 and 12 is obtained based on the following Equations 3-1 , 3-2.
Szm _ of _ the _Exact = Max (Sim _ bet __ two _ levels _ given _ cell _ corres S(i, j)) Vi, 0 < i < |M -O|
Vj,0 < i < |N -P|
(3-1)
Sim _bet _two _levels _gιven _cell corres S(i, j)
Mιn|N-Pι-l MιnιM-O - 1 ( Sim _of corres _two _cells(x,y,ι,j) v=o
Mιn(N,P)xMin(M,0)
(3-2)
When matching the similarity (Sim_of_the_Exact) between the same
levels, the above-described equation of corres _two _cells)
represents a sum of the matching with respect to the horizontal and vertical
sides of two corresponding cells
The similarity Sim_of_corrres_two_cells between two cells is obtained by
adapting Equation 404 to Equation 4-1 based on the aspect ratios M:N, O:P.
Sim(cei (x + i,y + j), cellG2 (x, y)), if(Min(N, P) = P) (Min(M, 0) = 0)
(4-1)
Sim(cei (x + 1, y), cell01 (x, y + j)), if(Min(N, P) = N) (Min(M ,0) = O)
(4-2)
Sim(cei (x,y + i),ceir2 (x + i,y)), if(Min(N,P) = P)rΛ (Min(M,0) =M)
(4-3)
Sιm(cei (x,y),celr2(x + ι,y + j)), ιf(Mm(N,P) = N) (Mm(M,0) = M)
(4-4)
Here, Equation 4-1 is applied when P is less than N and M is less than O and Equation 4-2 is applied when the length N of the grid Gi is shorter than length P of grid G2 and the width O of the grid G2 is shorter than width M of the grid G1 In addition, Equation 4-3 is applied when the vertical length P of the grid G2 is shorter than N of grid G1 and the horizontal length M of the grid G1 is shorter than O of G2, and Equation 4-4 is applied when N of G-| is shorter than P of G2 and M is shorter than O At this time, the shift amount (ι,j) with respect to the length difference (|M-
0|,|N-P|) between the length of the grid G-| and the grid G2 is added to the cell coordinate (x, y), and each of start point (1,1, x,y) becomes 0
The similarity Sιm_of_the_lnter between different grid levels(Max(M,N)
= =Max(O,P) is calculated by matching two different image grid levels This operation is performed similarly as the search of the grid level similarity
Sιm_of__the_Exact
In addition, the number of the matches of the image grids between different image grid levels is obtained based on (|M-O|+1 )x(|N-P|+1 )
The color region matching operation is performed for searching the region in which the representative color values are similar between the multilevel image grids The search is performed based on a method for searching the color similarity from a translation position and a relative position between the grid level(Exact scale matching) of the same size, and a method for searching the color similarity from a translation position and the relative position between the grid levels( Inter-scale matching) of different sizes
Namely, the color region matching operation between the image grid levels(Exact scale matching) of the same size is performed based on a method for searching a color region of the same levels from a target image The position is matched with the relative position based on the same image grid level of the target image, and then the similarity of the color region is computed, and the position is matched with a translation position at the same level of the target image for thereby computing a similarity of the color region
The color region matching operation between the different image grid levels(lnter-scale matching) is performed based on a method for searching the different level color regions among the target images, and a similarity of the color region of the same level is computed among the different image grid levels of the target image
In the color region matching method of between different image grid levels, the similarity of the color region is computed by matching the position with the same position among the different image grid levels of the target image, and the similarity of the color region is computed by matching the position with the translation position at another level of the target image
As described above, in the present invention, one image grid data structure is divided into multilevel grid data structures Therefore, it is possible to effectively response with respect to a subjective query by a user when searching a content-based image using the divided multilevel grid structures In addition, an image search speed is fast and accurate under a certain condition
Although the preferred embodiment of the present invention have been disclosed for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the invention as recited in the accompanying claims