AU2464600A

AU2464600A - Image texture retrieving method and apparatus thereof

Info

Publication number: AU2464600A
Application number: AU24646/00A
Authority: AU
Inventors: Yang Lim Choi; B. S. Manjunath; Hyun Doo Shin; Peng Wu
Original assignee: Samsung Electronics Co Ltd; University of California
Current assignee: Samsung Electronics Co Ltd; University of California
Priority date: 1999-02-05
Filing date: 2000-02-03
Publication date: 2000-08-25
Anticipated expiration: 2020-02-03
Also published as: JP2004158042A; KR100444776B1; DE60037919T2; DE60036082T2; EP1453000B1; EP1153365B1; US7199803B2; MY138084A; CN1341248A; BR0007956A; KR100483832B1; KR20040023676A; DE60037919D1; EP1777658B1; DE60045319D1; KR100444777B1; EP1153365A4; JP2002536750A; CA2625839A1; EP1153365A1

Abstract

A method for retrieving an image texture descriptor for describing te

Description

WO 00/46750 PCT/KROO/00091 IMAGE TEXTURE RETRIEVING METHOD AND APPARATUS THEREOF Technical Field 5 The present invention relates to a method and apparatus for retrieving an image texture descriptor, and more particularly, to an image texture descriptor retrieving method for retrieving a texture descriptor which is used in searching and browsing an image and describes texture characteristics of the image, and an apparatus thereof. 10 Background Art Recently, image texture has emerged as important visual features for searching and browsing a large set of similar image patterns. For example, a conventional texture descriptor for filtering a texture descriptor by a Gabor filter 15 extracts a texture descriptor consisting of coefficients obtained by Gabor filtering. However, although conventional image texture descriptors consist of numerous vectors, it is quite difficult to visually perceive texture structures from the texture descriptor. 20 Disclosure of the Invention It is an object of the present invention to provide a method for retrieving an image texture descriptor which can perceptually capture the texture structures present in an image. 25 It is another object of the present invention to provide a computer readable storage medium having a computer program stored therein, the program being arranged such that a computer executes the image texture descriptor retrieving method. It is still another object of the present invention to provide an image texture 30 descriptor retrieving apparatus which performs the image texture descriptor retrieving method.

WO 00/46750 PCT/KROO/00091 2 To achieve the above object, there is provided a method for retrieving an image texture descriptor for describing texture features of an image, including the steps of (a) filtering input images using predetermined filters having different orientation coefficients, (b) projecting the filtered images onto axes of each 5 predetermined direction to obtain data groups consisting of averages of each directional pixel values, (c) selecting candidate data groups among the data groups by a predetermined classification method, (d) determining a plurality of indicators based on orientation coefficients of the filters used in filtering the candidate data groups, and (e) determining the plurality of indicators as the texture descriptor of the 10 image. The step (a) may further include the step of (a-1) filtering input images using predetermined filters having different scale coefficients, and the step (d) further comprises the step of (d-1) determining a plurality of indicators based on scale coefficients of the filters used in filtering the candidate data groups. 15 The image texture descriptor retrieving method may further include the step of determining another indicator based on the presence of data groups filtered by filters having scale coefficients or orientation coefficients which are close to or identical with the scale coefficients or orientation coefficients of the filters used in filtering the selected candidate data groups. 20 The image texture descriptor retrieving method may further include the step of calculating the mean and variance of pixels with respect to the filtered images, and obtaining a predetermined vector using the calculated mean and variance. According to another aspect of the present invention, there is provide a method for retrieving an image texture descriptor for describing texture features of 25 an image, includes the steps of (a) filtering input images using predetermined filters having different scale coefficients, (b) projecting the filtered images onto axes of each predetermined direction to obtain data groups consisting of averages of each directional pixel values, (c) determining a plurality of indicators based on scale coefficients of the filters used in filtering data groups selected among the data groups 30 by a predetermined selection method, (d) determining the plurality of indicators as the texture descriptor of the image.

WO 00/46750 PCT/KROO/00091 3 According to still another aspect of the present invention, there is provided method for retrieving an image texture descriptor for describing texture features of an image, comprising the steps of (a) filtering input images using predetermined filters having different orientation coefficients and different scale coefficients, (b) 5 projecting the filtered images onto horizontal and vertical axes to obtain horizontal axis projection graphs and vertical-axis projection graphs, (c) calculating normalized auto-correlation values for each graph, (d) obtaining local maximums and local minimum for each normalized auto-correlation value, at which the calculated normalized auto-correlation values forms a local peak and a local valley at a 10 predetermined section, (e) defining the average of the local maximums and the average the local minimums as contrast, (f) selecting graphs in which the ratio of the standard deviation to the average of the local maximums is less than or equal to a predetermined threshold as first candidate graphs, (g) determining the type of the second candidate graphs according to the number of graphs filtered by the filters 15 having scale coefficients or orientation coefficients which are close to or identical with the scale coefficients or orientation coefficients of the filters used in filtering the selected second candidate graphs, (h) counting the numbers of graphs belonging to the respective types of second candidate graphs and determining predetermined weights of each type of second candidate graphs, (i) calculating the sum of products 20 of the counted numbers of graphs and the determined weights to determine the calculation result value as a first indicator constituting a texture descriptor, (j) determining the orientation coefficients and scale coefficients of the second candidate graphs having the biggest contrast as second through fifth indicators, and (k) determining indicators including the first indicator and the second through fifth 25 indicators as the texture descriptors of the corresponding image. The image texture descriptor retrieving method may further include the step of calculating the mean and variance of pixels with respect to the filtered images, and obtaining a predetermined vector using the calculated mean and variance, wherein the step (k) includes the step of determining indicators including the first indicator, 30 the second through fifth indicators and the predetermined vector as the texture descriptors of the corresponding image.

WO 00/46750 PCT/KROO/00091 4 The normalized auto-correlation, denoted by NAC(k), is preferably calculated by the following formula: N-1 Y, P(m -k) P(m) NAC(k)- -r=k I P2(m-k)L P2(m) m= k m=k wherein N is a predetermined positive integer, an input image consists of NxN 5 pixels, a pixel position is represented by i, where i is a number from 1 to N, the projection graphs expressed by pixels of the pixel position i is represented by P(i) and k is a number from 1 to N. The contrast is determined as: 1 I 1 L contrast = - P_ magn(i) - - Vmagn(i) M -= L i=1 10 wherein Pmagn (i) and Vmagn (i) are the local maximums and local minimums determined in the step (d). In the step (f), the graphs satisfying the following formula are selected as first candidate graphs: S d 15 wherein d and S are the average and standard deviation of the local maximums and a is a predetermined threshold. The step (g) includes the sub-steps of (g-1) if there are one or more graphs having scale or orientation coefficients identical with those of a pertinent candidate graph and one or more graphs having scale or orientation coefficients close to those 20 of the pertinent candidate graph, classifying the pertinent candidate graph as a first type graph, (g-2) if there are one or more graphs having scale or orientation coefficients identical with those of a pertinent candidate graph but there is no graph having scale or orientation coefficients close to those of the pertinent candidate WO 00/46750 PCT/KROO/00091 5 graph, classifying the pertinent candidate graph as a second type graph, and (g-3) if there is no graph having scale or orientation coefficients identical with or close to those of a pertinent candidate graph, classifying the pertinent candidate graph as a third type graph. 5 The step (h) includes the step of counting the number of graphs belonging to each of the first through third types of graphs and determining predetermined weights for each of the types of graphs. After the step of (f), there may be further included the step of applying a predetermined clustering algorithm to the first candidate graphs to select second 10 candidate graphs. The predetermined clustering algorithm is preferably modified agglomerative clustering. Preferably, in the step (j), the orientation coefficient of a graph having the biggest contrast, among the horizontal-axis projection graphs, is determined as a 15 second indicator; the orientation coefficient of a graph having the biggest contrast, among the vertical-axis projection graphs, is determined as a second indicator; the scale coefficient of a graph having the biggest contrast, among the horizontal-axis projection graphs, is determined as a fourth indicator; and the scale coefficient of a graph having the biggest contrast, among the vertical-axis projection graphs, is 20 determined as a fifth indicator. The step (j) may include the step of determining indicators including the first indicator, the second through fifth indicators and the predetermined vector as the texture descriptors of the corresponding image. The predetermined filters preferably include Gabor filters. 25 To achieve the second object of the present invention, there is provided a computer readable medium having program codes executable by a computer to perform a method for an image texture descriptor for describing texture features of an image, the method including the steps of (a) filtering input images using predetermined filters having different orientation coefficients and different scale 30 coefficients, (b) projecting the filtered images onto horizontal and vertical axes to obtain horizontal-axis projection graphs and vertical-axis projection graphs, (c) WO 00/46750 PCT/KROO/00091 6 calculating normalized auto-correlation values for each graph, (d) obtaining local maximums and local minimums for each of normalized auto-correlation values, at which the calculated normalized auto-correlation value forms a local peak and a local valley at a predetermined section, (e) defining the average of the local maximums and 5 the average the local minimums as contrast, (f) selecting graphs in which the ratio of the standard deviation to the average of the local maximums is less than or equal to a predetermined threshold as first candidate graphs, (g) determining the type of the second candidate graphs according to the number of graphs filtered by the filters having scale coefficients or orientation coefficients which are close to or identical 10 with the scale coefficients or orientation coefficients of the filters used in filtering the selected second candidate graphs, (h) counting the numbers of graphs belonging to the respective types of second candidate graphs and determining predetermined weights of each type of second candidate graph, (i) calculating the sum of products of the counted numbers of graphs and the determined weights to determine the 15 calculation result value as a first indicator constituting a texture descriptor, (j) determining the orientation coefficients and scale coefficients of the second candidate graphs having the biggest contrast as second through fifth indicators, and (k) determining indicators including the first indicator and the second through fifth indicators as the texture descriptors of the corresponding image. 20 To achieve the third object of the present invention, there is provided an apparatus method for retrieving an image texture descriptor for describing texture features of an image, the apparatus including filtering mean for filtering input images using predetermined filters having different orientation coefficients, projecting means for projecting the filtered images onto axes of each predetermined direction to obtain 25 data groups consisting of averages of each directional pixel values, classifying means for selecting candidate data groups among the data groups by a predetermined classification method, first indicator determining means for determining another indicator based on the number of graphs filtered by filters having scale coefficients or orientation coefficients which are close to or identical with the scale coefficients 30 or orientation coefficients of the filters used in filtering the selected candidate graph, and second indicator determining means for determining a plurality of indicators WO 00/46750 PCT/KROO/00091 7 based on scale coefficients and orientation coefficients of the filters used in filtering the determined candidate graphs. Alternatively, there is provided an apparatus for retrieving an image texture descriptor for describing texture features of an image, the apparatus including a 5 filtering unit for filtering input images using predetermined filters having different orientation coefficients and different scale coefficients, an image mean/variance calculating unit for calculating the mean and variance of pixels with respect to each of the filtered images, and obtaining a predetermined vector using the calculated mean and variance, a projecting unit for projecting the filtered images onto horizontal 10 and vertical axes to obtain horizontal-axis projection graphs and vertical-axis projection graphs, a calculating unit for calculating a normalized auto-correlation value for each graph, a peak detecting/analyzing unit for detecting local maximums and local minimums for each auto-correlation value, at which the calculated normalized auto-correlation values forms a local peak and a local valley at a 15 predetermined section, a mean/variance calculating unit for calculating the average of the local maximums and the average the local minimums, a first candidate graph selecting/storing unit for selecting the graphs satisfying the requirement that the ratio of the standard deviation to the average of the local maximums be less than or equal to a predetermined threshold, as first candidate graphs, a second candidate graph 20 selecting/storing unit for applying a predetermined clustering algorithm to the first candidate graphs to select the same as second candidate graphs, a classifying unit for counting the number of graphs belonging to each of the respective types of the second candidate graphs, outputting data signals indicative of the number of graphs of each type, determining predetermined weights of the graphs belonging to the 25 respective types and outputting data signals indicative of weights to be applied to each type, a first indicator determining unit for calculating the sum of the products of the data representing the number of graphs belonging to each type, and the data representing the weights to be applied to each type, determining and outputting the calculation result as a first indicator constituting a texture descriptor, a contrast 30 calculating unit for calculating the contrast according to formula (2) using the averages output from the mean/variance calculating unit and outputting a signal WO 00/46750 PCT/KROO/00091 8 indicating that the calculated contrast is biggest, a second candidate graph selecting/storing unit for outputting the candidate graphs having the biggest contrast among the second candidate graphs stored therein in response to the signal indicating that the calculated contrast is biggest, a second-to-fifth indicator determining unit for 5 determining the orientation coefficient of a graph having the biggest contrast, among the horizontal-axis projection graphs; the orientation coefficient of a graph having the biggest contrast, among the vertical-axis projection graphs, as a second indicator; the scale coefficient of a graph having the biggest contrast, among the horizontal-axis projection graphs, as a fourth indicator; and the scale coefficient of a graph having 10 the biggest contrast, among the vertical-axis projection graphs, as a fifth indicator, and a texture descriptor output unit for combining the first indicator, the second through fifth indicators and the predetermined vector and outputting the combination result as the texture descriptors of the corresponding image. 15 Brief Description of the Drawings The above objects and advantages of the present invention will become more apparent by describing in detail preferred embodiments thereof with reference to the attached drawings in which: 20 FIGS. 1A and 1B are flow diagrams showing an image texture descriptor retrieving method according to the present invention; FIG. 2 is a block diagram of an image texture descriptor retrieving apparatus according to the present invention; and FIG. 3 shows perceptual browsing components (PBCs) extracted from 25 Brodatz texture images by simulation based on the image texture descriptor retrieving method according to the present invention. Best mode for carrying out the Invention 30 Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

WO 00/46750 PCT/KROO/00091 9 Referring to FIG. 1A showing an image texture descriptor retrieving method according to the present invention, assuming that N is a predetermined positive integer, an input image consisting of NxN pixels, for example, 128x128 pixels, is filtered using a Gabor filter (step 100). The Gabor filter is constituted by filters 5 having different orientation coefficients and different scale coefficients. Assuming that C1 and C2 are predetermined positive integers, the input image is filtered by filters having C1 kinds of orientation coefficients and C2 kinds of scale coefficients, and the filters output ClxC2 kinds of filtered images. Next, the mean and variance of pixels are calculated for each of the C1 xC2 10 kinds of filtered images, and then a vector Z is obtained using the mean and variance (step 102). Then, the filtered images are projected onto x- and y-axes to obtain x projection graphs and y-projection graphs (step 104). The normalized auto correlation (NAC) value for each graph P(i) (i is a number from 1 to N) denoted by 15 NAC(k), is calculated by the following formula (1): N-1 Y P(m- k)P(m) NAC(k)= (-1=k) P2(m-k) P2(m) nm=k wherein a pixel position is represented by i, the projection graphs expressed by pixels of the pixel position i are represented by P(i) and k is a number from 1 to N (N is a positive integer.). 20 Next, local maximums Pmagn (i) and local minimums of V magn (i), at which the calculated NAC(k) forms a peak and a valley locally at a predetermined section, are obtained (step 108). Now, contrast is defined as the following formula (2): contrast - E P_ magn(i)- V_ magn(i) ...... (2) M L 25 (step 110).

WO 00/46750 PCT/KRO0/00091 10 Also, the graphs satisfying the following formula (3) are selected as first candidate graphs (step 112): S -:! a ....... (3) d wherein d and S are the average and standard deviation of the local maximums 5 P-magni (i) and a is a predetermined threshold. Referring to FIG. 1B, modified agglomerative clustering is applied to the first candidate graphs to select second candidate graphs (step 114). A modified agglomerative clustering algorithm is an appropriately modified algorithm of agglomerative clustering disclosed by R.O. Duda and P.E. Hart in "Pattern 10 Classification and Scene Analysis, John Wiley and Sons, New York, 1973," which will now be described briefly. First, in N graphs Pi, . . . .

, PN, let the mean and standard deviation of distances between peaks be di and Si, and each graph have a two-dimensional vector corresponding to (di, S). Now, Pi is clustered using the two dimensional vector corresponding to (di, Si) as follows. Assuming that the desired 15 number of clusters is Me, with respect to initial number of clusters N, each cluster C can be expressed such that C 1 ={Pj}, C 2

={P

2 }, ... , CN ={PN. If the number of clusters is smaller than Me, clustering is stopped. Next, two clusters C and Cj which are most distant from each other are obtained. If the distance between C and Cj is greater than a predetermined threshold, clustering is stopped. Otherwise, C, and Ci 20 are merged to remove one of the two clusters. This procedure is repeatedly performed until the number of clusters reaches a predetermined number. Then, among the clustered clusters, the cluster having the most graphs is selected and graphs in the selected cluster are selected as candidate graphs. Now, the second candidate graphs are classified into three types (step 116). 25 The classification is performed according to the number of graphs filtered by a filter having scale or orientation coefficients which are close to or identical with those of a filter used for filtering the second candidate graphs. Hereinafter, for the convenience' sake of explanation, the graphs filtered by a filter having a certain scale coefficient or a constant orientation coefficient will be referred to as certain-scale 30 coefficient graphs or certain-orientation-coefficient graphs.

WO 00/46750 PCT/KROO/00091 11 In more detail, first, in the case where there are one or more graphs having scale or orientation coefficients identical with those of a pertinent candidate graph and one or more graphs having scale or orientation coefficients close to those of the pertinent candidate graph, the pertinent candidate graph is classified as a C1 type 5 graph. Second, in the case where there are one or more graphs having scale or orientation coefficients identical with those of a pertinent candidate graph but there is no graph having scale or orientation coefficients close to those of the pertinent candidate graph, the pertinent candidate graph is classified as a C2 type graph. Third, in the case where there is no graph having scale or orientation coefficients 10 identical with or close to those of a pertinent candidate graph, the pertinent candidate graph is classified as a C3 type graph. Then, the numbers of graphs belonging to each of the Cl, C2 and C3 types are counted to be denoted by N 1 , N 2 and N 3 , respectively, and the respective weights of the graphs belonging to each of the Cl, C2 and C3 types are counted to be denoted by W 1 , W, and W 3 , respectively, which 15 will be described below. Now, using the determined numbers N 1 , N 2 and N 3 , and the weights W,, W, and W 3 , the following calculation is performed: 3 M=Z N, x W ...... (4) i=1 wherein the result M is determined as a first indicator V, constituting a texture 20 descriptor (step 118). With respect to the second candidate graphs, the orientation coefficients and scale coefficients of graphs that have the biggest contrast are determined as second through fifth indicators (step 120). In more detail, the orientation coefficient of a graph having the biggest contrast, among the x-projection graphs, is determined as 25 a second indicator V 2 . Also, the orientation coefficient of a graph having the biggest contrast, among the y-projection graphs, is determined as a third indicator V 3 . The scale coefficient of a graph having the biggest contrast, among the x-projection graphs, is determined as a fourth indicator V 4 . Also, the scale coefficient of a graph having the biggest contrast, among the y-projection graphs, is determined as a fifth WO 00/46750 PCT/KROO/00091 12 indicator

V

5 . Using the first indicator V, determined in the step 118, the second through fifth indicators V 2 , V 3 , V 4 and V, and the vector Z determined in the step 102, the texture descriptor, that is, the texture feature vector, is set to {[V 1 , V 2 , V 3 , V 4 , V 5 1, 5 Z} (step 122). A large first indicator V, indicates a high level of structuredness of the texture of an image. It has been experimentally confirmed that the first indicator V, represents quite well the structuredness of the texture of an image. The second and third indicators V 2 and V 3 represent two quantized orientations in which the 10 structuredness is captured most. The fourth and fifth indicators V 4 and V 5 represent two quantized scales in which the structuredness is captured most. The texture descriptor is used as an index of an image in browsing or searching-retrieval applications. Especially, the image texture descriptor retrieved by the image texture descriptor retrieving method according to the present invention 15 is suitably used in checker marks in which browsing patterns are regular, or structure oriented browsing, i.e., or embroidery patterns. Thus, in searching structurally similar patterns, image searching which is more adaptable to eye-perception is allowed by applying the image texture descriptor retrieving method according to the present invention to the applications based on the structured oriented browsing. 20 Therefore, among indicators constituting texture descriptors retrieved by the image texture descriptor retrieving method according to the present invention, the first through fifth indicators V 1 , V 2 , V 3 , V 4 and V 5 can be referred to as perceptual browsing components (PBCs). Also, with respect to each filtered image, the mean and variance of pixel 25 values are calculated. The vector Z obtained by using the mean and variance can be referred to as similarity retrieval components (SRCs). In other words, in the image texture descriptor retrieving method according to the present invention, the texture descriptor allows kinds of texture structures present in an image to be perceptually captured. 30 It has been described that a first indicator V, which is a quite a good indicator of the structuredness of the texture of an image, second and third indicators V2 and WO 00/46750 PCT/KROO/00091 13

V

3 representing two quantized orientations in which the structuredness is captured most, fourth and fifth indicators V 4 and V 5 representing two quantized scales in which the structuredness is captured most, are used as the texture descriptors of the image. However, the above-described embodiment is used in a descriptive sense 5 only and not for the purpose of limitation. A single indicator that is most suitable to the characteristics of an image and arbitrarily selected plural indicators, can also be used as the texture descriptor(s) of the image. Therefore, the above-described embodiment is not intended as a restriction on the scope of the invention. Also, the image texture descriptor retrieving method is programmable by a 10 computer program. Codes and code segments constituting the computer program can be easily derived by a computer programmer in the art. Also, the program is stored in computer readable media and is readable and executable by the computer, thereby embodying the image texture descriptor retrieving method. The media include magnetic recording media, optical recording media, carrier wave media, and the like. 15 Also, the image texture descriptor retrieving method can be embodied by an image texture descriptor retrieving apparatus. FIG. 2 is a block diagram of an image texture descriptor retrieving apparatus according to the present invention. Referring to FIG. 2, the image texture descriptor retrieving apparatus includes a Gabor filer 200, an image mean/variance calculating unit 202, an x-axis projector 204, a y-axis 20 projector 205, an NAC calculating unit 206 and a peak detecting/analyzing unit 208. Also, the image texture descriptor retrieving apparatus includes a mean/variance calculating unit 210, a first candidate graph selecting/storing unit 212, a second candidate graph selecting/storing unit 214, a classifying unit 216, a first indicator determining unit 218, a contrast calculating unit 220, a second-to-fifth indicator 25 determining unit 222 and a texture descriptor output unit 224. In the operation of the image texture descriptor retrieving apparatus, assuming that N is a predetermined positive integer, the Gabor filter 200 filters an input image consisting of NxN pixels, for example, 128x128 pixels using filters (not shown) having different orientation coefficients and different scale coefficients, and outputs 30 filtered images (image-filtered). Assuming that Cl and C2 are predetermined positive integers, the input image is filtered by filters having C1 kinds of orientation WO 00/46750 PCT/KROO/00091 14 coefficients and C2 kinds of scale coefficients, and the filters output C1 xC2 kinds of filtered images. The image mean/variance calculating unit 202 calculates the mean and variance of pixels for each of the C1xC2 kinds of filtered images, to then obtain a 5 vector Z using the mean and variance and outputs the obtained vector Z. The x-axis projector 204 and the y-axis projector 205 project the filtered images onto x- and y-axes to obtain x-projection graphs and y-projection graphs. In other words, suppose a pixel position is represented by i (i is a number from 1 to N), the x-axis projector 204 and the y-axis projector 205 output the projection graphs P(i) 10 expressed by pixels of the pixel position i (i-1,..., N). The NAC calculating unit 206 calculates the normalized auto-correlation (NAC) value for each graph P(i), denoted by NAC(k), using the formula (1). The peak detecting/analyzing unit 208 detects local maximums P_magn (i) and local minimums of V_magn (i), at which the calculated NAC(k) forms a local 15 peak and a local valley at a predetermined section. The mean/variance calculating unit 210 calculates the mean d and standard deviation S of the local maximums P_magn (i) and outputs the same. The first candidate graph selecting/storing unit 212 receives the mean d and standard deviation S, selects the graphs satisfying the formula (3) as first candidate graphs (1stCAND) 20 and stores the selected first candidate graphs, in which a is a predetermined threshold. The second candidate graph selecting/storing unit 214 applies modified agglomerative clustering to the first candidate graphs to select the same as second candidate graphs (2nd_CAND). 25 The classifying unit 216, as described with reference to FIG. 1B, counts the numbers of graphs belonging to each of the C1, C2 and C3 types to denote the same by N 1 , N 2 and N 3 , respectively, with respect to the second candidate graphs, and outputs data signals N indicative of the number of graphs of each type. Also, the classifying unit 216 determines predetermined weights of the graphs belonging to 30 each of the C1, C2 and C3 types to then denote the same by W 1 , W 2 and W 3 , respectively, and outputs data signals W indicative of weights to be applied to each WO 00/46750 PCT/KROO/00091 15 type. The first indicator determining unit 218 calculates M as represented by the formula (4) using the determined numbers N 1 , N 2 and N 3 , and the weights W 1 , W 2 and W 3 , and determines and outputs the calculation result as a first indicator V, 5 constituting a texture descriptor. The contrast calculating unit 220 calculates the contrast by the formula (2) and outputs a signal Contmax indicating that the calculated contrast is biggest. The second candidate graph selecting/storing unit 214 outputs the candidate graphs having the biggest contrast among the second candidate graphs stored therein 10 to the second-to-fifth indicator determining unit 222. The second-to-fifth indicator determining unit 222 determines the orientation coefficients and scale coefficients of graphs that have the biggest contrast as second through fifth indicators. In other words, the orientation coefficient of a graph having the biggest contrast, among the x-projection graphs, is determined as a second 15 indicator V 2 . Also, the orientation coefficient of a graph having the biggest contrast, among the y-projection graphs, is determined as a second indicator V 3 . The scale coefficient of a graph having the biggest contrast, among the x-projection graphs, is determined as a fourth indicator V 4 . Also, the scale coefficient of a graph having the biggest contrast, among the y-projection graphs, is determined as a fifth indicator V 5 . 20 The texture descriptor output unit 224 sets and outputs the texture descriptor, that is, the texture feature vector, as {1V 1 , V 2 , V 3 , V 4 , V 5 ], Z}, using the first indicator V 1 output from the first indicator determining unit 218, the second through fifth indicators V 2 , V 3 , V 4 and V 5 output from the second-to-fifth indicator determining unit 222 and the vector Z output from the image mean/variance 25 calculating unit 202. FIG. 3 shows perceptual browsing components (PBCs) extracted from Brodatz texture images by simulation based on the image texture descriptor retrieving method according to the present invention. As described above, according to the image texture descriptor retrieving 30 method of the present invention, texture descriptors which allow kinds of texture structure present in an image to be perceptually captured can be retrieved.

WO 00/46750 PCT/KROO/00091 16 Industrial Applicability The present invention can be applied to the fields of image browsing or searching-retrieval applications.

Claims

1. A method for retrieving an image texture descriptor for describing texture features of an image, comprising the steps of: (a) filtering input images using predetermined filters having different 5 orientation coefficients; (b) projecting the filtered images onto axes of each predetermined direction to obtain data groups consisting of averages of each directional pixel values; (c) selecting candidate data groups among the data groups by a predetermined classification method; 10 (d) determining a plurality of indicators based on orientation coefficients of the filters used in filtering the candidate data groups; and (e) determining the plurality of indicators as the texture descriptor of the image. 15

2. The image texture descriptor retrieving method according to claim 1, wherein the step (a) further comprises the step of (a-1) filtering input images using predetermined filters having different scale coefficients, and the step (d) further comprises the step of (d-1) determining a plurality of indicators based on scale coefficients of the filters used in filtering the candidate data groups. 20

3. The image texture descriptor retrieving method according to claim 2, further comprising the step of determining another indicator based on the presence of data groups filtered by filters having scale coefficients or orientation coefficients which are close to or identical with the scale coefficients or orientation coefficients 25 of the filters used in filtering the selected candidate data groups.

4. The image texture descriptor retrieving method according to claim 3, further comprising the step of calculating the mean and variance of pixels with respect to each of the filtered images, and obtaining a predetermined vector using the 30 calculated mean and variance. WO 00/46750 PCT/KROO/00091 18

5. The image texture descriptor retrieving method according to claim 2, further comprising the step of calculating the mean and variance of pixels with respect to the filtered images, and obtaining a predetermined vector using the calculated mean and variance. 5

6. The image texture descriptor retrieving method according to claim 1, further comprising the step of determining another indicator based on the presence of graphs filtered by filters having scale coefficients or orientation coefficients which are close to or identical with the scale coefficients or orientation coefficients of the 10 filters used in filtering the selected candidate data groups.

7. The image texture descriptor retrieving method according to claim 6, further comprising the step of calculating the mean and variance of pixels with respect to each of the filtered images, and obtaining a predetermined vector using the 15 calculated mean and variance.

8. The image texture descriptor retrieving method according to claim 1, further comprising the step of calculating the mean and variance of pixels with respect to each of the filtered images, and obtaining a predetermined vector using the 20 calculated mean and variance.

9. A method for retrieving an image texture descriptor for describing texture features of an image, comprising the steps of: (a) filtering input images using predetermined filters having different scale 25 coefficients; (b) projecting the filtered images onto axes of each predetermined direction to obtain data groups consisting of averages of each directional pixel values; (c) determining a plurality of indicators based on scale coefficients of the filters used in filtering data groups selected among the data groups by a 30 predetermined selection method; (d) determining the plurality of indicators as the texture descriptor of the WO 00/46750 PCT/KROO/00091 19 image.

10. The image texture descriptor retrieving method according to claim 9, further comprising the step of calculating the mean and variance of pixels with 5 respect to the filtered images, and obtaining a predetermined vector using the calculated mean and variance.

11. A method for retrieving an image texture descriptor for describing texture features of an image, comprising the steps of: 10 (a) filtering input images using predetermined filters having different orientation coefficients and different scale coefficients; (b) projecting the filtered images onto axes of each predetermined direction to obtain graphs consisting of averages of each directional pixel values; (c) selecting candidate graphs among the graphs obtained in the step (b) by 15 a predetermined classification method; (d) determining another indicator based on the presence of graphs filtered by filters having scale coefficients or orientation coefficients which are close to or identical with the scale coefficients or orientation coefficients of the filters used in filtering the selected candidate graphs; 20 (e) determining a plurality of indicators based on scale coefficients or orientation coefficients of the filters used in filtering the determined candidate graphs; and (f) determining the indicator determined in the step (d) and the plurality of indicators determined in the step (e) as the texture descriptor of the image. 25

12. The image texture descriptor retrieving method according to claim 11, further comprising the step of calculating the mean and variance of pixels with respect to the filtered images, and obtaining a predetermined vector using the calculated mean and variance. 30

13. A method for retrieving an image texture descriptor for describing WO 00/46750 PCT/KROO/00091 20 texture features of an image, comprising the steps of: (a) filtering input images using predetermined filters having different orientation coefficients and different scale coefficients; (b) projecting the filtered images onto horizontal and vertical axes to obtain 5 horizontal-axis projection graphs and vertical-axis projection graphs; (c) calculating normalized auto-correlation values for each graph; (d) obtaining local maximums and local minimum for each normalized auto correlation value, at which the calculated normalized auto-correlation values forms a local peak and a local valley at a predetermined section; 10 (e) defining the average of the local maximums and the average the local minimums as contrast; (f) selecting graphs in which the ratio of the standard deviation to the average of the local maximums is less than or equal to a predetermined threshold as first candidate graphs; 15 (g) determining the type of the second candidate graphs according to the number of graphs filtered by the filters having scale coefficients or orientation coefficients which are close to or identical with the scale coefficients or orientation coefficients of the filters used in filtering the selected second candidate graphs; (h) counting the numbers of graphs belonging to the respective types of 20 second candidate graphs and determining predetermined weights of each type of second candidate graphs; (i) calculating the sum of products of the counted numbers of graphs and the determined weights to determine the calculation result value as a first indicator constituting a texture descriptor; 25 (j) determining the orientation coefficients and scale coefficients of the second candidate graphs having the biggest contrast as second through fifth indicators; and (k) determining indicators including the first indicator and the second through fifth indicators as the texture descriptors of the corresponding image. 30

14. The image texture descriptor retrieving method according to claim 13, further comprising the step of calculating the mean and variance of pixels with WO 00/46750 PCT/KROO/00091 21 respect to the filtered images, and obtaining a predetermined vector using the calculated mean and variance, wherein the step (k) includes the step of determining indicators including the first indicator, the second through fifth indicators and the predetermined vector as the texture descriptors of the corresponding image. 5

15. The image texture descriptor retrieving method according to claim 13, wherein the normalized auto-correlation, denoted by NAC(k), is calculated by the following formula: N-1 I P(m - k)P(m) NAC(k)= k P( N-1 I P 2 ( k) p2 mt=k m=k 10 wherein N is a predetermined positive integer, an input image consists of NxN pixels, a pixel position is represented by i, where i is a number from 1 to N, the projection graphs expressed by pixels of the pixel position i is represented by P(i) and k is a number from 1 to N. 15

16. The image texture descriptor retrieving method according to claim 13, wherein the contrast is determined as: I M 1L contrast - I P_ magn(i) - - V magn(i) M i L = wherein P magn (i) and V magn (i) are the local maximums and local minimums determined in the step (d). 20

17. The image texture descriptor retrieving method according to claim 13, wherein in the step (f), the graphs satisfying the following formula are selected as first candidate graphs: S d WO 00/46750 PCT/KROO/00091 22 wherein d and S are the average and standard deviation of the local maximums and a is a predetermined threshold.

18. The image texture descriptor retrieving method according to claim 13, 5 wherein the step (g) comprises the sub-steps of: (g-1) if there are one or more graphs having scale or orientation coefficients identical with those of a pertinent candidate graph and one or more graphs having scale or orientation coefficients close to those of the pertinent candidate graph, classifying the pertinent candidate graph as a first type graph; 10 (g-2) if there are one or more graphs having scale or orientation coefficients identical with those of a pertinent candidate graph but there is no graph having scale or orientation coefficients close to those of the pertinent candidate graph, classifying the pertinent candidate graph as a second type graph; and (g-3) if there is no graph having scale or orientation coefficients identical with 15 or close to those of a pertinent candidate graph, classifying the pertinent candidate graph as a third type graph.

19. The image texture descriptor retrieving method according to claim 13, wherein the step (h) includes the step of counting the number of graphs belonging to 20 each of the first through third types of graphs and determining predetermined weights for each of the types of graphs.

20. The image texture descriptor retrieving method according to claim 13, after the step of (f), further comprising the step of applying a predetermined 25 clustering algorithm to the first candidate graphs to select second candidate graphs.

21. The image texture descriptor retrieving method according to claim 20, wherein the predetermined clustering algorithm is modified agglomerative clustering. 30

22. The image texture descriptor retrieving method according to claim 13, wherein in the step (j), the orientation coefficient of a graph having the biggest WO 00/46750 PCT/KROO/00091 23 contrast, among the horizontal-axis projection graphs, is determined as a second indicator; the orientation coefficient of a graph having the biggest contrast, among the vertical-axis projection graphs, is determined as a second indicator; the scale coefficient of a graph having the biggest contrast, among the horizontal-axis 5 projection graphs, is determined as a fourth indicator; and the scale coefficient of a graph having the biggest contrast, among the vertical-axis projection graphs, is determined as a fifth indicator.

23. The image texture descriptor retrieving method according to claim 13, 10 wherein the step (j) includes the step of determining indicators including the first indicator, the second through fifth indicators and the predetermined vector as the texture descriptors of the corresponding image.

24. The image texture descriptor retrieving method according to claim 13, 15 wherein the predetermined filters include Gabor filters.

25. The image texture descriptor retrieving method according to claim 14, wherein the predetermined filters include Gabor filters. 20

26. The image texture descriptor retrieving method according to claim 15, wherein the predetermined filters include Gabor filters.

27. The image texture descriptor retrieving method according to claim 16, wherein the predetermined filters include Gabor filters. 25

28. The image texture descriptor retrieving method according to claim 17, wherein the predetermined filters include Gabor filters.

29. The image texture descriptor retrieving method according to claim 18, 30 wherein the predetermined filters include Gabor filters. WO 00/46750 PCT/KROO/00091 24

30. The image texture descriptor retrieving method according to claim 19, wherein the predetermined filters include Gabor filters.

31. A computer readable medium having program codes executable by a 5 computer to perform a method for an image texture descriptor for describing texture features of an image, the method comprising the steps of: (a) filtering input images using predetermined filters having different orientation coefficients and different scale coefficients; (b) projecting the filtered images onto horizontal and vertical axes to obtain 10 horizontal-axis projection graphs and vertical-axis projection graphs; (c) calculating normalized auto-correlation values for each graph; (d) obtaining local maximums and local minimums for each of normalized auto-correlation values, at which the calculated normalized auto-correlation value forms a local peak and a local valley at a predetermined section; 15 (e) defining the average of the local maximums and the average the local minimums as contrast; (f) selecting graphs in which the ratio of the standard deviation to the average of the local maximums is less than or equal to a predetermined threshold as first candidate graphs; 20 (g) determining the type of the second candidate graphs according to the number of graphs filtered by the filters having scale coefficients or orientation coefficients which are close to or identical with the scale coefficients or orientation coefficients of the filters used in filtering the selected second candidate graphs; (h) counting the numbers of graphs belonging to the respective types of 25 second candidate graphs and determining predetermined weights of each type of second candidate graph; (i) calculating the sum of products of the counted numbers of graphs and the determined weights to determine the calculation result value as a first indicator constituting a texture descriptor; 30 (j) determining the orientation coefficients and scale coefficients of the second candidate graphs having the biggest contrast as second through fifth indicators; and WO 00/46750 PCT/KROO/00091 25 (k) determining indicators including the first indicator and the second through fifth indicators as the texture descriptors of the corresponding image.

32. The computer readable medium according to claim 31, wherein the 5 image texture descriptor retrieving method further comprises the step of calculating the mean and variance of pixels with respect to the filtered images, and obtaining a predetermined vector using the calculated mean and variance, and wherein the step (k) includes the step of determining indicators including the first indicator, the second through fifth indicators and the predetermined vector as the texture descriptors of the 10 corresponding image.

33. An apparatus method for retrieving an image texture descriptor for describing texture features of an image, comprising: filtering mean for filtering input images using predetermined filters having 15 different orientation coefficients; projecting means for projecting the filtered images onto axes of each predetermined direction to obtain data groups consisting of averages of each directional pixel values; classifying means for selecting candidate data groups among the data groups 20 by a predetermined classification method; first indicator determining means for determining another indicator based on the number of graphs filtered by filters having scale coefficients or orientation coefficients which are close to or identical with the scale coefficients or orientation coefficients of the filters used in filtering the selected candidate graph; and 25 second indicator determining means for determining a plurality of indicators based on scale coefficients and orientation coefficients of the filters used in filtering the determined candidate graphs.

34. The image texture descriptor retrieving apparatus according to claim 30 33, further comprising mean/variance calculating means for calculating the mean and variance of pixels with respect to each of the filtered images, and obtaining a WO 00/46750 PCT/KROO/00091 26 predetermined vector using the calculated mean and variance.

35. An apparatus for retrieving an image texture descriptor for describing texture features of an image, comprising: 5 a filtering unit for filtering input images using predetermined filters having different orientation coefficients and different scale coefficients; an image mean/variance calculating unit for calculating the mean and variance of pixels with respect to each of the filtered images, and obtaining a predetermined vector using the calculated mean and variance; 10 a projecting unit for projecting the filtered images onto horizontal and vertical axes to obtain horizontal-axis projection graphs and vertical-axis projection graphs; a calculating unit for calculating a normalized auto-correlation value for each graph; a peak detecting/analyzing unit for detecting local maximums and local 15 minimums for each auto-correlation value, at which the calculated normalized auto correlation values forms a local peak and a local valley at a predetermined section; a mean/variance calculating unit for calculating the average of the local maximums and the average the local minimums; a first candidate graph selecting/storing unit for selecting the graphs satisfying 20 the requirement that the ratio of the standard deviation to the average of the local maximums be less than or equal to a predetermined threshold, as first candidate graphs; a second candidate graph selecting/storing unit for applying a predetermined clustering algorithm to the first candidate graphs to select the same as second 25 candidate graphs; a classifying unit for counting the number of graphs belonging to each of the respective types of the second candidate graphs, outputting data signals indicative of the number of graphs of each type, determining predetermined weights of the graphs belonging to the respective types and outputting data signals indicative of weights to 30 be applied to each type; a first indicator determining unit for calculating the sum of the products of the WO 00/46750 PCT/KROO/00091 27 data representing the number of graphs belonging to each type, and the data representing the weights to be applied to each type, determining and outputting the calculation result as a first indicator constituting a texture descriptor; a contrast calculating unit for calculating the contrast according to formula (2) 5 using the averages output from the mean/variance calculating unit and outputting a signal indicating that the calculated contrast is biggest; a second candidate graph selecting/storing unit for outputting the candidate graphs having the biggest contrast among the second candidate graphs stored therein in response to the signal indicating that the calculated contrast is biggest; 10 a second-to-fifth indicator determining unit for determining the orientation coefficient of a graph having the biggest contrast, among the horizontal-axis projection graphs; the orientation coefficient of a graph having the biggest contrast, among the vertical-axis projection graphs, as a second indicator; the scale coefficient of a graph having the biggest contrast, among the horizontal-axis projection graphs, 15 as a fourth indicator; and the scale coefficient of a graph having the biggest contrast, among the vertical-axis projection graphs, as a fifth indicator; and a texture descriptor output unit for combining the first indicator, the second through fifth indicators and the predetermined vector and outputting the combination result as the texture descriptors of the corresponding image. 20