JP4036009B2 - Image data classification device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to image data classification.
[0002]
[Prior art]
Conventionally, in order to classify and search registered image data, it has been common to add code information such as keywords together with the image data. This code information has been input from a keyboard or the like when registering image data or classifying and organizing already registered image data. However, in such a method, when registering image data to be classified or searched, there is a problem that code information must be manually input, and a large amount of work is required.
[0003]
In view of this, a method has been proposed in which a feature vector is constructed by extracting various feature amounts from image data, and this is used for classification and retrieval as information replacing code information. The similarity between image data is defined according to the distance between feature vectors extracted from the image data. However, the similarity depends on the distribution of feature vectors and the definition of vector distance calculation. It was pointed out that the desired similarity may not be required. In order to eliminate the dependency due to the feature vector distribution, the technique described in Japanese Patent Laid-Open No. 9-265529 eliminates the influence of the distribution by correcting the classification result based on the feature vector distribution. .
[0004]
In addition, when performing classification based on feature vectors, many types of feature quantities are extracted in order to capture image data from various aspects, and the feature vectors are high-dimensional. When performing classification and retrieval using feature vectors, there is a problem that the processing time of vector matching increases if this is high-dimensional.
[0005]
Therefore, in many cases, by performing principal component analysis on the accumulated feature vectors, the orthogonality of the feature vectors is guaranteed, and by eliminating the spatial axis with small variance, the number of feature vector dimensions can be reduced. Yes. In Japanese Patent Laid-Open No. 11-219347, the accumulated feature vectors are described hierarchically, and when searching or classifying, the collation of the unlikely hierarchy is omitted, thereby speeding up the processing. I am trying.
[0006]
[Problems to be solved by the invention]
However, even with the technique described in the above-mentioned publication, it has not been possible to guarantee whether the result classified or searched by the feature vector is valid. That is, there are cases where the result classified by the feature vector is not the classification desired by the user.
[0007]
Also, if the number of stored feature vectors has a large number of spatial axes, or if the distribution of stored feature vectors is large, the number of feature vectors can be reduced sufficiently even if principal component analysis is performed. There was a case that could not be.
[0008]
The present invention has been made in consideration of such problems, and it is an object of the present invention to make classification based on feature vectors extracted from image data close to classification desired by a user and to reduce processing load. To do.
[0009]
[Means for Solving the Problems]
In order to solve the above-described problem, in the image data classification device according to the present invention, first, based on the feature vectors of image data previously classified into a plurality of reference groups, individual images are defined by two or more types of distance definitions. A vector section distance between data is calculated, and image data belonging to the reference group is classified based on the calculated distance between vectors. Then, the classification result is compared with the classification method of the reference group to obtain a distance definition that provides a classification result similar to the classification of the reference group. This distance definition is applied to classify unclassified image data.
[0010]
Further, the calculation of the distance between vectors by the distance calculation means can be performed based on at least one of variance, covariance and density of the feature vector.
[0011]
Further, the calculation of the distance between vectors by the distance calculation means can be performed based on the appearance probability of the feature vector with respect to a predetermined spatial axis.
[0012]
Further, the feature vector of the image data includes a spatial axis related to registration of the image data, and the calculation of the inter-vector distance by the distance calculation means is performed based on the appearance probability corresponding to the spatial axis related to the registration of the image. Can do.
[0013]
Another image data classification device calculates the inter-vector distance between the reference groups based on the n-dimensional feature vectors of the image data previously classified into a plurality of reference groups. In addition, the degree of contribution related to the inter-vector distance of each space axis of the n-dimensional feature vector is calculated, m space axes having a high degree of contribution are extracted, and m-dimensional feature vector spaces are extracted. . Then, the unclassified image data is classified in the m-dimensional feature vector space based on the inter-vector distance.
[0014]
In calculating the contribution, a distance between reference groups is calculated based on a feature vector excluding at least one spatial axis of the n-dimensional feature vector, and a comparison between the distance and a distance based on the n-dimensional feature vector is performed. The degree of contribution of the excluded space axis can be calculated.
[0015]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention (hereinafter referred to as embodiments) will be described with reference to the drawings. FIG. 1 is a diagram showing a schematic configuration of a network to which the image data classification device of this embodiment is applied. The network includes a client 1 that is a computer functioning as an image data analysis apparatus according to the present embodiment by a predetermined program installed, a scanner 2 that captures an image on a print medium such as paper, a database server 3 that provides a database, and a network Is connected to a web server 4 for managing the above. A router 6 is also provided for connecting a network with an external terminal 5 such as a notebook computer or a portable information terminal. In addition to the scanner, the image data can be read from the network, from the external terminal 5, and from the client 1 external storage medium drive.
[0016]
FIG. 2 is a functional configuration diagram of the present embodiment, and particularly shows a configuration related to selection of a distance definition for classifying image data. The image data 10 is classified and stored in advance in two reference groups 12 and 14. This classification is preferable for the user. A feature vector extraction unit 16 extracts feature vectors from the image data classified into the first reference group 12 and the second reference group 14. The distance between the extracted feature vectors is calculated by the distance calculation means 18. The distance calculating means 18 includes a first distance calculating unit 18-1, a second distance calculating unit 18-2,..., An Nth distance calculating unit 18− that calculate each distance based on N distance definitions. Contains N. Each of the plurality of distances calculated by the distance calculation means 18 is evaluated by the evaluation means 20 for the validity of the distance definition. The evaluation means 20 has N evaluation units corresponding to the N distance calculation units 18-1 to 18-N on a one-to-one basis, that is, the first evaluation unit 20-1, the second evaluation unit 20-2,. ..Nth evaluation unit 20-N is included. Each of the evaluation units 20-1 to 20-N classifies image data belonging to the two reference groups 12 and 14 again based on the distances calculated by the corresponding distance calculation units 18-1 to 18-N. Then, the degree of similarity between the reclassification and the classification of the preferably classified reference group is evaluated. Based on the evaluation results of the evaluation units 20-1 to 20-N, the distance definition selecting unit 22 selects an appropriate distance definition. The selection of the distance definition can be performed based on how the image data classified by the distance definition is classified in the same manner as the reference groups 12 and 14 or how similar.
[0017]
FIG. 3 is a functional configuration diagram of the present embodiment, and particularly shows a configuration related to classification of unclassified image data based on a distance selected and defined for performing preferable classification. Each feature vector of the unclassified image data 24 is extracted by the feature vector extraction means 26. Furthermore, the distance calculation means 28 calculates the distance between the extracted feature vectors. The distance definition used for the distance calculation at this time is selected by the distance definition selection means 22 described above. The classifying unit 30 classifies the unclassified image data 24 based on the calculated distance between the vectors, and registers the classified image data in the database 32.
[0018]
FIG. 4 is a process flowchart relating to selection of a distance definition of image data according to the present embodiment. First, image data that has already been classified preferably is input (S100), feature vectors are extracted (S102), and registered in the database 32. The preferable classification at this time is, for example, classification performed by the user and its evaluation in advance, and includes classification based on subjective evaluation in addition to objective evaluation. A feature vector is an n-dimensional vector composed of values that can be quantitatively extracted from image data. For example, it consists of n feature quantities obtained by image processing, such as color information and edge information included in image data. The above series of processing is repeated until there is no image data to be analyzed (S104).
[0019]
Next, one is selected from a plurality of predetermined distance definitions (S106), and a distance between feature vectors is obtained between the classified groups (reference groups) of the image data input in step S100 (S108). The distance definition here is a definition for obtaining a distance between vectors (which may be a similarity or dissimilarity). For example, the Chebyshev distance, feature vector variance, and covariance are taken into account. Use Mahalanobis distance or Euclidean distance. For example, a distance that takes into account the density of each classification, a distance that reflects the probability of appearance corresponding to at least one of the registrant of image data, registration order, registration interval, registration date, etc. May be. The appearance probability indicates, for example, the probability that the image data of the latter is classified into the same group as the former image data when the image data having a similar registration date and time comes to certain image data. If the registrants are the same, it is considered that image data having the same tendency is strongly registered. In addition, if the registration order and the registration date and time are close to each other, it is highly possible that the images are scene images of one event (for example, athletic meet).
[0020]
Based on the calculated distance, the feature vectors are classified (S110). Here, the classification is to group together those having a short distance based on the distance between feature vectors. In general, a method called cluster analysis is used. There are a hierarchical method and a non-hierarchical method for cluster analysis, but either can be used here. When the classification of all feature vectors is completed (S112), the comparison with the prior classification, that is, the classification into which the reference group is divided is performed (S114). The above steps S106 to S114 are repeated until there is no distance definition to be evaluated (S116). When the evaluation is completed for all the distance definitions, the distance definition is finally selected (S118). The selection of the distance definition compares the results of evaluations made for each distance definition, and selects a distance definition that has been classified more similar to a reference group that has been preferably classified in advance.
[0021]
FIG. 5 is a flowchart showing processing relating to classification of unclassified image data according to the present embodiment. First, image data to be classified (unclassified) is fetched (S120), feature vectors of these image data are extracted and registered in the database 32 (S122). This feature vector is a vector equivalent to that extracted in step S102 of the chart shown in FIG. 4, and is an n-dimensional vector composed of values that can be extracted quantitatively from the image data. The feature vector extraction is repeated until there is no image data to be classified (S124).
[0022]
Next, the distance definition selected in step S118 of the chart of FIG. 4 is called (S126), and the distance between each feature vector of unclassified image data is calculated based on this distance calculation method (S128). Based on the calculated distance, the feature vector, that is, the unclassified image data is classified by the same classification method as in step S110 of the chart of FIG. 4 (S130). If the classification is completed (S132), the classification result is registered in the database 32 (S138).
[0023]
As described above, in this embodiment, new image data is classified based on a distance definition that is similar to or similar to already classified image data. Therefore, when classifying new image data, it is possible to automatically perform classification equivalent to or similar to this classification with reference to already classified image data.
[0024]
FIG. 6 is a functional configuration diagram of the related art of the present invention , and particularly shows a configuration related to extraction of feature vectors for classifying image data. The image data 50 is classified and stored in advance in two reference groups 52 and 54. This classification is preferable for the user. Each image data classified into the first reference group 52 and the second reference group 54 has an n-dimensional feature vector. The feature quantity represented by each spatial axis of the feature vector is extracted one by one by the feature quantity extraction means 58. Then, the feature amount evaluation means 60 evaluates how much the feature amount contributes to dividing the image data 50 into the two reference groups 52 and 54 for each feature amount. Specifically, the feature quantity extraction unit 58 extracts first to n-th feature quantity extraction units 58-1 to 58-1 for extracting one of the first to n-th spatial axes constituting the n-dimensional feature vector. Includes 58-n. The feature quantity evaluation means 60 has the first to nth feature quantity evaluation units 60-1 to 60-n corresponding to the first to nth feature quantity extraction units 58-1 to 58-n on a one-to-one basis. Including. Each of the feature quantity evaluation units 60-1 to 60-n evaluates the contribution of the classification of the reference groups 52 and 54 to the feature quantities extracted by the corresponding feature quantity extraction units 58-1 to 58-n. To do. Specifically, an (n−1) -dimensional feature vector excluding one extracted feature quantity (spatial axis) is formed, and the image data 50 belonging to the reference groups 52 and 54 is reclassified based on this feature vector. To do. If this reclassification is similar to or similar to the two reference groups 52 and 54, it can be determined that the excluded feature amount has a low contribution to the classification of the reference groups 52 and 54. On the other hand, if the classification is greatly different in the reclassification, it can be determined that the excluded feature amount has a high contribution to the classification of the reference groups 52 and 54. Evaluation is performed for each spatial axis, and an m-th order feature vector is configured by m (n> m) spatial axes having a high degree of contribution by the feature vector configuration unit 62.
[0025]
FIG. 7 is a functional configuration diagram of the related art of the present invention , and particularly shows a configuration related to classification of unclassified image data based on an m-dimensional feature vector extracted for performing preferable classification. The m feature amounts of each of the unclassified image data 64 are extracted by the feature amount extraction unit 66. The feature quantity extraction means is composed of m first to m-th feature quantity extraction units 66-1 to 66-m, and one extraction unit corresponds to one feature quantity. An m-dimensional feature vector is generated by the feature vector generation means 68 from the extracted feature amount. In this m-dimensional feature vector space, classification is performed by the classification means 70, and the classification result is registered in the database 72.
[0026]
FIG. 8 is a flowchart of processing related to feature amount extraction for classification according to the related art of the present invention . First, image data that has been preferably classified is input (S200), n feature quantities are extracted for each image data, and registered in the database 72 (S202). Here, the feature amount is a value that can be quantitatively extracted from the image data, for example, a value obtained by image processing such as color information and edge information included in the image data. This process is repeated until there is no image data to be analyzed (S204).
[0027]
Next, an (n−1) -dimensional feature vector space composed of features obtained by removing one feature (space axis) is extracted from the n-dimensional feature vector space (S206), and the distance between the reference groups 52 and 54 is extracted. Is measured (S208). At this time, the centroids of the reference groups in the (n−1) -dimensional feature vector space are obtained, and the distance between the centroids is measured to obtain the distance between the reference groups. The Λ statistic of Wilkes is obtained from the distance between the reference groups in the (n−1) -dimensional feature vector space thus obtained and the distance between the reference groups in the n-dimensional feature vector space, and is compared. A contribution indicating whether it contributes to the classification is obtained (S210). This process is performed for all n feature values (S212). Based on the above-mentioned contribution degree, an m-dimensional feature vector space used for actual classification is selected. At this time, a feature amount that is statistically significant is selected, or the top i or top j% feature amount of the degree of contribution extracted under a predetermined condition is selected (S214).
[0028]
FIG. 9 is a flowchart showing processing relating to classification of unclassified image data according to the related art of the present invention . First, the image data to be classified (unclassified) is fetched (S216), and the feature amount of each image data is extracted and registered in the database 72 (S218). This feature quantity is a vector equivalent to that extracted in step S202 of the chart shown in FIG. 8, and is n feature quantities having values that can be extracted quantitatively from the image data. The feature vector extraction is repeated until there is no image data to be classified (S220).
[0029]
Next, using the feature vector space determined in step S214 of the chart of FIG. 8 (S222), the distance between each feature vector of unclassified image data is calculated (S224). Based on the calculated distance, the feature vectors are classified (S226). This classification is based on the distances between feature vectors, and those with close distances are collected. In general, a method called cluster analysis is used. Cluster analysis includes a hierarchical method and a non-hierarchical method, either of which may be used. When the feature vector classification is completed (S228), the classification result is registered in the database (S230), and the process ends.
[0030]
According to the related technology , new image data is classified based on feature quantity vectors composed of feature quantities having a high contribution degree with respect to classification of already classified image data. Therefore, when new image data is classified, classification similar to or similar to the classification of already classified image data can be automatically performed. Further, by selecting feature amounts having a high contribution, the dimension of the feature vector used for classification can be reduced, and the processing burden is reduced.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating an example of a network environment.
FIG. 2 is a block diagram illustrating functions of the present embodiment, and particularly a configuration related to determination of a distance definition used for classification.
FIG. 3 is a block diagram illustrating functions of the present embodiment, and particularly a configuration related to classification of unclassified image data.
FIG. 4 is a flowchart showing processing of the present embodiment, and is a chart relating to determination of a distance definition used for classification in particular.
FIG. 5 is a flowchart showing processing of the present embodiment, particularly a chart relating to classification of unclassified image data.
FIG. 6 is a block diagram showing the function of, and is a diagram showing a configuration related to extraction of feature amounts used for classification in particular.
FIG. 7 is a block diagram showing functions of a related technique of the present invention , and particularly a diagram showing a configuration relating to classification of unclassified image data.
FIG. 8 is a flowchart showing processing according to a related technique of the present invention , and particularly shows a configuration related to extraction of feature amounts used for classification.
FIG. 9 is a flowchart showing processing according to the related art of the present invention , particularly a chart relating to classification of unclassified image data.
[Explanation of symbols]
10, 50 Image data (classified), 12, 52 First reference group, 14, 54 Second reference group, 16 Feature vector extraction means, 18 Distance calculation means, 20 Evaluation means, 22 Distance definition selection means, 24, 64 Image data (unclassified), 26 feature vector extraction means, 28 distance calculation means, 30 classification means, 32, 72 database, 58 feature quantity extraction means, 60 feature quantity evaluation means, 62 feature vector construction means, 66 feature quantity extraction means 68 Feature vector generation means, 70 Classification means.

Claims (4)

An image data classification device that classifies the image data based on feature vectors extracted from the image data,
Distance calculating means for calculating a distance between vectors of individual image data based on two or more types of distance definitions based on feature vectors of image data classified in advance into a plurality of reference groups;
The image data belonging to the reference group is classified based on the calculated inter-vector distance, the classification result is compared with the reference group, the distance definition for calculating the inter-vector distance is evaluated, and the reference group A distance definition selection means for selecting a distance definition having a classification similar to the classification;
Applying the selected distance definition to unclassified image data, classifying means for classifying the unclassified image data,
An image data classification device having 2. The image data classification device according to claim 1, wherein the calculation of the inter-vector distance by the distance calculation means is performed based on at least one of variance, covariance and density of the feature vector. . The image data classification apparatus according to claim 1, wherein the calculation of the distance between vectors by the distance calculation unit is performed based on an appearance probability of the feature vector with respect to a predetermined spatial axis. 2. The image data classification apparatus according to claim 1, wherein the feature vector of the image data includes a spatial axis related to registration of the image data, and the calculation of the inter-vector distance by the distance calculation means relates to registration of the image. An image data classification device, which is performed based on an appearance probability corresponding to a spatial axis.

Images