JP2003242160A - Image data classifying apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To classify a plurality of image data according to user's desire. <P>SOLUTION: The plurality of image data 10 are classified into a plurality of reference groups 12, 14 in advance. This classification is preferably made according to the user's desire, purpose or the like. For the image data belonging to these reference groups, a distance is calculated based on a plurality of types of distance definitions by a distance calculating means 18. Using an evaluating means 20, the image data 10 are classified again based on the distance, and a comparative evaluation is made with previous classification. Based on the distance definitions in which the nearest classification to the initial one is made, image data which have not been classified yet are classified. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention relates to classification of image data.

[0002]

2. Description of the Related Art Conventionally, in order to classify and search registered image data, it has been common to add code information such as a keyword together with the image data. This code information has been input from a keyboard or the like when registering image data or when classifying and organizing already registered image data. However, in such a method, when registering image data to be classified or searched,
There is a problem that a lot of work is required because the code information has to be manually input.

Therefore, a method has been proposed in which various feature quantities are extracted from image data to form a feature vector, and the feature vector is used for classification and retrieval as information in place of code information. Further, the similarity between image data is defined according to the closeness of the distance between the feature vectors extracted from the image data, but the similarity depends on the distribution of the feature vectors and the definition of the vector distance calculation. It was pointed out that the desired degree of similarity may not be required. In order to eliminate the dependency due to the distribution of this feature vector, Japanese Patent Laid-Open No. 9-26552
In the technique described in Japanese Patent Publication No. 9, the effect of the distribution is eliminated by correcting the classification result based on the distribution of the feature vector.

In addition, when classifying by a feature vector, many kinds of feature quantities are extracted in order to capture image data from various aspects, and the feature vector has a high dimension. When performing classification and retrieval using feature vectors, there is a problem that the processing time for vector matching increases if the dimensions are high.

Therefore, in many cases, by performing principal component analysis on the accumulated feature vectors, the orthogonality of the feature vectors is assured, and the spatial axis having a small variance is eliminated, whereby the dimension number of the feature vector is reduced. Is suppressed. Further, in Japanese Patent Laid-Open No. 11-219347, accumulated feature vectors are described in a hierarchical manner, and at the time of search or classification, collation of a hierarchy having a low possibility is omitted to speed up processing. I am trying to

[0006]

However, even with the technique described in the above publication, it is not possible to guarantee whether the results classified or searched by the feature vector are valid. In other words, the result classified by the feature vector may not be the classification desired by the user.

Further, when the number of spatial axes of accumulated feature vectors is very large or the variance of accumulated feature vectors is large, the number of spatial axes of feature vectors is sufficient even if principal component analysis is performed. In some cases it could not be reduced to.

The present invention has been made in consideration of such problems, and makes the classification by the feature vector extracted from the image data closer to the classification desired by the user, and reduces the processing load. With the goal.

[0009]

In order to solve the above-mentioned problems, in the image data classification device according to the present invention,
First, based on the feature vector of the image data classified in advance into a plurality of reference groups, the vector interval distance between the individual image data is calculated by the distance definition of two or more types,
The image data belonging to the reference group is classified according to the calculated inter-vector distance. Then, this classification result is compared with the classification method of the reference group to obtain a distance definition that gives a classification result similar to the classification of the reference group. By applying this distance definition, unclassified image data is classified.

Further, the calculation of the inter-vector distance by the distance calculating means can be performed in consideration of at least a part of the variance, covariance and density of the feature vector.

Further, the calculation of the inter-vector distance by the distance calculating means can be performed in consideration of the appearance probability of the feature vector with respect to a predetermined space axis.

The feature vector of the image data is
The calculation of the inter-vector distance including the spatial axis related to the registration of the image data can be performed by taking the appearance probability corresponding to the spatial axis related to the registration of the image into consideration.

Further, another image data classifying apparatus according to the present invention calculates the inter-vector distance between the reference groups based on the n-dimensional feature vector of the image data classified in advance into the plurality of reference groups. Also, the contribution of each spatial axis of the n-dimensional feature vector to the inter-vector distance is calculated, and m smaller than n are extracted from the spatial axes having a high contribution to extract an m-dimensional feature vector space. . And
The unclassified image data is classified in the m-dimensional feature vector space based on the inter-vector distance.

In calculating the contribution, the distance between the reference groups is calculated based on the feature vector excluding at least one spatial axis of the n-dimensional feature vector, and this distance and the distance based on the n-dimensional feature vector are calculated. By comparison, the contribution of the excluded spatial axis can be calculated.

[0015]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention (hereinafter referred to as embodiments) will be described below with reference to the drawings. Figure 1
It is a figure which shows schematic structure of the network to which the image data classification device of this embodiment is applied. The network includes a client 1, which is a computer that functions as an image data analysis apparatus according to the present embodiment, a scanner 2 that captures an image on a print medium such as paper, a database server 3 that provides a database, and a network. A web server 4 that manages is connected. Also, a laptop computer,
A router 6 for connecting a network to an external terminal 5 such as a mobile information terminal is also provided. The image data can be read on the net, from the external terminal 5, and further from the drive for the external storage medium of the client 1, in addition to the scanner described above.

FIG. 2 is a functional block diagram of this embodiment, and particularly shows a configuration related to selection of a distance definition for classifying image data. The image data 10 is classified into two reference groups 12 and 14 and stored in advance. This classification has become preferable to the user. The feature vector extraction means 16 extracts a feature vector from each image data classified into the first reference group 12 and the second reference group 14. The distance calculation unit 18 calculates the distance between the extracted feature vectors. The distance calculation means 18 calculates the respective distances based on the N distance definitions, the first distance calculation unit 18-1 and the second distance calculation unit 1.
8-2, ... Includes an Nth distance calculation unit 18-N. The plurality of distances calculated by the distance calculating means 18 are respectively evaluated by the evaluating means 2.
Zero evaluates the validity of the distance definition. The evaluation means 20 corresponds to the N distance calculation units 18-1 to 18-N in a one-to-one correspondence with N evaluation units, that is, the first evaluation unit 20-1 and the second evaluation unit 20-1.
The evaluation unit 20-2, ... Includes an Nth evaluation unit 20-N.
Each of the evaluation units 20-1 to 20-N has a corresponding distance calculation unit 18-1.
Two reference groups 1 based on each distance calculated by ~ 18-N
The image data belonging to 2 and 14 are classified again. And
Assess how similar this reclassification is to the classification of the preferably classified reference group. Each evaluation unit 20-1 to 20-N
The distance definition selecting means 22 selects an appropriate distance definition based on the evaluation result of 1. The distance definition can be selected based on how much the image data classified by the distance definition is classified in the same manner as the reference groups 12 and 14, or to what degree the image data is classified.

FIG. 3 is a functional configuration diagram of this embodiment, and particularly shows a configuration related to classification of unclassified image data based on the distance selected and defined for performing preferable classification. Each feature vector of the unclassified image data 24 is extracted by the feature vector extraction means 26. Further, the distance calculating means 28 calculates the distance between the extracted feature vectors. The distance definition used for the distance calculation at this time is selected by the distance definition selecting means 22 described above. The classification means 30 classifies the unclassified image data 24 based on the calculated inter-vector distances, and classifies the classified image data as
Register in the database 32.

FIG. 4 is a processing flowchart relating to the selection of the distance definition of the image data according to the present embodiment. First, image data that has already been subjected to preferable classification is input (S100), and feature vectors are extracted (S10).
2), register in the database 32. The preferable classification at this time is, for example, classification performed by the user and evaluation thereof in advance, and includes not only objective evaluation but also subjective evaluation. In addition, the feature vector is an n-dimensional vector composed of values that can be quantitatively extracted from the image data. For example, n pieces of feature amounts obtained by image processing, such as color information and edge information included in image data. The above series of processing is repeated until there is no image data to be analyzed (S104).

Next, one is selected from a plurality of predetermined distance definitions (S106), and the inter-feature vector distance is obtained between the classified groups (reference groups) of the image data input in step S100 (S108). . The distance definition here is the distance between vectors (may be similarity or dissimilarity).
For example, the Chebyshev distance, the variance of the feature vector, the Mahalanobis distance or the Euclidean distance in consideration of the covariance are used for the definition. Defined uniquely, for example, the distance that takes the density of each classification into consideration, the distance that reflects the appearance probability corresponding to at least one of the registrant of image data, registration order, registration interval, registration date, etc. May be. The appearance probability indicates, for example, the probability that the image data of the latter is classified into the same group as the image data of the former when the image data of which the registration date and time are close to the image data. If the registrants are the same, it is considered that image data having the same tendency is strongly registered. Further, if the images have the same registration order or registration date and time, it is highly likely that the images are scenes of one event (for example, an athletic meet).

The feature vectors are classified based on the calculated distances (S110). Here, the classification is based on the distance between the feature vectors and collects those having a short distance. A method generally called cluster analysis is used. There are hierarchical and non-hierarchical methods for cluster analysis, but either method can be used here.
If the classification of all feature vectors is completed (S112),
A comparison is performed with the classification in advance, that is, the classification in which the reference group is divided (S114). The above steps S106 to S114,
Repeat until there is no distance definition to be evaluated (S1
16) When the evaluation is completed for all distance definitions, the distance definition is finally selected (S118). This distance definition selection compares the results of the evaluations made for each distance definition and selects a distance definition with a more similar classification to the previously favorably classified reference group.

FIG. 5 is a flowchart showing the processing relating to the classification of unclassified image data according to this embodiment. First,
Image data to be classified (unclassified) is fetched (S120), feature vectors of each of these image data are extracted, and registered in the database 32 (S12).
2). This feature vector is a vector equivalent to that extracted in step S102 of the chart shown in FIG.
It is an n-dimensional vector consisting of values that can be quantitatively extracted from image data. The extraction of the feature vector is repeated until there is no image data to be classified (S12).
4).

Next, step S118 of the chart of FIG.
The distance definition selected in step S126 is called (S126), and the distance between the feature vectors of the unclassified image data is calculated based on this distance calculation method (S128). Based on the calculated distance, the feature vector, that is, unclassified image data is classified by the same classification method as in step S110 of the chart of FIG. 4 (S130). Classification is completed (S1
If 32), the classification result is registered in the database 32 (S138).

As described above, in the present embodiment, new image data is classified based on the distance definition that is similar or similar to the already classified image data. Therefore, when classifying new image data, it is possible to refer to the already classified image data and automatically perform a classification equivalent to or similar to this classification.

FIG. 6 is a functional block diagram of the second embodiment according to the present invention, and particularly shows a configuration for extracting a feature vector for classifying image data. The image data 50 is pre-classified and stored in two reference groups 52 and 54. This classification has become preferable to the user. Each image data classified into the first reference group 52 and the second reference group 54 has an n-dimensional feature vector. The feature quantity represented by each spatial axis of this feature vector is extracted by the feature quantity extraction means 58 one by one. Then, for each of the feature amounts, the feature amount evaluation unit 60 evaluates how much the feature amount contributes to dividing the image data 50 into the two reference groups 52 and 54. Specifically, the feature amount extraction means 58
Includes first to nth feature amount extraction units 58-1 to 58-n for extracting one of the first to nth spatial axes forming the n-dimensional feature vector, respectively. The feature amount evaluation means 60,
One-to-one correspondence with the first to n-th feature amount extraction units 58-1 to 58-n.
Corresponding to the first to nth feature amount evaluation units 60-1 to 60-n. Each of the feature amount evaluation units 60-1 to 60-n evaluates the contribution of the classification of the reference groups 52 and 54 to the feature amount extracted by the corresponding feature amount extraction unit 58-1 to 58-n. To do. Specifically, an (n-1) -dimensional feature vector excluding one extracted feature amount (spatial axis) is formed, and the image data 50 belonging to the reference groups 52 and 54 is formed based on this feature vector.
Reclassify. This reclassification results in two reference groups 52, 54.
If it is similar or similar to the above, it can be determined that the removed feature amount has a low contribution to the classification of the reference groups 52 and 54. On the other hand, if the classifications are largely different in the reclassification, it can be determined that the removed feature amount has a high contribution to the classification of the reference groups 52 and 54. Each spatial axis is evaluated, and the feature vector constructing unit 62 constructs an m-th order feature vector from m (n> m) spatial axes having a high degree of contribution.

FIG. 7 is a functional block diagram of the second embodiment, and particularly shows a configuration related to classification of unclassified image data based on the m-dimensional feature vector extracted to perform preferable classification. There is. The m feature amounts of each of the unclassified image data 64 are extracted by the feature amount extraction means 66. The feature amount extraction means is composed of m first to m-th feature amount extraction units 66-1 to 66-m, and for one feature amount,
One extraction unit corresponds. From the extracted features,
The feature vector generation means 68 generates an m-dimensional feature vector. In this m-dimensional feature vector space, the classification means 70 performs classification, and the classification results are stored in the database 72.
Register with.

FIG. 8 is a processing flowchart relating to the extraction of the feature quantity for classification according to the second embodiment. First,
Image data that has already been subjected to preferable classification is input (S200), n feature quantities are extracted for each image data, and registered in the database 72 (S202). Here, the feature amount is a value that can be quantitatively extracted from the image data, and is, for example, a value obtained by image processing such as color information or edge information included in the image data. This process is repeated until there is no image data to be analyzed (S204).

Next, the n-dimensional feature vector space is constructed by a feature amount excluding one feature amount (spatial axis) (n-
1) The dimensional feature vector space is extracted (S206) and the distance between the reference groups 52 and 54 is measured (S208). At this time, the barycenter of the reference group in the (n-1) -dimensional feature vector space is obtained, and the distance between the barycenters is measured and used as the distance between the reference groups. From the distance between the reference groups in the (n-1) -dimensional feature vector space thus obtained and the distance between the reference groups in the n-dimensional feature vector space, Wilx's Λ statistic is calculated and compared, and the removed feature amount is A contribution degree indicating whether or not it contributes to the classification is obtained (S210). This process is performed for all n feature quantities (S212). An m-dimensional feature vector space used for actual classification is selected based on the above contribution. At this time, a feature amount that is statistically significant is selected, or the top i of contributions extracted under a predetermined condition are selected.
Individual or top j% feature quantities are selected (S214).

FIG. 9 is a flow chart showing a process relating to classification of unclassified image data according to the second embodiment. First, the image data to be classified (unclassified) is fetched (S216), the feature amount of each of these image data is extracted, and this is registered in the database 72 (S218).
This feature amount is obtained in step S202 of the chart shown in FIG.
The vector is equivalent to the vector extracted in step (4), and is the n feature quantities having values that can be quantitatively extracted from the image data. The extraction of the feature vector is repeated until there is no image data to be classified (S220).

Next, step S214 in the chart of FIG.
Using the feature vector space determined in (S222), the distance between the feature vectors of the unclassified image data is calculated (S224). The feature vector is classified based on the calculated distance (S226). This classification is based on the distance between the feature vectors and collects those having a short distance. A method generally called cluster analysis is used. Cluster analysis includes a hierarchical method and a non-hierarchical method, and either method may be used. When the feature vector classification is completed (S228), the classification result is registered in the database (S230), and the process is completed.

According to the second embodiment, regarding the classification of the image data that has already been classified, new image data is classified by the feature quantity vector composed of the feature quantity having a high contribution. Therefore, when classifying new image data, classification similar to or similar to the classification of image data already classified can be automatically performed. In addition, by selecting features with high contribution,
The dimension of the feature quantity vector used for classification can be reduced, and the processing load can be reduced.

[Brief description of drawings]

FIG. 1 is a diagram illustrating an example of a network environment.

FIG. 2 is a block diagram showing functions of the present embodiment,
It is a figure which shows the structure regarding the determination of the distance definition especially used for classification.

FIG. 3 is a block diagram showing functions of the present embodiment,
It is a figure which shows the structure regarding classification of unclassified image data especially.

FIG. 4 is a flowchart showing processing of the present embodiment, and is a chart relating to determination of a distance definition used for classification in particular.

FIG. 5 is a flowchart showing the processing of this embodiment, and is a chart relating to the classification of unclassified image data.

FIG. 6 is a block diagram showing functions of the second embodiment, and is a diagram showing a configuration particularly related to extraction of a feature amount used for classification.

FIG. 7 is a block diagram showing functions of the second embodiment, and particularly a diagram showing a configuration relating to classification of unclassified image data.

FIG. 8 is a flowchart showing a process of the second embodiment, and is a diagram showing a configuration particularly related to extraction of a feature amount used for classification.

FIG. 9 is a flowchart showing the processing of the second embodiment, and is a chart relating to classification of unclassified image data in particular.

[Explanation of symbols]

10, 50 Image data (classified), 12, 52 First reference group, 14, 54 Second reference group, 16 Feature vector extraction means, 18 Distance calculation means, 20 Evaluation means, 22
Distance definition selection means, 24, 64 image data (unclassified), 26 feature vector extraction means, 28 distance calculation means, 30 classification means, 32, 72 database, 58
Feature quantity extraction means, 60 Feature quantity evaluation means, 62 Feature vector construction means, 66 Feature quantity extraction means, 68 Feature vector generation means, 70 Classification means.

Claims (6)

[Claims] 1. An image data classification device for classifying the image data based on a feature vector extracted from the image data, wherein two or more types of image data are classified based on the feature vector of the image data previously classified into a plurality of reference groups. By the distance definition, distance calculating means for calculating the inter-vector distance between individual image data, and by the calculated inter-vector distance, the image data belonging to the reference group is classified, and this classification result is referred to as the reference group. By comparing, the distance definition calculating the distance between the vectors is evaluated, and the distance definition selecting means for selecting a distance definition having a classification similar to the classification of the reference group, and the selection for the unclassified image data. And an image data classification device that applies the defined distance definition to classify the unclassified image data. 2. The image data classification device according to claim 1, wherein the calculation of the inter-vector distance by the distance calculating means takes into consideration at least a part of the variance, covariance and density of the feature vector. An image data classification device performed. 3. The image data classification device according to claim 1, wherein the calculation of the inter-vector distance by the distance calculating means is performed in consideration of the appearance probability of the feature vector with respect to a predetermined space axis. Image data classification device. 4. The image data classification device according to claim 1, wherein the feature vector of the image data includes a spatial axis relating to registration of the image data, and calculation of the inter-vector distance by the distance calculation means, An image data classification device, which is performed in consideration of an appearance probability corresponding to a spatial axis regarding registration of the image. 5. An image data classification device for classifying the image data based on an n-dimensional feature vector extracted from the image data, the image data classification device based on the n-dimensional feature vector of the image data previously classified into a plurality of reference groups. An n-dimensional distance calculating means for calculating an inter-vector distance between the reference groups, a contribution degree calculating means for calculating a contribution degree of each spatial axis of the n-dimensional feature vector with respect to the inter-vector distance, and a high contribution degree. An m-dimensional feature vector space extracting means for extracting m pieces of space axes smaller than n and extracting an m-dimensional feature vector space; and for the unclassified image data, the reference in the m-dimensional feature vector space. Calculate the vector distance to the group,
An image data classification device having a classification means for classifying the unclassified image data based on the calculation result. 6. The image data classification device according to claim 5, wherein the contribution degree calculation unit calculates the distance between the reference groups based on a feature vector excluding at least one spatial axis of the n-dimensional feature vector. An image data classification device that calculates and calculates the contribution of the excluded spatial axis by comparing this distance with the distance based on the n-dimensional feature vector.