JP4708319B2 - Metadata similarity measurement device and metadata hierarchization device, metadata similarity measurement method and metadata hierarchization method, metadata similarity measurement program, metadata hierarchization program, and recording medium on which these programs are recorded - Google Patents

Description

  The present invention relates to a method and apparatus for measuring the similarity between metadata, a method and apparatus for automatically hierarchizing metadata, a program thereof, and the program recorded in a document provided with metadata. The present invention relates to a recording medium.

  Here, the metadata in the present invention is mainly referred to as a tag, category, genre, etc. in a blog, which is a word or phrase that expresses the content of a document set by a user to organize the document created by the user. It is an expression.

  The document group in the present invention is a plurality of documents with the same metadata, and specifically, an article composed of blog articles created by a plurality of users with the same tag set. Such as a set.

  With regard to the relationship between metadata and document groups in the present invention, it can be said that a representative value representing the contents of a document group is metadata.

  Many of the blogs that are sufficiently popular nowadays have a function of adding metadata representing the content of an article as described above. A search service using the aggregation of metadata is also published as a tag search from Technorati.

  However, since the blog metadata is provided by bloggers themselves for organizing their articles, there are problems with many synonymous tags and notation fluctuations. Further, in the above service, since the relationship between tags is not taken into consideration, similar tags are scattered, and there is a problem in convenience.

  The similarity between metadata can be regarded as the similarity between document groups to which the metadata is assigned. On the other hand, as a technique for measuring the similarity between documents, as described in Patent Document 1 below, as a part of the topic word extraction method, the appearance frequency of the words in the document is used as the weight of the words and is shared. There is a technique of measuring the similarity in the range of 0 to 1 based on the phrase ratio.

  However, since the above method is a method for obtaining the similarity between documents and not for obtaining a similarity between document groups, it cannot be directly used for measuring the similarity between metadata.

  Therefore, as described in Non-Patent Document 1 below, by applying similarity measurement between documents calculated in the range of 0 to 1, the similarity between documents across document groups is used as an average value. There is a method to measure the similarity between blog metadata.

The residual IDF (Inverse Document Frequency), which is an index using the estimated document frequency based on the Poisson distribution, is known in Non-Patent Document 2 below, and the hierarchical clustering technique is described in Non-Patent Document 3 below. Those described in (1) are known.
JP 2004-258723 A Chirstopher H.M. Brooks, Nancy Montanez. Improve Annotation of the Blogosphere via Autotagging and Hierarchical Clustering. 15th International World Wide Web Conference, 2006.5. Kitakenji et al., “Information Retrieval Algorithm”, Kyoritsu Shuppan, January 1, 2002, first edition, first edition, pp. 41-45 Toshihiro Kamisu, “Clustering Method in Data Mining Field (1)-Let's Use Clustering!”, Journal of Artificial Intelligence Society, vol. 18, no. 1, pp. 59-65 (2003)

  In the technique of Non-Patent Document 1, since the similarity between articles is 0 when the same word is not used at all, there are many cases where the same word does not exist in the article unit, and between the metadata obtained by the average value There is a tendency that the degree of similarity also becomes a small value.

  Non-Patent Document 1 also shows the above tendency because the average similarity between blog articles with the same tag set is 0.2 to 0.4.

  Therefore, when determining whether or not the tags are similar, if the average value of the similarity between articles is the similarity between the tags, the measured value of the similarity has a narrow range of values. There is a problem that the result is greatly changed only by changing a little, and it is difficult to set the threshold value.

  An object of the present invention is to set a threshold by using a feature amount obtained based on the assumption that a phrase that appears biased in a specific document group is a characteristic phrase that represents the contents of the document group. Provides a similarity measurement device and method that make it easier to perform, and provides a metadata hierarchization device and method using this similarity measurement device to integrate and stratify highly related metadata This is to improve convenience.

  The present invention provides a method for obtaining the similarity of metadata in a document set to which metadata is assigned, using the scoring method of feature words in a document group when all document sets are given, and the similarity The target is clustering methods using. Here, as a document set to which metadata is given, a blog, a social bookmark service, or the like is mainly assumed.

  In these services, the user can freely set metadata, but on the other hand, there is a situation where metadata is flooded due to synonymous metadata and fluctuation of notation. As a solution, the idea of measuring the similarity between metadata is common, but there is no method for directly obtaining the feature amount of a document group in which metadata is set.

  On the other hand, the feature vector of each document in the document group is obtained by the TF / IDF method, etc., and the similarity between metadata is measured by the average cosine similarity of the document among the document group, A method of setting a feature vector of metadata by an average vector of feature vectors of documents is considered, but there is a problem of calculation efficiency and a problem that a general word weight becomes too strong by taking an average, respectively.

  Since the calculation method of the similarity is basically a cosine similarity based on the vector space model of the phrase, including the Patent Document 1 and the Non-Patent Document 1, the question is how to perform the phrase scoring method. Come back.

  As a method for obtaining a feature vector of a document group by using the document group as one document, the most easily conceivable method is a method using the TF / IDF method. The TF / IDF method is based on two assumptions: a phrase used multiple times in a document is important, and an assumption that a word used only in a specific document is characteristic.

  Here, by regarding the document group as one document, the types of phrases per document (here, corresponding to the document group) increase.

  Therefore, since the effect of the IDF method for removing common words is dull, general words also have large values due to the size of their TF values when the document group is regarded as one document. There is concern that it will have a large impact on the calculation of the degree.

  On the other hand, in the present invention, since a word weighting method based on the assumption that words that appear biased in a certain document group are important words is used, it is difficult for general words to become feature quantities. Improvement in accuracy can be expected.

  That is, it can be said that the present invention is an optimized method in a situation where a document group is determined in advance and the feature values of all documents can be calculated, for example, when articles with tagged blogs have been collected.

The inter-metadata similarity measuring apparatus according to claim 1 for solving the above-mentioned problem is a device for measuring the similarity between meta-data given to a document, and searches for the same-meta-data attached to the document. And a document search unit that performs a search and a score pair representing the relationship between a word in the document group and the content of the document of the word as a feature amount of metadata from the document group searched by the document search unit A vector extracting means for obtaining a probability that the word does not appear once in the document by Poisson distribution from the statistic of the word of the whole document set from the document group searched by the document searching means, The difference between the document frequency in the document group and the document frequency based on the Poisson distribution is obtained as a residual DF value, words having the residual DF value equal to or less than a predetermined value are excluded as insignificant words, and the phrase and residual D It is characterized by having a feature extraction means for outputting a vector pair was an element of the value as a feature quantity of documents, and a similarity calculation means for calculating the degree of similarity between the metadata using the feature quantity .

  Further, the metadata similarity measuring apparatus according to claim 2 is characterized in that, in claim 1, the feature amount extraction means uses the document frequency in the document group of the phrase and the target for all documents as the score. It is characterized by using the difference from the document frequency estimated by the Poisson distribution based on the appearance frequency of the phrase.

The method for measuring the similarity between metadata according to claim 6 is a method for measuring the similarity between metadata given to a document, wherein the document search means searches for a document to which the same metadata is assigned. The document search step to be performed and the feature amount extraction unit represent the relationship between the words in the document group and the content of the document of the phrase as the feature amount of the metadata from the document group searched by the document search step. A step of extracting a vector having a score pair as an element, and from the document group searched by the document search step, the word / phrase never appears in the document by Poisson distribution based on the statistic of the word / phrase of the whole document set. A step of obtaining a probability, a step of obtaining a difference between a document frequency in a document group of words and a document frequency based on a Poisson distribution as a residual DF value, and the residual DF value A step of excluding a word not important the following words value, the feature amount extraction step and a step of outputting the vector was paired elements of words and residual DF value as a feature quantity of documents, the similarity calculating unit Has a similarity calculation step of calculating a similarity between metadata using the feature amount extracted by the feature amount extraction step.

Further, the method for measuring the similarity between metadata according to claim 7 is the method according to claim 6 , wherein the feature amount extraction step includes, as the score, the document frequency in the document group of the phrase and all documents. It is characterized by using the difference from the document frequency estimated by the Poisson distribution based on the appearance frequency of the phrase.

  According to the above configuration, since the feature amount of the document group in which metadata is set is statistically obtained directly, the similarity can be obtained with high accuracy, and the similarity between the metadata is reduced to a small value. Since the difference in similarity can be widened, the threshold value can be easily set.

  Further, the metadata similarity measurement apparatus according to claim 3 is characterized in that, in claim 1 or 2, the feature amount extraction unit reduces the extreme score disparity between words as the score. It is characterized by adopting the logarithm of the numerical value for the quantity as the score.

  Further, in the metadata similarity measuring apparatus leaf according to claim 4 or any one of claims 1 to 3, the feature amount extraction unit eases an extreme disparity in scores due to document amount as the score. Therefore, the score obtained by normalizing using the number of documents in the document group is used as a score.

In addition, the method for measuring the similarity between metadata according to claim 8 is characterized in that, in the feature extraction step according to claim 6 or 7 , the feature amount extraction step reduces the extreme score disparity between words as the score. It is characterized by adopting the logarithm of the numerical value for the quantity as the score.
The metadata similarity measuring method according to claim 9 is the method according to any one of claims 6 to 8 , wherein the feature amount extraction step alleviates an extreme disparity in scores due to document amount as the score. Therefore, the score obtained by normalizing using the number of documents in the document group is used as a score.

  According to the above configuration, even if the number of documents increases, the difference in score does not become extremely large, and the reliability is improved.

Further, the metadata hierarchization apparatus according to claim 5 integrates similar metadata as a cluster in addition to the inter-metadata similarity measurement apparatus according to any one of claims 1 to 4 , and hierarchizes the same. , And having a combination / hierarchization means.

Further, the metadata hierarchization method according to claim 10 is similar to the steps of the inter-metadata similarity measurement method according to any one of claims 6 to 9 , and the combination / hierarchization means includes similar meta data. It is characterized by having a joining / hierarchizing step for integrating and hierarchizing data as clusters.

  According to the above configuration, highly relevant metadata can be combined and hierarchized, and convenience is improved.

The meta-data-similarity-measuring program according to claim 11, it has a program for executing the steps of the meta-data-similarity measuring method according to the computer in any one of claims 6 to 9 It is characterized by.

A metadata hierarchization program according to claim 12 is a program for causing a computer to execute each step of the metadata hierarchization method according to claim 10 .

According to a thirteenth aspect of the present invention, there is provided a recording medium on which the inter-metadata similarity measurement program is recorded, wherein the program according to the eleventh aspect is recorded on a recording medium readable by the computer.

A recording medium recording the metadata hierarchization program according to claim 14 is characterized in that the program according to claim 12 is recorded on a recording medium readable by the computer.

(1) According to the inventions described in claims 1 to 14 , since the feature amount of the document group in which the metadata is set is statistically obtained directly, the similarity can be obtained with high accuracy. Since the similarity between metadata does not become a small value, and the disparity in similarity can be widened, the threshold value can be easily set.

  Further, by using the similarity obtained by the present invention, for example, a search system using a tag of a blog can provide a user with a similar tag with higher accuracy.

Furthermore, by gathering similar blog tags, you can obtain a larger group of blog articles with similar concepts as a result. It can be applied to marketing, such as finding out where the area is.
(2) According to the inventions described in claims 3, 4, 8 , and 9 , even if the number of documents increases, the difference in scores does not become extremely large, and the reliability is improved.
(3) According to the inventions described in claims 5 and 10 , highly relevant metadata can be combined and hierarchized, and convenience is improved.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings, but the present invention is not limited to the following embodiments. First, an apparatus and method for measuring the similarity between metadata will be described.

  FIG. 1 is a diagram showing a configuration of an inter-metadata similarity measuring apparatus which is an example of an embodiment of the present invention. The inter-metadata similarity measuring apparatus 10 according to the present embodiment includes a general computer device that operates based on a predetermined program, and is a document to which specific metadata is assigned from a metadata-added document DB (database) 14. Between the target metadata using the document search means 11 for searching for the feature, the feature quantity extraction means 12 for extracting the feature quantity of the metadata, and the metadata feature quantity DB (database) 15 storing the obtained feature quantity. It is comprised from the similarity calculation means 13 which measures the similarity of these.

  As shown in FIG. 2, the overall processing flow first searches for documents to which metadata for measuring similarity is added to obtain a document group (step S11). Next, metadata feature amounts are extracted using the obtained statistic values of the phrases in the document group and the statistic values of the phrases obtained from all the documents included in the metadata-added document DB 14 (step S12). Further, a similarity is calculated between the feature quantities of the obtained metadata, and the value is output as the similarity (step S13).

  In the following, detailed description of each means will be given based on the processing procedure shown in FIG. The document retrieval unit 11 retrieves a document to which the same metadata as the specified metadata is assigned from a set of documents to which the metadata is assigned, as in a general search system that has been used conventionally. The result is output as a list to the next means.

  Here, the number of documents to be passed to the next means as a search result may be reduced to about several thousand to several tens of thousands in order to reduce the calculation amount.

  As shown in FIG. 3, the metadata-added document DB 14 holds document ID, metadata, and text as data in a form in which each document is associated. Here, the document ID is an identifier uniquely given to each document. Metadata is a word that expresses the content of the text. The body is the text of the document itself. Further, as in the URL in FIG. 3, additional information can be added to perform association and hold data.

  The feature amount extraction unit 12 uses the statistics of each word in the document group to which the specific metadata obtained by the document search unit 11 and the statistics of each word in the entire document set are used to calculate the metadata. Extract features. Here, the feature quantity is expressed as a vector having a phrase and a pair of residual DF (Document Frequency) values indicating whether or not the phrase represents the contents in the document group.

The residual DF value is represented by the difference between the document frequency of the phrase in the document group and the document frequency in the document group estimated by the Poisson distribution. For example, if the total number of documents in the search result is n, the document frequency of phrase i in the document group is df _i , the total number of documents is N, and the global frequency of phrase i in all documents is F _i , the residual The DF value is obtained by the following equation (1).

  As an example, the residual DF value of the phrase “J-League” in the blog entry group to which the category “soccer” is assigned is the number of collected blog entries 400000 and the number of blog entries to which the category “soccer” is assigned. Is 1000, the document frequency of "J League" in this entry group is 300, and the global frequency of "J League" in the collected blog entries is 700, the residual DF is 300-1000 (1-exp. (−700/400000)) = 298.25. . . become that way.

An example of the processing flow of the feature quantity extraction means 12 is shown in FIG. In FIG. 4, first, a document group is received as an input from the document search means 11 (step S121). Next, from the group of documents searched by the document search means 11, the probability P = e ^{−Fi / N} that the ^{word /} phrase does not appear in the document is obtained by the Poisson distribution from the statistic of the word / phrase of the whole document set (step S122). .

  Next, the difference between the document frequency in the document group of the phrase and the document frequency n (1-P) based on the Poisson distribution is obtained as a residual DF value (step S123). Next, words having a residual DF value of 1.0 or less are excluded as insignificant words (step S124). Next, a vector having a pair of a word and a residual DF value as an element is output as a feature amount of the document group (step S125).

  As an index using the estimated document frequency based on the Poisson distribution, there is a residual IDF (Inverse Document Frequency) (see Non-Patent Document 2). Conventionally, the residual IDF has a problem that a word with a large global frequency results in a large DF value and a small IDF value, so that the TF (Term Frequency) IDF method, which is a word weighting method, does not work well. Has been used to solve. Since important words are used multiple times in the same document, this is based on the assumption that the actual DF value is smaller than the DF value estimated by the Poisson distribution.

  On the other hand, a group of documents to which the same metadata is assigned is likely to be similar in terms of content and phrase used. In the residual DF value in the feature amount extraction unit 12 of the present embodiment example, in a document group to which the same metadata is assigned, a phrase that expresses the content appears biased to the documents in the document group. It is likely that the DF value is different in that it is based on the assumption that it takes a value larger than the DF value estimated by the Poisson distribution.

  That is, the residual IDF value is a method for examining characteristic words in the entire document set, and the residual DF value in the feature amount extraction unit 12 of the present embodiment is a feature in a specific document group. It is a technique for examining typical words. As described above, the objects for examining characteristic words are different.

  Further, since the residual DF value can be intuitively grasped as the frequency of the document, there is an advantage that it is easy to understand the meaning of the numerical value. The residual DF value has a feature that the difference in value becomes very large as the number of documents to which the same metadata is assigned increases. Therefore, a method of logarithmizing the residual DF value in order to reduce the disparity or adopting a value normalized by the number of documents as a feature amount is also conceivable. Here, as an example of normalization, there is a method of simply dividing the residual DF value by the number of documents.

  As shown in FIG. 5, the metadata feature DB 15 holds metadata, phrases, and scores as data in a form in which each metadata is associated. Here, the metadata is an identifier. The phrase is a phrase used in the document group to which the metadata is assigned. The score is calculated by the feature amount extraction unit 12.

The similarity calculation unit 13 is a unit for calculating the similarity between metadata using the feature amount of the metadata obtained by the feature amount extraction unit 12. Here, since the feature amount of the metadata is a vector having a pair of scores including a phrase and a residual DF value as an element, the similarity between the metadata is obtained by a cosine of the vectors. For example, when C _i and C _j are feature vectors of metadata, the similarity is expressed by the following equation (2).

  Next, a metadata hierarchization apparatus and method will be described. FIG. 6 is a diagram showing a configuration of a metadata hierarchization apparatus which is an example of an embodiment of the present invention. The metadata hierarchization apparatus 20 according to the present embodiment includes a general computer apparatus that operates based on a predetermined program. For example, the inter-metadata similarity measurement includes the inter-metadata similarity measurement apparatus 10 of FIG. A device 21 and a combination / hierarchization unit 22 that uses a metadata similarity DB (database) 23 for storing the obtained similarity and combines the metadata in descending order of similarity to form a hierarchy. Has been.

  As shown in FIG. 7, the overall processing flow is as follows. First, similarity between metadata is calculated by the inter-metadata similarity measuring device 21 using a metadata-added document (step S21). . Next, metadata is combined and hierarchized using a hierarchical clustering technique (step S22).

  In the following, a detailed description of each means will be given based on the processing procedure shown in FIG. As shown in FIG. 8, the metadata similarity DB 23 holds metadata 1, metadata 2, and similarity as data in an associated form. Here, the metadata 1 and 2 are two pieces of metadata that are targets of similarity calculation. The similarity is a similarity calculated by the similarity measuring device 21.

  Further, FIG. 9 shows an example in which the similarity between metadata is visualized by a graph using the metadata similarity DB 23. This is a graph that visualizes the similarity between the category “gourmet” and the top 200 categories for the set number of people based on the data for one month of the blog so that the left side of the x-axis is the highest for the set number of people. Here, only the names of tags with a similarity of 0.6 or higher are displayed, but the similarity is high although it is considered intuitively related to the contents.

  FIG. 10 and FIG. 11 are graphs visualizing the similarity between the categories “daily life” and “movie” and the top 200 categories with the same number of people as in FIG.

  From these results of FIGS. 9 to 11, it can be seen that the similarity between categories considered to indicate the same concept by notation fluctuations and synonyms is particularly high. In addition, it can be seen that a category such as “daily life” with an ambiguous concept has a lower similarity peak than “movie” or “gourmet”.

  As described above, according to the present embodiment example, a difference between a highly related category and a less related category appears remarkably, so that a good similarity can be obtained.

  The combining / hierarchizing means 22 combines the metadata having the same meaning into one set by combining the metadata into one set in descending order of the similarity based on the obtained similarity. It is a means for giving a hierarchy to metadata. Specifically, a technique called hierarchical clustering is used (see Non-Patent Document 3). In the hierarchical clustering method, there are differences depending on how the distance between clusters is defined, and there are a shortest distance method, a longest distance method, a group average method, and the like. Any method may be used in this means.

  As an example, FIG. 12 shows a part of the result obtained by clustering the blog category by the shortest distance method using the similarity obtained by the similarity measuring device 21. FIG. 12 is a diagram showing the growth of the clusters of the category related to “food”, and it can be seen that a hierarchy is formed by combining one after another.

  The clustering end condition setting method includes a method that ends when the number of clusters reaches a preset threshold, a method that uses a preset similarity as a threshold, and a method that combines both thresholds. .

  Various embodiments can be considered in addition to the examples shown above. For example, a method of using a part of the higher rank of the existing classification system at the time of hierarchization and combining metadata having a high similarity at the end thereof can be considered to make the problem of hierarchization easier.

  In addition, a program for causing a computer to execute the above-described similarity measurement method between metadata and metadata hierarchization method is constructed.

  It is also possible to supply a recording medium recording the program to a system or apparatus, and the CPU (MPU) of the system or apparatus reads and executes the program stored in the recording medium. In this case, the program itself read from the recording medium realizes the functions of the above-described embodiment, and examples of the recording medium on which the program is recorded include CD-ROM, DVD-ROM, CD-R, CD- There are RW, MO, and HDD.

The block diagram of the similarity measuring apparatus between metadata of one embodiment of this invention. The flowchart which shows the flow of a process of the similarity measuring apparatus between metadata of one embodiment of this invention. Explanatory drawing which shows an example of document DB with metadata used by one Example of this invention. The flowchart which shows the flow of a process of the feature-value extraction means of one embodiment of this invention. Explanatory drawing which shows an example of metadata feature-value DB used by one Embodiment of this invention. The block diagram of the metadata hierarchy apparatus of the other embodiment of this invention. The flowchart which shows the flow of a process of the metadata hierarchization apparatus of the other embodiment of this invention. Explanatory drawing which shows an example of metadata similarity DB used in the other embodiment of this invention. The graph which shows the similarity regarding the category "gourmet" by the embodiment of this invention. The graph which shows the similarity regarding the category "daily" by the example embodiment of this invention. The graph which shows the similarity regarding the category "movie" by the example embodiment of this invention. Explanatory drawing which shows an example of the clustering result by other embodiment examples of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10,21 ... Metadata similarity measuring apparatus, 11 ... Document search means, 12 ... Feature quantity extraction means, 13 ... Similarity calculation means, 14 ... Document DB with metadata, 15 ... Metadata feature quantity DB, 20 ... Metadata hierarchization apparatus, 22 ... combination / hierarchization means, 23 ... metadata similarity DB.

Claims (14)

In a device that measures the similarity between metadata assigned to documents,
A document search means for searching for documents with the same metadata;
Means for extracting, from the document group searched by the document search means, as a feature quantity of metadata, a vector having as an element a pair of scores representing the relationship between a word / phrase in the document group and the document content of the word / phrase. Then, from the document group searched by the document search means, the probability of the word not appearing in the document by the Poisson distribution is obtained from the statistic of the word of the whole document set, and the document in the document group of the word A difference between the frequency and the document frequency based on the Poisson distribution is obtained as a residual DF value, a phrase having the residual DF value equal to or smaller than a predetermined value is excluded as an insignificant word, and a vector having a pair of the phrase and the residual DF value as elements A feature amount extraction means for outputting a document group feature amount
A similarity calculation unit between metadata, comprising: similarity calculation means for calculating similarity between metadata using the feature quantity. The feature amount extraction means includes:
The difference between a document frequency in a document group of a word and a document frequency estimated by a Poisson distribution based on an appearance frequency of the word for all documents is used as the score. The apparatus for measuring similarity between metadata described in 1. The feature amount extraction means includes:
3. The meta according to claim 1, wherein the score is a logarithm of the numerical value of the feature amount of the phrase as a score in order to reduce an extreme score disparity between the phrases. Data similarity measurement device. The feature amount extraction means includes:
4. The score according to claim 1, wherein the score obtained by normalizing using the number of documents in the document group is adopted as the score in order to alleviate an extreme difference in score due to the amount of documents. The apparatus for measuring the similarity between metadata described in the paragraph. In addition to the meta-data-similarity-measuring device according to any one of claims 1 to 4, it integrates metadata similar as a cluster, stratified, and having a coupling-layered unit meth Data tiering device. In a method for measuring the similarity between metadata assigned to documents,
A document search step in which the document search means searches for a document with the same metadata;
The feature amount extraction means uses, as an element, a pair of scores representing a relationship between a word / phrase in the document group and the content of the document of the word / phrase as a feature amount of metadata from the document group searched by the document search step. A step of extracting a vector obtained from the group of documents searched by the document search step, and determining a probability that the word / phrase never appears in the document by Poisson distribution from a statistic of the word / phrase of the whole document set; A step of obtaining a difference between a document frequency in a document group of a word and a document frequency based on a Poisson distribution as a residual DF value, a step of excluding a word whose residual DF value is a predetermined value or less as an insignificant word, And a step of outputting a vector having a pair of residual DF values as an element as a feature amount of the document group ,
A similarity calculation means comprising: a similarity calculation step in which similarity calculation means calculates a similarity between metadata using the feature amount extracted in the feature amount extraction step. The feature amount extraction step includes:
As the score, claim, characterized in that use and document frequency in a document group of words, the difference between the document frequency estimated by the Poisson distribution based on the appearance frequency of the word intended for the entire document 6 Method for measuring the similarity between metadata described in 1. The feature amount extraction step includes:
As the score, in order to relieve the extreme score disparity between phrases, the logarithm of the numerical value for the feature amount of the phrase is adopted as the score.
The method for measuring similarity between metadata according to claim 6 or 7 . The feature amount extraction step includes:
As the score, to alleviate extreme disparity in the score by document size, either 6 to claim, characterized in that to adopt what was normalized using the number of documents in the document group as a score of 8 1 The method for measuring the similarity between metadata described in the paragraph. In addition to each step of the method for measuring similarity between metadata according to any one of claims 6 to 9 , a combination / hierarchy unit combines and hierarchizes similar metadata as clusters. A metadata hierarchization method characterized by comprising a conversion step. 10. A program for measuring similarity between metadata, which is a program for causing a computer to execute each step of the method for measuring similarity between metadata according to any one of claims 6 to 9 . 11. A metadata hierarchization program characterized by being a program for causing a computer to execute each step of the metadata hierarchization method according to claim 10 . A recording medium recording a similarity measurement program between metadata, wherein the program according to claim 11 is recorded on a recording medium readable by the computer. 13. A recording medium recording a metadata hierarchization program, wherein the program according to claim 12 is recorded on a recording medium readable by the computer.

Images