JP2010117764A - Inter-word association degree determining device, inter-word association degree determination method, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an inter-word association degree determination device for efficiently and successively calculating the approximate value of an association degree between two words, i.e., efficiently performing integration processing which is to be originally increased in proportion with a dimension number, without significantly deteriorating the overall accuracy. <P>SOLUTION: A corpus is input, and the co-occurrence frequency between a word and a word meaning attribute with respect to the dimension of the word is tabulated by each arbitrary word, so as to generate a frequency vector. The frequency vector is normalized as a probability, according to a tree structure held by the word meaning attribute so as to generate a concept vector. A local distance, which is obtained by calculating a distance between the two concept vectors acquired with respect to the two arbitrary words, based on the probability corresponding to a master node and the probability corresponding to a slave node according to the tree structure, is integrated from a root node to a leaf node, thereby calculating the association degree between the two words. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an inter-word relevance determination device, an inter-word relevance determination method, a program, and a recording medium that efficiently calculate the relevance between words.

  A concept base is used as one of databases for determining similarity between words, searching for synonyms, and searching for related documents.

  This “concept base” is a database composed of pairs of words and concept vectors corresponding to the words. As a concept base, a dictionary concept base that is created based on a word meaning sentence in a Japanese dictionary is known (for example, see Patent Document 1).

  Further, as a concept base, a corpus concept base created based on a corpus obtained by collecting a large number of documents such as newspaper articles is known (for example, see Non-Patent Document 1).

  The “concept vector” of the predetermined word is calculated according to the frequency of co-occurrence with each of a plurality of predetermined co-occurrence words within the range (eg, sentence) to which the predetermined word belongs.

  As a co-occurrence word in the dictionary concept base, a word appearing in a word meaning sentence obtained by lexicographically extracting the word is used, and as a co-occurrence word in the corpus concept base, a word appearing frequently in the corpus is used. A co-occurrence matrix is created in which each word is a row, a co-occurrence word is a column, and the co-occurrence frequency of the word and the co-occurrence word is a matrix component.

  In the case of the dictionary concept base, the row vector of each row in the co-occurrence matrix is a concept vector of a word, and usually, using a word meaning sentence obtained by subtracting the word included in the word meaning sentence, The concept vector is refined.

  In the corpus concept base, a matrix in which the dimension of the column of the co-occurrence matrix is compressed by singular value decomposition, and the row vector of each row of the compressed matrix is a concept vector.

The concept base created in this manner has the property that the higher the similarity between words, the closer the distance between the concept vectors of the words, so it is effective in determining similarity between words. That is, it can be determined that the closer the concept vector between two words is, the higher the similarity between the two words is.
Japanese Patent No. 3379603 H. Schutze, “Dimensions of meaning”, Proceedings of Supercomputing '92, pp.787-796, 1992

  In the concept base created by these methods, the number of dimensions of the concept vector greatly affects the accuracy of the distance calculation and the calculation time.

  In other words, in the above conventional example, if the number of dimensions is increased in order to improve the accuracy of distance calculation, a calculation time proportional to the number of dimensions is required. Conversely, if the number of dimensions is decreased in order to reduce the calculation time. However, there is a problem that originally necessary information is missing from the concept vector and the accuracy of distance calculation is lowered.

  The present invention can efficiently calculate an approximate value of the degree of association between two words, that is, an integration process that increases in proportion to the number of dimensions without significantly reducing the overall accuracy. An object of the present invention is to provide an inter-word relevance determination device, an inter-word relevance determination method, a program, and a recording medium that can be made efficient.

  The present invention inputs a corpus and, for an arbitrary word, frequency vector generation means for generating a frequency vector by counting the co-occurrence frequencies of the word and the word semantic attribute corresponding to each dimension, and the word semantic attribute In accordance with the tree structure included in the above, the frequency vector is generated with respect to two arbitrary words, a concept vector generation unit that normalizes the frequency vector as a probability to generate a concept vector, a storage unit that stores the concept vector, and The distance between two concept vectors obtained by applying the means and the concept vector generation means is calculated based on the probability corresponding to the parent node and the probability corresponding to the child node according to the tree structure. An inter-word relevance calculating means for calculating the relevance between the two words by integrating the local distance from the root node toward the leaf node; A word relevancy determining apparatus having.

  According to the present invention, each dimension of the concept vector is associated with each node of the tree structure of the word semantic attribute, and is calculated based on the node value corresponding to the parent node and the node value corresponding to the child node. Even if the local distance is integrated from the root node toward the leaf node, and this integration is stopped halfway, there is an effect that relatively good accuracy is guaranteed. In other words, according to the present invention, the calculation amount can be reduced if the integration is stopped halfway, and the calculation amount can be reduced while maintaining the accuracy of the approximate value of the degree of association between the two words. There is an effect.

  The best mode for carrying out the invention is the following examples.

  FIG. 1 is a block diagram illustrating an inter-word relevance determination apparatus 100 that is Embodiment 1 of the present invention.

  The inter-word association degree determination device 100 includes a corpus input unit 10, a frequency vector generation unit 20, a concept vector generation unit 30, a concept vector storage unit 40, and an inter-word association degree calculation unit 50.

  A corpus to be processed is input via the corpus input means 10. The frequency vector generation means 20 generates a frequency vector using the input corpus. The concept vector generation means 30 normalizes the frequency vector as a probability and stores it in the concept vector storage means 40.

  The above-mentioned “normalize frequency vectors as probabilities” means that the frequency vectors are (x1, x2,..., XN), and the sum of each element is S, the frequency vectors (x1, x2,. ) Is converted to (x1 / S, x2 / S,..., XN / S), and the sum of the elements is set to 1.

  Finally, the inter-word relevance calculation means 50 calculates the relevance between the words.

  Next, the frequency vector generation means 20 will be described.

  FIG. 2 is a block diagram showing the frequency vector generation means 20.

  The frequency vector generation unit 20 includes a sentence division unit 21, a word extraction unit 22, a word-word meaning attribute co-occurrence counting unit 23, and a word-word meaning attribute dictionary 24.

  The sentence dividing means 21 divides the input corpus into sentences. When a corpus includes a plurality of punctuation marks “.” And symbols “?”, “.”, Etc. used at the end of a sentence, the corpus is divided immediately after this. The word extracting means 22 is composed of a morphological analyzer and divides a sentence into words. Based on the part-of-speech information of each word simultaneously output from the morphological analyzer, words other than the content word are removed. From the extracted word, the word-word meaning attribute dictionary 24 is referred to, and the corresponding word meaning attribute is extracted.

  FIG. 3 is a diagram illustrating an example of the word-word meaning attribute dictionary 24 according to the first embodiment.

  The words are classified in advance into a plurality of categories according to their semantic attributes. Between each word meaning attribute, for example, as shown in FIG. 3, a word and a corresponding word meaning attribute are described. For example, Non-Patent Document 2 (Satoru Ikehara, Masahiro Miyazaki, Atsushi Shirai, Akio Yokoo, Hiroaki Nakaiwa, Kentaro Ogura, Yoshifumi Oyama, Yoshihiko Hayashi, “Japanese vocabulary system 1 semantic system”, Iwanami Shoten, 1999 )It is described in.

  As shown in FIG. 3, the word meaning attribute “election” is given to “general election”, and this attribute corresponds to the node 1804 in FIG. As attributes higher than the “election” attribute, there are “promotion”, “personnel”, “management”, “dominance”, “action”, “human activity”, “thing”, “abstract”, and “noun”.

  FIG. 13 is a diagram illustrating an example of the structure of word meaning attributes (portion related to the “secretary general”) in the first embodiment.

  FIG. 14 is a diagram illustrating an example of a word meaning attribute structure (portion related to “command”) in the first embodiment.

  Note that “secretary chief” and “command” correspond to the respective nodes in FIGS. 13 and 14 in the same manner as described above. That is, as shown in FIG. 3, the “secretary general” is given the word meaning attributes “long”, “politician”, and “curator”, and these attributes are numbered 323, 260, 167 in FIG. Corresponds to the numbered node. For example, “person <position>”, “person”, “subject”, “specific”, and “noun” are attributes higher than the “long” attribute. Further, as shown in FIG. 3, the word meaning attributes of “management”, “command”, and “creation (other)” are given to “command”, which are 1779, 1824, 1559 in FIG. Corresponds to the numbered node. For example, “dominance”, “action”, “human activity”, “thing”, “abstract”, and “noun” are attributes higher than the “management” attribute.

  Based on these, the word-word meaning attribute co-occurrence counting means 23 totals the word-word meaning attribute co-occurrence frequency as a frequency vector.

  FIG. 4 is a diagram illustrating an example of a structure of word meaning attributes in the first embodiment.

  Here, the “each dimension of the frequency vector” corresponds to each node obtained by serializing the tree structure as shown in FIG. Note that serializing the tree structure means arranging nodes in numerical order.

  FIG. 4 is a diagram showing a part of the tree structure, but as a whole, includes 2715 nodes corresponding to word semantic attributes (see Non-Patent Document 2 above). Therefore, the frequency vector is a 2715-dimensional vector.

  An example of counting the co-occurrence frequencies of word-word semantic attributes is shown below.

  For the frequency vector of each word extracted from the sentence, 1 is added to the word semantic attribute of another word appearing in the same sentence and the dimension corresponding to the higher attribute. For example, if the word “general election” appears in a sentence in which the word “ruling party” included in the corpus is written, the frequency vector dimensions 1804, 1802, 1793, 1779, 1778, “ For 1560, 1236, 1235, 1000, 1 is added.

  Note that the word meaning attribute “election” (see FIG. 3) of “general election” is noted, and the relationship in the tree structure of FIG. 4 is referred to.

  Assuming that words appearing in the same sentence as “general election” have a meaning related to “general election”, the attribute of “general election” is given to these words. Specifically, one point is added to each dimension 1804, 1802, 1793, 1779, 1778, 1560, 1236, 1235, 1000, 1 corresponding to the attribute of “general election”. When “secretary chief” and “command” appear in the same sentence, the attributes of “secretary chief” and “command” are given in the same manner as described above. As shown in FIG. 5, the dimension to which one point is added at this time is finally given a high score to the dimension that best represents the concept of the sentence.

  The same operation as described above is performed for all corpora, and frequency vectors are obtained for all words included in the corpus.

  An example of finding the frequency vector of the word “the ruling party committee” with the sentence “The ruling party has led the general election by the secretary” as a corpus.

  In this example, “the ruling party”, “secretary general”, “general election”, and “command” are extracted as content words. The word semantic attributes of “secretary chief”, “general election”, and “command” co-occurring with “the ruling party” are defined as shown in FIG.

  FIG. 5 is a diagram illustrating an example of words and upper word meaning attributes according to the first embodiment.

  An example of obtaining upper word semantic attributes of these words according to the tree structure shown in FIG. 4 is an example of the word and upper word semantic attributes shown in FIG.

  FIG. 6 is a diagram showing the frequency of appearance of word semantic attributes for each word of “secretary chief”, “general election”, and “command”, and showing the totaled appearance frequency on a tree structure.

  In FIG. 6, the notation “= 0” is omitted for a node having a value of 0.

  FIG. 7 is a diagram illustrating an example of a frequency vector in the first embodiment.

  By serializing the tree structure shown in FIG. 6 (arranging nodes in numerical order), the frequency vector shown in FIG. 7 can be obtained.

  Next, the concept vector generation means 30 will be described in detail.

  FIG. 8 is a block diagram showing the configuration of the concept vector generation means 30.

  First, the vector-tree structure conversion means 31 converts the frequency vector into a tree structure by an operation reverse to the above serialization.

  Next, the sibling node normalizing means 32 normalizes each sibling node. Specifically, the value of the root node is first set to 1. The value of each node is defined in proportion to the total value so that the sum of the values of the child nodes immediately below the root node is 1. Similarly, the value is defined in proportion to the total value so that the sum of the values of the child nodes immediately below each parent node is 1. The value of each node corresponds to a conditional probability having a semantic attribute corresponding to the node at the same time when the focused word has a semantic attribute corresponding to the parent node of the node.

  The conditional probability is a number described to the right of “=” described to the right of the word in FIG. 9 described later.

  Finally, the tree structure-vector conversion means 33 serializes the tree structure again to obtain a concept vector.

  FIG. 9 is a diagram illustrating an example of concept vector calculation in the first embodiment.

  FIG. 10 is a diagram illustrating an example of concept vectors in the first embodiment.

  In the above example, the frequency vector (FIG. 7) corresponding to the tree structure shown in FIG. 6 is converted into a tree structure, normalized for each sibling node (FIG. 9), and serialized again, so that FIG. The concept vector shown can be obtained.

  Next, the inter-word relevance calculating means 50 will be described in detail.

  Two concept vectors

を、以下のように計算する。

Is calculated as follows.

As described in Equation 1, here, the degree of association between the two concept vectors v and w is calculated, and the set of all internal nodes is S, and v _k and w _k are those of the concept vectors v and w. The value of each node k.

A set of child nodes directly under the node k and S _k. At this time, the distance between child nodes (sibling nodes)

をＫｕｌｌｂａｃｋ−Ｌｅｉｂｌｅｒ距離

Kullback-Leibler distance

で定義し、この親ノードｋと、その直下の子ノードとに関する局所的な距離

Local distance between this parent node k and its immediate child nodes

を、

The

で定義する。ここで、

Define in. here,

は、親ノードｋとその上位にある根ノードを含む全てのノードの集合をＲ_ｋとすると、

R _k is a set of all nodes including the parent node k and the root node above it.

とする。なお、上記式（１）の総和は、概念ベクトルｖ、ｗのそれぞのノードｉの値であるｖ_ｋとｗ_ｋとが、いずれも０でないｉについて演算したものである。

And Note that the sum of the above formula (1) is calculated for i in which v _k and w _k which are values of the respective nodes i of the concept vectors v and w are not 0.

Instead of the Kullback-Leibler distance, by introducing α divergence and appropriately setting the parameter α, it is possible to obtain a degree of relevance that emphasizes the higher and lower concepts of the word. This is described in, for example, Non-Patent Document 3 (T. Minka, “Divergence measures and message passing,” MSR-TR-2005-173, 2005.) In this case, the distance between child nodes (sibling nodes)

は

Is

で定義され、親ノードｋと、その直下の子ノードとに関する局所的な距離

Local distance between parent node k and its immediate child nodes

は、

Is

として定義される。

Is defined as

The sum of the above equation (4) is calculated for i in which v _k and w _k which are values of the respective nodes i of the concept vectors v and w are not 0, similarly to the sum of the above equation (1). Is done.

  From the root node to the leaf node,

を積算することによって、２つの概念ベクトル

By multiplying the two concept vectors

を

The

として評価する。

Evaluate as

  The process of executing the above equation (6) from the root node to the leaf node is a process of integrating the degree of association between words from the higher attribute of the word semantic attribute toward the lower attribute.

  The value obtained in the middle of this process can be used as an appropriate approximate value of the degree of association between words in consideration of the word semantic attribute from the top.

FIG. 11 is a diagram illustrating an example of a set R ₁₇₇₉ of all nodes above the node ₁₇₇₉ and a set S ₁₇₇₉ of child nodes immediately below the node 1779 when attention is paid to the node 1779.

  FIG. 12 is a flowchart illustrating the operation of the first embodiment.

  In S1, a corpus to be processed is input. In S2, a frequency vector is generated from an input corpus by counting the co-occurrence frequencies of the word and the word semantic attribute corresponding to each dimension for an arbitrary word, and stored in the storage device.

  In S3, a concept vector is generated by normalizing the frequency vector for each sibling node according to the tree structure of the word meaning attribute, and stored in S4.

  Based on the stored concept vector, in S5, the degree of association between words is calculated according to the above formulas (1) to (6) and stored in the storage device.

  According to the above embodiment, since the similarity determination is accumulated from the root toward the leaf, it can be determined that the similarity is not similar in the middle of the node. Since it is not necessary to process up to (child node), the processing time is shortened.

  In the above embodiment, if the means is replaced with a process, the above embodiment is an example of the invention of the method for determining the degree of association between words.

  Further, the frequency vector generation means 20 is a frequency vector generation means for inputting a corpus and generating a frequency vector by counting the co-occurrence frequencies of the word and the word meaning attribute corresponding to each dimension for an arbitrary word. It is an example.

  The concept vector generation unit 30 is an example of a concept vector generation unit that generates a concept vector by normalizing the frequency vector as a probability according to a tree structure included in the word meaning attribute.

  The concept vector storage unit 40 is an example of a storage unit that stores a concept vector.

  The word-to-word relevance calculating unit 50 calculates the distance between two concept vectors obtained by applying the frequency vector generating unit and the concept vector generating unit to any two words according to the tree structure. A word for calculating the degree of association between the two words by accumulating the local distance calculated based on the probability corresponding to the parent node and the probability corresponding to the child node from the root node toward the leaf node. It is an example of an inter-relationship degree calculation means.

  The inter-word relevance calculating means is a means for interrupting the integration when the approximate value of the relevance between the two words exceeds a preset value, thereby improving the accuracy of the relevance between the two words. It is possible to reduce the calculation time of the relevance degree without greatly damaging the. Originally, the integration process that increases in proportion to the number of dimensions can be made efficient.

  The inter-word relevance calculation means is a means for performing the above integration up to the last child node of the tree structure, and thereby the relevance between the two words is the highest.

1 is a block diagram illustrating an inter-word association degree determination device 100 that is Embodiment 1 of the present invention. FIG. FIG. 4 is a block diagram showing a frequency vector generation means 20. It is a figure which shows the example of the word-word meaning attribute dictionary 24 in Example 1. FIG. It is a figure which shows the example of the structure of the word meaning attribute in Example 1. FIG. It is a figure which shows the example in the word in Example 1, and a high-order word meaning attribute. It is a figure which totals the appearance frequency of a word meaning attribute about each word of "secretary chief", "general election", and "command", and shows the tabulated appearance frequency on a tree structure. It is a figure which shows the example of the frequency vector in Example 1. FIG. 3 is a block diagram showing a concept vector generation means 30. FIG. It is a figure which shows the example of the concept vector calculation in Example 1. FIG. It is a figure which shows the example of the concept vector in an Example. FIG. 38 is a diagram illustrating an example of a set R ₁₇₇₉ of all nodes above the node ₁₇₇₉ and a set S1779 of child nodes immediately below the node ₁₇₇₉ . 3 is a flowchart showing the operation of the first embodiment. It is a figure which shows the example of the structure of the word meaning attribute in Example 1 (part related to "secretary general"). It is a figure which shows the example of the structure (part relevant to "command") of the word meaning attribute in Example 1. FIG.

Explanation of symbols

100: Inter-word relevance determination device,
10 ... Corpus input means,
20 ... Frequency vector generation means,
21 ... sentence dividing means,
22 ... word extraction means,
23 ... Word-word semantic attribute co-occurrence counting means,
24 ... Word-word semantic attribute dictionary,
30 ... concept vector generation means,
31: Vector-tree structure conversion means,
32 ... brother node normalization means,
33 ... Tree structure-vector conversion means,
40. Concept vector storage means,
50: Inter-word relevance calculation means.

Claims (8)

A frequency vector generation means for inputting a corpus and generating a frequency vector for an arbitrary word by counting the co-occurrence frequencies of the word and the word semantic attribute corresponding to each dimension;
Concept vector generation means for generating a concept vector by normalizing the frequency vector as a probability according to the tree structure of the word semantic attribute;
Storage means for storing the concept vector;
The distance between two concept vectors obtained by applying the frequency vector generation means and the concept vector generation means to any two words is represented by the probability and child corresponding to the parent node according to the tree structure. An inter-word relevance calculating means for calculating the relevance between the two words by integrating the local distance calculated based on the probability corresponding to the node from the root node toward the leaf node;
A device for determining the degree of association between words, comprising: In claim 1,
The inter-word relevance calculation means is a means for interrupting the integration when the approximate value of the relevance between two words exceeds a preset value. In claim 1,
The inter-word relevance calculation means is a means for executing the above integration up to the last child node of the tree structure. A frequency vector generating step of inputting a corpus, generating a frequency vector by counting the co-occurrence frequencies of the word and the word meaning attribute corresponding to each dimension for an arbitrary word, and storing the frequency vector in a storage device;
A concept vector generation step of generating the concept vector by normalizing the probability according to the tree structure of the word meaning attribute and storing the concept vector in a storage device;
The distance between two concept vectors obtained by applying the frequency vector generation step and the concept vector generation step to any two words is determined according to the tree structure and the probability and child corresponding to the parent node. The degree of association between the two words is calculated by accumulating the local distance calculated based on the probability corresponding to the node from the root node toward the leaf node, and stored in the storage device. A calculation step;
A method for determining the degree of association between words, comprising: In claim 4,
The inter-word relevance calculation step is a step of interrupting the integration when the approximate value of the relevance between two words exceeds a preset value. In claim 4,
The inter-word relevance calculation step is a step of executing the above integration up to the last child node of the tree structure.   A program that causes a computer to function as each means that constitutes the inter-word relevance determination device according to claim 1.   A computer-readable recording medium in which a program that causes a computer to function as each means constituting the inter-word relevance determination device according to claim 1 is recorded.

Images