JP2008233963A - Inter-word correlation degree calculation device and method, program and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To calculate a correlation degree showing relation between words appearing in a document related to a specific topic in a short time. <P>SOLUTION: A specific document set 21 related to the specific topic and a correlation DB 22 storing a general correlation degree showing the relation between the target words previously calculated from a general document set having no specific topic are stored by a storage part 2, each document of a specific document set 21 is read from the storage part 2 by a specific correlation calculation part 11, a specific correlation degree between the target words is calculated based on appearance frequency of the target word in the documents, the general correlation degree between the target words is retrieved from the correlation DB 22 of the storage part 2 by a correlation integration part 12, and the correlation degree between the target words in the whole document set comprising the specific document set and the general document set is calculated based on the general correlation degree and the specific correlation degree obtained in the specific correlation calculation part 11. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a natural language analysis technique, and more particularly to an inter-word correlation calculation technique for calculating a correlation degree between words appearing in a document related to a specific topic.

  When investigating the relationship between words appearing in a document related to a specific topic, a corpus of document sets related to the topic is collected and vectorized by a technique called TF-IDF (Term Frequency-Inverted Document Frequency), for example. A method of taking out the relationship between words using can be considered. The TF-IDF method is one of known methods for calculating the importance of an arbitrary word (see, for example, Non-Patent Document 1). Alternatively, the method according to Patent Document 1 can be used.

JP 2004-005337 A "Keyword extraction with morphological analysis and search API and TF-IDF", http://chalow.net/2005-10-12-1.html Japanese vocabulary system, Iwanami Shoten, 1997

  When examining the relationship between words, it is difficult to collect a large amount of documents focused on a specific topic, so a method of substituting with a small corpus can be considered. However, in such a conventional technique, when a small corpus is used, the number of words is limited during vectorization by TF-IDF, and the vectors become sparse. As a result, the relationship between words is correlated. There was a problem that it could not be reflected sufficiently as information.

  On the other hand, a method in which a target small-scale corpus is embedded in a large-scale corpus is also considered. However, in the prior art, every time the relationship between words is examined, it is necessary to recalculate the entire corpus, so that recalculation takes a lot of time. Therefore, for example, as a service on the web, there is a problem that it takes too much response time to examine and provide a relationship between words in response to a request from a user. Further, the vector generated from the corpus reflects the co-occurrence information of the word, and there is a problem that the semantic information of the word is not reflected.

  The present invention is for solving such a problem, and an inter-word correlation degree calculation device and method, and a program capable of calculating a correlation degree indicating a relation between words appearing in a document related to a specific topic in a short time An object of the present invention is to provide a recording medium.

  In order to achieve such an object, the inter-word correlation degree calculating apparatus according to the present invention is an inter-word correlation degree calculating apparatus that calculates a correlation degree indicating a relationship between target words included in a document set, A storage unit that stores a specific document set related to a specific topic, a correlation database that accumulates general correlations indicating relationships between target words calculated in advance from a general document set in which no topic is specified, and a specific document from the storage unit Read each document of the set, and search for the general correlation between the target words from the correlation database of the storage unit and the specific correlation calculator that calculates the specific correlation between the target words based on the appearance frequency of the target words in these documents , Based on the general correlation level and the specific correlation level obtained by the specific correlation calculation unit, And a correlation integration section for calculating a correlation degree between.

At this time, the general document set may be composed of a dictionary consisting of a set of headwords and their meaning sentences, or a large-scale corpus, and is generated by a recursive expansion method (for example, see Patent Document 1) as a general correlation. Vector may be used.
Further, the correlation integration unit may use the probability of recalling the other target word from the co-occurrence information or semantic information of one target word as the degree of correlation between the target words.
Moreover, you may further provide the correlation DB update function which updates a correlation database with the correlation degree calculated in the correlation integration part.

  A word correlation calculation method according to the present invention is a word correlation calculation method for calculating a correlation indicating a relationship between target words included in a document set, and relates to a specific topic by a storage unit. A storage step for storing a specific document set and a correlation database for storing a general correlation indicating a relationship between target words calculated in advance from a general document set in which a topic is not specified; A specific correlation calculation step of reading each document of the specific document set and calculating a specific correlation degree between the target words based on the appearance frequency of the target words in these documents, and a correlation integration unit between the target words from the correlation database of the storage unit The general correlation is searched for, and based on the general correlation and the specific correlation obtained by the specific correlation calculation unit, the specific document set and the general sentence And a correlation integration calculating the correlation between the target word in the entire document set consisting of a set.

Moreover, the program concerning this invention is a program for making a computer perform each step of the said correlation degree calculation method between words.
A recording medium according to the present invention is a recording medium on which the program is recorded.

  According to the present invention, the specific document set and the correlation DB are stored in the storage unit, and the specific correlation between the target words is determined by the specific correlation calculation unit based on the appearance frequency of the target word in each document of the specific document set. The degree of correlation is calculated, the correlation integration unit searches the correlation DB for the general correlation degree between the target words, and based on the general correlation degree and the specific correlation degree obtained by the specific correlation calculation unit, the specific document set and the general document Since the degree of correlation between the target words in the entire document set consisting of the set is calculated, the general degree of correlation can be obtained from the correlation DB, so that the processing required for the calculation is omitted compared to the case of calculating the general degree of correlation. Thus, the degree of correlation indicating the relationship between words appearing in a document related to a specific topic can be calculated in a short time.

Next, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
First, an inter-word correlation degree calculation apparatus according to a first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing the configuration of the inter-word correlation degree calculation apparatus according to the first embodiment of the present invention.
The inter-word correlation degree calculation device 10 includes a general information processing apparatus such as a server or a personal computer, and has a function of calculating a correlation degree indicating a relationship between target words included in a document set.

  In the present embodiment, the storage unit stores a specific document set related to a specific topic and a correlation database that accumulates a general correlation indicating a relationship between target words calculated in advance from a general document set in which no topic is specified. The specific correlation calculation unit reads out each document of the specific document set from the storage unit, calculates the specific correlation degree between the target words based on the appearance frequency of the target words in these documents, and stores it in the correlation integration unit. The general correlation for the target words is searched from the correlation database of the part, and the target in the entire document set including the specific document set and the general document set based on the general correlation and the specific correlation obtained by the specific correlation calculation unit The degree of correlation between words is calculated.

Hereinafter, the configuration of the inter-word correlation degree calculation apparatus according to the first embodiment of the present invention will be described in detail with reference to FIG.
The inter-word correlation degree calculation device 10 has, as main functional units, an arithmetic processing unit 1, a storage unit 2, an input / output interface unit (hereinafter referred to as an input / output I / F unit), as in a general information processing device. 3, a communication interface unit (hereinafter referred to as a communication I / F unit) 4, an operation input unit 5, and a screen display unit 6 are provided.

The arithmetic processing unit 1 is composed of a microprocessor such as a CPU and its peripheral circuits, and reads and executes the program 20 stored in the storage unit 2, thereby causing the hardware and the program 20 to cooperate with each other to perform various processes. Realize the part.
The main processing units realized by the arithmetic processing unit 1 include a specific correlation calculation unit 11 and a correlation integration unit 12.

The storage unit 2 includes a storage device such as a hard disk or a memory, and stores a program 20 executed by the arithmetic processing unit 1 and various processing information used for correlation degree calculation processing. The program 20 is read from the recording medium M via, for example, the input / output I / F unit 3 or read from an external device (not shown) via the communication I / F unit 4 and stored in the storage unit 2 in advance. The
Main processing information stored in the storage unit 2 includes a specific document set 21 and a correlation database (hereinafter referred to as a correlation DB) 22.

The input / output I / F unit 3 is composed of a dedicated data input / output circuit, and is connected to a recording medium M such as a CD, a DVD, or a nonvolatile memory card in accordance with an instruction from the arithmetic processing unit 1. It has a function of inputting / outputting various data and programs such as word W, correlation result information Y, dictionary, and database.
The communication I / F unit 4 is composed of a dedicated data communication circuit and communicates with an external device such as a server connected via a communication line such as a LAN according to an instruction from the arithmetic processing unit 1. It has a function of transmitting and receiving various data and programs such as W, correlation result information Y, dictionary, and database.

The operation input unit 5 includes an operation input device such as a keyboard and a mouse, and has a function of detecting an operation of the operator and outputting the operation to the arithmetic processing unit 1.
The screen display unit 6 includes a screen display device such as an LCD or a PDP, and has a function of displaying various data such as the target word W and correlation result information Y and an operation screen in accordance with an instruction from the arithmetic processing unit 1. ing.

  FIG. 2 is a block diagram showing a main part of the inter-word correlation degree calculating apparatus according to the first embodiment of the present invention. The specific correlation calculation unit 11 receives a target word W composed of natural language data from the storage unit 2, the input / output I / F unit 3, the communication I / F unit 4, the operation input unit 5, and the like. It has a function of reading each document in the specific document set 21 and a function of calculating a specific correlation between target words based on the appearance frequency of the target word W in these documents. The specific document set 21 is a document set made up of a plurality of documents in which contents related to a specific topic are described, and these documents include target words for which the degree of correlation is calculated. As a specific example of the specific document set 21, a specific corpus related to a specific topic extracted from a large-scale corpus may be used.

  The correlation integration unit 12 is specified based on the function of searching for the general correlation between the target words W from the correlation DB 22 of the storage unit 2, and the general correlation and the specific correlation obtained by the specific correlation calculation unit 11. A function for calculating the correlation between target words in the entire document set including the document set 21 and the general document set, and the obtained correlation as well as the target word as correlation result information Y, the storage unit 2, input / output I / O It has the function to output to F part 3, communication I / F part 4, screen display part 6, etc.

The correlation DB 22 is a database that accumulates a general correlation degree indicating a relationship between target words calculated in advance from a general document set in which a topic is not specified. About this general correlation degree, what is necessary is just to calculate beforehand by well-known methods, such as patent document 1. FIG.
As a specific example of the general document set, a large-scale corpus that is not biased may be used, or an Internet dictionary such as a national language dictionary, a technical dictionary, or Wikipedia may be used. As specific examples of the general correlation, word similarity, word co-occurrence frequency, or a probabilistic measure between words can be used.

  Next, referring to FIG. 3 and FIG. 4, two correlation degree calculation methods having different document set configurations will be compared. FIG. 3 is a schematic flow showing an example of correlation degree calculation using one document set. FIG. 4 is a schematic flow showing an example of correlation degree calculation using two document sets, a specific document set and a general document set. Here, a case where the degree of correlation between the designated target word and a specific word included in the document set is calculated will be described as an example.

  In the correlation degree calculation example of FIG. 3, one large-scale document set including both a document related to a specific topic and a general document with no topic bias is prepared in advance, and the target word for which correlation is to be calculated is input. In response (step 100), first, a document including the target word is searched from the document set (step 101). Next, specific words that appear in common in each searched document are extracted (step 102), so that words that appear in common in each searched document and are less likely to appear in other documents have a higher correlation. The degree of correlation between the target word and the specific word is calculated (step 103).

  On the other hand, in the correlation degree calculation example of FIG. 4, two small document sets including a document related to a specific topic and a general document set including a general document with no biased topics are prepared separately. In response to the input of the target word whose correlation is to be calculated (step 110), first, a document including the target word is searched from the specific document set 21 related to the specific topic (step 111). Here, a dictionary is used to search for a headword including the target word in the word meaning sentence (step 112). Next, a specific word that appears in common in each searched document and each meaning sentence is extracted (step 113), and a word that appears in common in the searched document and meaning sentence and is difficult to appear in other documents and meaning sentences. The degree of correlation between the target word and the specific word is calculated so that the correlation is higher (step 114).

Here, the correlation between words calculated based on a general document in the document set has a constant appearance frequency of words in these documents. For this reason, since it is always unchanged regardless of the topic of the designated target word, once it is calculated, there is no need for recalculation.
The present invention pays attention to such points, calculates the degree of correlation between words calculated based on a general document in advance and stores it in the correlation DB 22, and the object specified by the correlation integration unit 12. When calculating the word correlation, the general correlation between target words calculated based on a general document is retrieved from the correlation DB 22 and integrated with the specific correlation calculated by the specific correlation calculation unit 11. The degree of correlation between target words in the entire document set including the specific document set 21 and the general document set is calculated.

  As a result, when calculating the degree of correlation for the specified target word, only the calculation related to the specific correlation based on the specific document set and the calculation related to the integration of the specific correlation and the general correlation are necessary. Thus, the calculation regarding the general correlation can be omitted. In particular, the general document set that is the basis for the general correlation needs to use a large amount of documents that are not biased in terms of topics, and therefore requires a huge amount of calculation processing compared to the calculation of a specific correlation with a relatively small number of documents. It becomes. According to the present invention, it is possible to omit the calculation related to the general correlation that requires such a huge amount of calculation processing, and the calculation processing load can be greatly reduced.

[Operation of First Embodiment]
Next, the operation of the inter-word correlation degree calculation apparatus according to the first embodiment of the present invention will be described with reference to FIG. Here, a case where the degree of correlation between the designated target word W and a specific word highly related will be described as an example. In starting the correlation calculation, the correlation DB 22 that accumulates the general correlation indicating the relationship between the target document calculated in advance from the specific document set 21 related to a specific topic and the general document set in which the topic is not specified, It is assumed that it is prepared in advance.

The arithmetic processing unit 1 of the inter-word correlation degree calculation device 10 uses the operation input unit 5 to detect the start operation of the correlation degree calculation process by the operator. W is received, and a specific correlation degree calculation process for the target word W is performed.
First, the specific correlation calculation unit 11 reads each document of the specific document set 21 from the storage unit 2, searches for a document including the target word W, and among a plurality of words having a high appearance frequency among the words included in the searched documents. Search for a word as a specific word.

  Next, the specific correlation calculation unit 11 calculates the specific correlation between the target word W and the sentence in the specific document set 21. Here, when the probabilistic method is applied, the degree of correlation between words can be calculated as follows. The specific word that can be recalled from the designated target word Wa is set as Wb, the sentence included in each document constituting the specific document set 21 is set as Cj, and the probability that the sentence Cj including Wa in the specific document set 21 appears is P (Cj | Wa), the probability that Wb is selected as a specific word as a calculation target is P (Wb), and the probability that a sentence Cj is selected from the specific document set 21 is P (Cj). A probability P (Wb | Cj) for recalling Wb from given Cj is obtained by Expression (1).

  Next, the arithmetic processing unit 1 searches the correlation DB 22 of the storage unit 2 for a general correlation degree between the target word W and an arbitrary word by the correlation integration unit 12. When the word in the document of the general document set is Di and the probability that the word Di appears in the document in the general document set including the target word Wa is P (Di | Wa), the general correlation, that is, Di is given. The probability P (Wb | Dj) of recalling the specific word Wb at the time has a relationship represented by the equation (2).

After calculating the specific correlation and the general correlation for the target word W in this way, the correlation integration unit 12 integrates the specific correlation and the general correlation as follows, and the specific document set 21 and the general document set The degree of correlation P (Wb | Wa) between the target word Wa and the specific word Wb in the entire document set consisting of is calculated and output as correlation result information Y.
P (Wb | Wa) can be calculated by summing, for each Di, Cj, the probability P (Wb, Di, Cj | Wa) that Wb, Di, Cj is selected when Wa is given, It can be expanded as shown in Equation (3).

  Here, if the prior probabilities of the word Wx are equal, P (Wb | Wa) can be expressed by the following equation (4) using the specific correlation P (Wb | Cj) and the general correlation P (Wb | Di). Can be expressed as:

Therefore, the correlation integration unit 12 substitutes the specific correlation degree P (Wb | Cj) and the general correlation degree P (Wb | Di) into the equation (4), thereby calculating the correlation degree between the target word Wa and the specific word Wb. Should be calculated.
As a correlation calculation method, other known methods such as a method using an angle between vectors or an inner product, a method of simply summing up as a correlation integration method, or a method of calculating a product may be applied.

[Effect of the first embodiment]
As described above, in the present embodiment, the storage unit 2 accumulates the general correlation indicating the relationship between the target word calculated in advance from the specific document set 21 related to a specific topic and the general document set in which the topic is not specified. The correlation DB 22 is stored, and the specific correlation calculation unit 11 reads each document of the specific document set 21 from the storage unit 2 and calculates a specific correlation between the target words based on the appearance frequency of the target words in these documents. Then, the correlation integration unit 12 searches the correlation DB 22 of the storage unit 2 for the general correlation degree between the target words, and the specific document set based on the general correlation degree and the specific correlation degree obtained by the specific correlation calculation unit 11 In addition, the degree of correlation between the target words in the entire document set including the general document set is calculated.

Therefore, since the general correlation can be obtained from the correlation DB, the processing required for the calculation can be omitted compared with the case of calculating the general correlation, and the relationship between words appearing in a document related to a specific topic can be reduced. The degree of correlation shown can be calculated in a short time.
In addition, the degree of correlation between words can be calculated via a word that is not described in a document set related to a specific topic. This is clear from the equation (4). As a result, it is possible to perform a calculation in consideration of a wider correlation than to calculate the document set alone.

  Moreover, when each general correlation degree of correlation DB is calculated using a dictionary, there exists the characteristic that a word vector is comprised by semantic information by the property of a dictionary. FIG. 5 is a graph showing a comparison result between a word vector related to a broader word and a Japanese vocabulary system. FIG. 6 is a graph showing a comparison result between a word vector related to a synonym and a Japanese vocabulary system. Here, for 100 words obtained in the descending order of an arbitrary target word and word vector element, description is made in a Japanese vocabulary system (see, for example, Non-Patent Document 2), which is a dictionary that describes the semantic relationship between words. The correct / incorrect answer is determined according to the presence / absence, and these determination results are statistically processed for a plurality of target words.

  These FIG. 5 and FIG. 6 show that the higher the word vector element rank, the higher the correct answer rate and the higher the probability of being a broader word or a synonym. In other words, it can be said that the method of calculating the unit vector is a method of directly vectorizing semantic distances such as broader terms and synonyms between words. This is clearly different from a vector generated from co-occurrence information such as TF-IDF. Therefore, by using the vector generated in this way for the calculation of correlation information, a semantic distance can be directly reflected on the degree of correlation between words.

  In addition, when a large-scale corpus is used instead of a dictionary for a general document set, calculation based on semantic information is not included, but even in that case, calculation that considers a wider correlation is possible than calculation using a single document set. Become. In general, even if there are two or more corpora, if the languages are the same, it is a normal usage method to collect them as one corpus and treat them as one large-scale corpus. Therefore, as in the present invention, there is no point in the prior art that a document set that can be aggregated into one is treated as two specific document sets and a general document set independently of each other. It is.

  By dividing the document set into two, once the general correlation in the general document set is calculated, there is no need to re-calculate, the calculation time is shortened accordingly, and various topics are targeted. Even if it changes, the answer can be obtained in a short response time. This is a very important factor for services that require a short response time, such as web services.

[Second Embodiment]
Next, an inter-word correlation degree calculation apparatus according to the second embodiment of the present invention will be described.
In the first embodiment, the case where a large-scale corpus or Internet dictionary without topic bias is used alone as the general document set that is the basis of the correlation DB 22 has been described as an example. However, as in the present embodiment, these May be used in combination.
For example, when using a large-scale corpus, words biased to topics are interpolated, and when using a dictionary, it is combined with a corpus. As a result, co-occurrence information and semantic information can be used simultaneously, and specialized words that are biased to the topic of the corpus can be interpolated by general words in the dictionary. The degree can be calculated.

  FIG. 7 is an example of a calculation result of the degree of correlation between target words. Here, a correlation DB 22 that accumulates general correlations calculated using a corpus composed of a plurality of articles that have asked for opinions regarding the target word “tire” as a specific document set 21 and a general Japanese dictionary as a general document set is used. When used, each correlation degree calculated from the equation (4) shows the top 30 words having the highest correlation degree for each specific word. These correlations were also calculated for the Japanese dictionary only and the corpus only.

  When the corpus and the Japanese dictionary are combined, it is narrowed down to automobile-related words, and it is shown that the degree of correlation can be calculated with higher accuracy than when using the Japanese dictionary and the corpus alone. Yes. Conversely, the article in the corpus is created by people who are interested in automobiles, and the features of the corpus domain (collection of article producers) can be extracted. When a general large-scale corpus is used as the general document set instead of a dictionary, the same calculation can be performed by regarding Di in the above calculation formula as the i-th document in the large-scale corpus.

[Third Embodiment]
Next, with reference to FIG. 8, a word correlation calculation apparatus according to a third embodiment of the present invention will be described. FIG. 8 is a block diagram showing an inter-word correlation degree calculating apparatus according to the third embodiment of the present invention, in which the same or equivalent parts as those in FIG.
In the first embodiment, the case where the correlation DB 22 is prepared in advance as in the specific document set 21 has been described. In the present embodiment, a case where the correlation DB 22 is calculated in advance by the inter-word correlation degree calculation device 10 will be described.

  Compared with the first embodiment, the inter-word correlation degree calculation apparatus 10 according to the present embodiment is provided with a general correlation calculation unit 13 in the arithmetic processing unit 1, and the storage unit 2 includes a general correlation calculation unit 13. A document set 23 is stored in advance. Other configurations are the same as those in the first embodiment, and a description thereof is omitted here.

  The general correlation calculation unit 13 has a function of calculating a general correlation between words from the general document set 23 of the storage unit 2 using the above-described formula (2). The general document set 23 is composed of a plurality of documents whose topics are not specified. As a specific example, a large-scale corpus with no biased topics may be used, and an Internet dictionary such as a national language dictionary, technical dictionary, or Wikipedia may be used. Use it.

  When the operation input unit 5 detects the start operation of the correlation DB creation process by the operator using the operation input unit 5, the arithmetic processing unit 1 of the inter-word correlation degree calculation device 10 uses the general correlation calculation unit 13 to search from the general document set 23 in the storage unit 2. Each document is read, and a general correlation calculation process is performed on the words included in these documents using Equation (2). The general correlation obtained in this way is accumulated in the correlation DB 22 as a pair with the word pair and stored in the storage unit 2.

  In the present embodiment, since the general correlation calculation unit 13 that calculates the general correlation between the words from the general document set 23 of the storage unit 2 is provided, the inter-word correlation calculation device 10 for any general document set 23 is provided. Thus, a desired correlation DB 22 can be created.

[Extended embodiment]
In each of the above embodiments, an example is given in which a specific word having a high co-occurrence frequency and high relevance with a specified target word is extracted from the document set, and the degree of correlation between the target word and these specific words is calculated. However, the specific word is not limited to this. For example, the target word and one or more specific words may be specified, and the degree of correlation between the target word and the specific word may be calculated.

  In each embodiment, a correlation DB update unit is additionally provided in the arithmetic processing unit 1, and the correlation DB update unit associates the correlation DB 22 with the target word pair with the degree of correlation calculated by the correlation integration unit 12. You may make it update. As a result, the calculated new correlation degree is updated and registered in the correlation DB 22 so that the correlation DB 22 can be learned, and in the subsequent calculation of the correlation degree, improvement in the calculation accuracy of the correlation degree can be expected.

It is a block diagram which shows the structure of the correlation calculation apparatus between words concerning the 1st Embodiment of this invention. It is a block diagram which shows the principal part of the correlation calculation apparatus between words concerning the 1st Embodiment of this invention. It is a general | schematic flow which shows the example of correlation calculation using one document set. It is a schematic flow showing an example of correlation degree calculation using two document sets, a specific document set and a general document set. It is a graph which shows the comparison result of the word vector regarding a broad word, and a Japanese vocabulary system. It is a graph which shows the comparison result of the word vector regarding a synonym, and a Japanese vocabulary system. It is an example of the calculation result of the correlation between target words. It is a block diagram which shows the correlation calculation apparatus between words concerning the 2nd Embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Inter-word correlation calculation apparatus, 1 ... Operation processing part, 11 ... Specific correlation calculation part, 12 ... Correlation integration part, 13 ... General correlation calculation part, 2 ... Memory | storage part, 20 ... Program, 21 ... Specific document collection, 22 ... correlation DB, 23 ... general document set, 3 ... input / output I / F unit, 4 ... communication I / F unit, 5 ... operation input unit, 6 ... screen display unit, W ... target word, Y ... correlation result information , M: Recording medium.

Claims (8)

An inter-word correlation degree calculation device for calculating a correlation degree indicating a relationship between target words included in a document set,
A storage unit that stores a specific document set related to a specific topic, and a correlation database that stores a general correlation indicating a relationship between the target words calculated in advance from a general document set in which no topic is specified;
A specific correlation calculation unit that reads each document of the specific document set from the storage unit and calculates a specific correlation between the target words based on the appearance frequency of the target words in these documents;
The general correlation degree between the target words is searched from the correlation database of the storage unit, and the specific document set and the general document set are searched based on the general correlation degree and the specific correlation degree obtained by the specific correlation calculation unit. A correlation integration unit that calculates a correlation between the target words in the entire document set. In the inter-word correlation degree calculation apparatus according to claim 1,
The inter-word correlation calculation apparatus according to claim 1, wherein the general document set is composed of a dictionary including a set of headwords and their meaning sentences, or a large-scale corpus. In the inter-word correlation degree calculation device according to claim 2,
As the general correlation, a vector generated by a recursive expansion method is used. In the inter-word correlation degree calculation device according to claim 3,
The correlation integration unit uses the probability of recalling the other target word from the co-occurrence information or semantic information of one target word as the correlation between the target words. In the inter-word correlation degree calculation apparatus according to claim 1,
The inter-word correlation degree calculation apparatus further comprising a correlation DB update function for updating the correlation database with the correlation degree calculated by the correlation integration unit. A correlation calculation method between words that calculates a correlation indicating the relationship between target words included in a document set,
A storage step of storing, by the storage unit, a specific document set relating to a specific topic and a correlation database storing a general correlation indicating a relationship between the target words calculated in advance from a general document set in which no topic is specified;
A specific correlation calculation step of reading each document of the specific document set from the storage unit by the specific correlation calculation unit, and calculating a specific correlation degree between the target words based on the appearance frequency of the target words in these documents;
A correlation integration unit searches the correlation database of the storage unit for a general correlation degree between the target words, and based on the general correlation degree and the specific correlation degree obtained by the specific correlation calculation unit, the specific document set and A correlation degree integration step for calculating a correlation degree between the target words in the entire document set including the general document set.   The program for making a computer perform each step of the correlation calculation method between words of Claim 5.   A recording medium on which the program according to claim 7 is recorded.

Images