JP2006215850A - Apparatus and method for creating concept information database, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus and a method for creating a concept information database, which allow users to easily create the concept information database that has eventually enough accuracy, a program and a recording medium therefor. <P>SOLUTION: A provided document set is analyzed, and words that exist in the provided document set are extracted. In addition, word chains that exist in the provided document set are extracted, and the co-occurrence frequency between each of the words and each of the word chains is detected. A level for co-occurrence is detected corresponding to the co-occurrence frequency, and then concept information on the words is quantified based on the detected co-occurrence level to store the acquired concept information on the words into a database. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

The present invention relates to an apparatus and a method for quantifying conceptual information of words or word chains and creating a database, and in particular, concept information of words or word chains appearing in a given document is stored in the given document. The present invention relates to an apparatus and method for creating a conceptual information database for quantification based on the co-occurrence of a word or word chain that appears and the word or word chain.

  2. Description of the Related Art Conventionally, devices and methods for creating word concept information in a database have been proposed for the purpose of determining similarity between words and improving document search accuracy. That is, for the purpose of determining similarity between words, a database creation method has been proposed in which a multidimensional space having words as elements is prepared and each word is arranged as a vector in the multidimensional space (for example, (See Non-Patent Document 1 and Non-Patent Document 2).

  In addition, when quantifying the concept of “word” in a sentence, using “syntactic analysis”, “relationship words” that have a relationship that forms a grammatical pair with “word” are extracted, and “degree of association” between them is extracted. A method of using is proposed (for example, see Patent Document 1).

In the conventional examples described in Non-Patent Document 1 and Non-Patent Document 2, the concept of a certain word is expressed in a multidimensional space including a plurality of co-occurring words. On the other hand, the conventional example described in Patent Document 1 is expressed not only in co-occurrence but also in a multidimensional space composed of a plurality of words that have a grammatical relationship with a certain word (for example, a relationship between a subject and a predicate). The point of doing is very different.
Schuetze, H., “Dimensions of Meaning”, in Proceedings of Supercomputing '92, pp.787-796, 1992 Kasahara, Matsuzawa, Ishikawa, “Difference of everyday words using Japanese dictionary”, Transactions of Information Processing Society of Japan, Vol.38, No.7, pp.1272-1284, 1997 JP-A-9-134360

  However, in the conventional examples described in Non-Patent Document 1 and Non-Patent Document 2, the co-occurrence frequency is calculated without considering the grammatical / semantic relationship. There is a problem that it is not sufficient for use in increasing accuracy.

  Further, in the conventional example described in Patent Document 1, syntax analysis is performed in order to capture a grammatical / semantic relationship, but it is difficult to completely capture a grammatical / semantic relationship with the current technology. There is.

  That is, the conventional example has a problem that it is difficult to obtain a conceptual information database with sufficient accuracy.

As a result, it is an object of the present invention to provide a conceptual information database creation apparatus, a conceptual information database creation method, a program, and a recording medium that can easily obtain a conceptual information database with sufficient accuracy.

  The present invention provides a document analysis means for analyzing a given document set, a word extraction means for extracting a word existing in the given document set and storing it in a storage device, and the given document set. The word chain extraction means for extracting the word chain existing in the storage unit and storing it in the storage device, and detecting the number of times of co-occurrence between each of the words and each of the word chains, and storing the number of co-occurrence in the storage device Detecting means, and detecting the co-occurrence degree according to the number of co-occurrence, quantifying the conceptual information of the word based on the detected co-occurrence degree, and storing the concept information in the storage device A concept information database creation device having concept information database creation means that uses the concept information of the word obtained by the concept information quantification means as a database.

The present invention also provides a document analysis means for analyzing a given document set and a word chain existing in the given document set, or a word chain and a word are extracted and stored. An extraction means stored in the device; a co-occurrence count between each of the word chains and the word chain or each word; and a co-occurrence count detection means stored in the storage device; and a co-occurrence count according to the co-occurrence count. The degree of occurrence is detected, and based on the detected degree of co-occurrence, the concept information of the word chain is quantified and obtained by the concept information quantification means for storing in the storage device and the concept information quantification means. A conceptual information database creation device having concept information database creation means that uses the word chain concept information as a database.

According to the present invention, since co-occurrence with a word chain is used, it is possible to indirectly grasp a grammatical / semantic relationship without performing parsing, and to determine similarity between words or a document search In the case of increasing the accuracy of the data, the concept information database can be created with sufficient accuracy.

  The best mode for carrying out the invention is the following examples.

  FIG. 1 is a block diagram showing a basic configuration of a conceptual information database creation apparatus 10 that is Embodiment 1 of the present invention.

  The concept information database creation apparatus 10 inputs a large amount of document set 20 that is a source for creating a concept information database, outputs a concept information database 30, a document analysis unit 11, a word extraction unit 12, and a word chain extraction unit 13. The co-occurrence number detection unit 14, the concept information quantification unit 15, and the concept information database creation unit 16 are included.

  The document analysis unit 11 performs morphological analysis on all sentences included in the document set 20, divides it into words, and gives parts of speech to each word.

  The word extraction unit 12 extracts words included in the document set 20 and stores them in the storage device.

  The word chain extraction unit 13 extracts word chains included in the document set 20 and stores them in the storage device.

  The co-occurrence count detection unit 14 extracts words or word chains co-occurring with respect to a certain word or word chain based on the analysis result performed by the document analysis unit 11, counts the number of co-occurrence, and stores the storage device To remember.

  The “word chain” is a chain of n words continuous in the document (n is an integer of 2 or more).

  Based on the number of co-occurrence counted by the co-occurrence number detection unit 14, the conceptual information quantification unit 15 calculates the co-occurrence degree of the word or word chain of interest and the word or word chain, Quantify conceptual information for a word or word chain. The quantification will be described later.

  The concept information database creation unit 16 creates a database so that the concept information quantified by the concept information quantification unit 15 can be searched using words or word chains as keys.

  FIG. 2 is a flowchart showing a schematic operation of the conceptual information database creation device 10.

  In S <b> 1, the document analysis unit 11 extracts one document from the document set 20.

  In S2, the document analysis unit 11 extracts one sentence from the document extracted in S1.

  In S3, the document analysis unit 11 performs morphological analysis on the one sentence extracted in S2, divides it into words, divides them into words, and gives parts of speech to each word.

  In S4, it is determined whether or not the entire sentence in the extracted document has been processed. If there is an unprocessed sentence, the process proceeds to S5. If there is no unprocessed sentence (when all sentences have been processed), the process proceeds to S6.

  In S5, the process of S2-S5 is repeated for the next sentence as a processing target.

  In S6, it is determined whether or not all documents included in the document set 20 have been processed. When there is an unprocessed document, the process proceeds to S7. When there is no unprocessed document (when all the documents have been processed), the morphological analysis results of all the documents in the document set 20 are converted into the word extraction unit 12, word The process is sent to the chain extraction unit 13, and the process proceeds to S8.

  In S7, the processing of S1 to S6 is repeated for the next document in the document set 20 as a processing target.

  In S8, the word extraction unit 12 and the word chain extraction unit 13 extract all words or word chains (chains of two or more word chains) from the morphological analysis result, and store them in the storage device.

  In S9, the co-occurrence number detection unit 14 extracts a co-occurring independent word or word chain for each of the extracted independent words (nouns, pronouns, verbs, adjectives, adverbs) or word chains, and the number of appearances And the count result is sent to the conceptual information quantification unit 15.

In the embodiment, when the number of appearances is counted, there are the following three types. That is,
(1) A word (independent word) and a word chain are extracted from the document set 20, and each of the extracted words existing in a predetermined document range and a word existing in the predetermined document range A type that counts the number of co-occurrence with each chain.
(2) A word chain is extracted from the document set 20, and each of the extracted first word chains existing in the predetermined document range and the second word existing in the predetermined document range A type that counts the number of co-occurrence with each chain.
(3) A word chain and a word (independent word) are extracted from the document set 20, and each of the extracted word chains existing in a predetermined document range is present in the predetermined document range. A type that counts the number of co-occurrence with each word.
There is.

  The “predetermined document range” is a document range in which the number of times of co-occurrence is counted, for example, a subset of the given document set, at least one paragraph included in the document, and the one paragraph Is one of at least one sentence included in.

  In S10, based on the result counted by the co-occurrence number detection unit 14, the concept information quantification unit 15 calculates the degree of co-occurrence with each word or word chain for each extracted word or word chain. .

  In S11, based on the calculated co-occurrence degree, the concept information quantification unit 15 quantifies the concept information of each word or word chain, and sends the quantified result to the concept information database creation unit 16. .

  Here, the above-mentioned “concept information” is quantified as a matrix having a word or word chain as a row, a word or word chain examined as a co-occurrence target as a column, and a co-occurrence degree as a value. That is, the concept information of a word or word chain is expressed as a row vector whose element is the co-occurrence degree of the word or word chain.

  In this case, the number of rows and columns of the matrix may be set according to the frequency of words or word chains.

  Further, when there are many cases where the co-occurrence degree is 0, as shown in FIG. 8, the number of word chains extracted from the document set 20 may be selectively reduced. In addition, when there are many cases with co-occurrence degrees of 0, a word chain in which the number of words constituting the word chain is reduced (for example, a two-word chain (word bigram) with respect to a three-word chain or a one-word chain (word unigram). )), And the concept information may be supplemented using the calculated degree of co-occurrence. Further, the number of columns may be reduced by singular value decomposition. Here, the method of complementing the co-occurrence degree is not particularly limited.

  In S12, the concept information database creation unit 16 creates the concept information database 30 so that the concept information of the word or word chain can be searched using the word or word chain as a key. Then, the conceptual information database creation process ends.

  Next, the operation of the conceptual information database creation device 10 will be described using a specific example.

  FIG. 3 is a diagram illustrating an example of the contents of the document set 20 used in the specific example of the embodiment.

  The document set 20 includes n (n is an integer) documents.

  The first document 21, the second document 22,..., The nth document 2n are documents included in the document set 20, and are in the order of the first document 21, the second document 22,. The nth document 2n is the final document included in the document set 20.

[Specific Example 1] (Example of word-to-word trigram)
In specific example 1, a concept information database 30 is created for the document set 20 shown in FIG. The word extracted from the first document 21 is an independent word, and the word chain as a target for examining the number of times that the extracted independent word co-occurs is a three-word chain (word trigram).

  As a specific example, an example is shown in which the document range for counting the number of times of co-occurrence is in the same document. That is, for example, for independent words included in the first document 21, co-occurrence with a three-word chain included only in the first document 21 is counted.

  The document analysis unit 11 extracts the first document 21 from the document set 20 shown in FIG. 3 (S1). Next, the document analysis unit 11 extracts the first sentence from the first document 21 extracted in S1 (S2). The first document 21 is the document shown in FIG. 3, and “we are researching and developing a search system” is extracted as the first sentence. Further, the document analysis unit 11 performs morphological analysis on the first sentence, stores it in the storage device, divides it into words, gives each word an identifier representing the part of speech, and stores it in the storage device (S3). ).

  When the word boundary is indicated by “/”, the result of the morphological analysis is as follows. The symbol [] indicates a part of speech or the like.

1st document: 1st paragraph: 1st sentence: We [pronoun] / ha [adverb] / search system [compound noun] / [case particle] / research [sa variable noun] / development [sa variable noun] / Case particle] / advance [verb] / te [connecting particle] / is [auxiliary verb] /. [symbol]/
Next, it is determined whether or not the entire sentence in the retrieved first document 21 has been processed (S4). Since there is a sentence that has not been processed yet, the next second sentence is “input from PB telephone”. Is adopted as a processing target (S5). Also for the second sentence, similarly to the process in the first sentence, morphological analysis and identifier assignment are performed and stored in the storage device (S3).

  When all the sentences included in the first document 21 have been processed (S4), it is determined whether all the documents included in the document set 20 have been processed (S6). Since the document set 20 is the set shown in FIG. 3, the second document 22 is set as the next processing target (S7).

  The entire text of the second document 22 is also processed (S4), and all the documents of the document set 20 are processed. That is, when the processing up to the nth document 2n is completed (S6), the document analysis unit 2 The morphological analysis result of the document is sent to the word extraction unit 12 and the word chain extraction unit 13 (S7).

  The word extraction unit 12 and the word chain extraction unit 13 extract all words or word chains from the morphological analysis result and store them in the storage device (S8). Here, in the specific example 1, the word to be extracted is an independent word. Therefore, from the morphological analysis result of the first sentence of the first document 21, “we”, “search system”, “research”, “development” , 5 independent words of “advance” are extracted. Similarly, after the second sentence, all independent words are extracted from the morphological analysis results of the whole sentence included in the document set 20.

  Further, the co-occurrence count detection unit 14 extracts a word chain (3-word chain) for each of the extracted independent words, and counts the number of times that the extracted independent word and the 3-word chain co-occur. Then, the counted result is sent to the conceptual information quantification unit 15 (S9).

  FIG. 4 shows an example of the number of times of co-occurrence between “words extracted from the document set 20 (independent words)” and “word chains extracted from the document set 20 (three-word chains)” in Example 1. FIG.

  In FIG. 4, first, for example, the number of co-occurrence of the independent word “we” and the word chain “research system” is counted in the range of the first document 21, and then in the range of the second document 22, Count the number of co-occurrence between the independent word "we" and the word chain "research system" ... Finally, within the nth document 2n, the independent word "we" and the word chain "research system research" The total number of co-occurrence times is counted as 56, as shown in FIG.

  In addition, when counting the number of co-occurrence between the independent word “we” and the word chain “research system research”, the independent word “we” and the word chain “research system research” Count any number of words between, and count the number of words that exist between the independent word "we" and the word chain "Research system research" . Furthermore, even if the word chain “Research System Research” appears before the independent word “we” in the same document, it is counted.

  Similarly, for example, when focusing on the independent word “research”, the number of co-occurrence of this independent word “research” and the word chain “has been” is counted for each document, and the counted number of co-occurrence The total value is 76 times as shown in FIG.

  In specific example 1, since the co-occurring word chain is a three-word chain (word trigram), from the morphological analysis result of the first sentence of the first document 21, “★★ we”, “★ we” "We are a search system", "is a search system", "search system research", "research and development", "research and development", "proceed development", "progress", "progress" , “Is”, “is. Book”, “. Book system” are extracted. Note that the ★ mark included in the first two word chains represents an empty word. The last two three-word chains include “book” and “system” which are words at the beginning of the second sentence.

  In the same manner as described above, all three-word chains (word trigrams) are extracted from all sentences included in the document set 20.

  In FIG. 4, the “word chain” is three words, but may be a two-word chain or a word chain in which four or more words are chained.

  Next, based on the result counted by the co-occurrence number detection unit 14, the concept information quantification unit 15 determines the degree of co-occurrence with the three-word chain (word trigram) for each independent word extracted from the first document 21. Is calculated (S10).

  Here, a normalized value is used as the co-occurrence degree, and the ratio of the number of occurrences of each three-word chain to the number of all occurrences of the three-word chain regarding the independent word of interest is used as the normalized value. .

  For example, in FIG. 4, assuming that the total number of occurrences of the three-word chain relating to the independent word “we” is 1000, the ratio of the number of appearances of the individual three-word chain is the count value for the independent word “we” in FIG. Is a normalized value.

  FIG. 5 is a diagram illustrating an example of co-occurrence degrees of “independent words” and “word chains” in the case of FIG.

  Further, the concept information quantification unit 15 quantifies the concept information of each independent word based on the co-occurrence degree, and then sends the quantified result to the concept information database creation unit 16 (S11).

  Further, in the case shown in FIG. 5, if a self-supported word is a row, a three-word chain is a column, and a co-occurrence is directly used as a value, a matrix is formed, and this matrix quantifies conceptual information. The co-occurrence degree calculation result example shown in FIG. 5 becomes the conceptual information quantification result example as it is.

  Receiving the quantification result of the concept information, the concept information database creation unit 16 creates the concept information database 30 so that the concept information of the independent word can be searched using the independent word as a key, and the conceptual information database creation process is completed. (S12).

  With the above operation, the concept information database 30 in which the concept information of independent words is expressed by the row vector having the co-occurrence degree of the three word chain (word trigram) as an element can be created from the document set 20.

[Specific Example 2] (Example of word trigram vs. word trigram)
Also in the specific example 2, the concept information database 30 is created for the document set 20 shown in FIG. Instead of the independent words in Example 1, a three-word chain (word trigram) is used, and the word chain in Example 1 similarly uses the word chain, counts the number of co-occurrence between the word chains, 3 word chain (word trigram) is used. The document range for counting the number of times of co-occurrence is within the same document (for example, counting only the three-word chain included in the first document 21 with respect to the three-word chain included in the first document 21). .

  Since the operations corresponding to S1 to S7 shown in FIG. 2 are the same as those in the first specific example, the description thereof is omitted.

  Next, the word chain extraction unit 13 extracts all word chains from the morphological analysis result (S8). Here, in specific example 2, since the word chain to be extracted is a three-word chain (word trigram), from the morphological analysis result of the first sentence of the first document 21, “★★ we”, “★ we are ”,“ We are a search system ”,“ Has a search system ”,“ Research system research ”,“ Research and development ”,“ Research and development ”,“ Proceed development ”,“ Proceed ”,“ Proceed ” Thirteen three-word chains (word trigrams) of “Yes”, “Yes.”, “Yes. Book”, “.Book System” are extracted. Note that the ★ marks included in the first two represent empty words. “Book” and “system” included in the last two words in the three-word chain are words in the second sentence.

  Similarly to the above, from the second sentence onward, all three word chains (word trigrams) are extracted from the morphological analysis results of all the sentences included in the document set 20.

  Further, the co-occurrence count detection unit 14 extracts co-occurring word chains for each of the extracted three word chains (word trigram), counts the number of co-occurrence, and quantifies the count result as conceptual information. The data is sent to the unit 15 (S9).

  In specific example 2, since the extracted word chain is a three-word chain (word trigram), the above 13 three-word chains (word trigram) are extracted from the morphological analysis result of the first sentence of the first document 21. The In the same manner, all three word chains (word trigrams) are extracted from all sentences included in the document set 20.

  FIG. 6 is a diagram illustrating an example of the number of co-occurrence of word chains extracted from the document set 20 in the specific example 2.

  Next, based on the result counted by the co-occurrence number detection unit 14, the concept information quantification unit 15 performs co-occurrence of three word chains (word trigrams) in the same document for each of the three word chains (word trigrams). ) Is calculated (S10).

  Here, a normalized value is used as the degree of co-occurrence, and the ratio of the number of occurrences of each three-word chain to the number of all occurrences of the three-word chain regarding the independent word of interest is used as the normalized value. .

  For example, in FIG. 6, the number of all occurrences of the three-word chain relating to the three-word chain (word trigram) “★★ we” (the total number of co-occurrence of the three-word chain “★★ we” with the three-word chain) Is 200, the ratio of the number of occurrences of each co-occurring three-word chain is the value obtained by dividing each of the count values for the three-word chain (word trigram) “★★ we” in FIG. Is the normalized value.

  FIG. 7 is a diagram illustrating an example of the co-occurrence degree between word chains in the specific example 2 shown in FIG.

  Further, the concept information quantification unit 15 quantifies the concept information of each three-word chain (word trigram) based on the co-occurrence degree, and then sends the result to the concept information database creation unit 16 (S11).

  A three-word chain (word trigram) is a row, and a co-occurring three-word chain (word trigram) is a column.

  FIG. 8 is a diagram illustrating an example of the degree of co-occurrence between word chains when the number of one of the two word chains for checking the number of times of co-occurrence is reduced in the specific example 2.

  In the example shown in FIG. 7, instead of using all the word chains extracted from the document set 20, some are selected, that is, as one of the three word chains, “we are a search system”, “is a search system” ”,“ Search system research ”,“ Research and development ”,“ Research and development ”,“ And Web ”, and“ Use of ”conceptual information quantification result examples, It is shown in FIG.

  The number of the one three-word chain may be limited to other than seven.

  In this way, if the one three-word chain is limited to a small number, the calculation becomes easy.

  Here, the one three-word chain may be limited, and the co-occurrence degree may be recalculated based on the number of co-occurrence with the limited three-word chain.

  When the conceptual information database creation unit 16 receives the quantification result of the conceptual information, the conceptual information database 30 can search the conceptual information of the three-word chain (word trigram) using the three-word chain (word trigram) as a key. And the conceptual information database creation process is terminated (S12).

  As a key, not only a three-word chain (word trigram) but also a word included in each three-word chain may be used as a secondary key. That is, a word may be used as a secondary key, and the 3-word chain may be searched from this word, or a self-supporting word may be used as a secondary key. Here, the secondary key is not limited.

  That is, in the specific example 2, the conceptual information quantification means obtains the co-occurrence degree based on the co-occurrence recovery of each of the first word chain and each of the second word chain, and the calculated co-occurrence degree Based on the above, the word chain concept information is quantified and stored in a storage device, and the first word chain is a chain of n words consecutive in the document (n is an integer of 2 or more). The second word chain is a chain of m words continuous in the document (m is an integer of 2 or more).

  Through the above operation, the concept information database 30 in which the concept information of the three word chain (word trigram) is expressed from the document set 20 by the row vector having the co-occurrence degree of the three word chain (word trigram) as an element is created. be able to.

[Specific Example 3]
Concrete example 3 counts the number of co-occurrence of “word chain” extracted from first document 21 and “word” and, based on the counted number of co-occurrence, in the same manner as above, This is an example in which conceptual information is quantified to determine the degree of co-occurrence.

  That is, the specific example 3 is an example in which the number of co-occurrence is obtained by transposing the rows and columns shown in FIG. 4 to obtain the co-occurrence degree.

  According to the above embodiment, a given document is divided into words, grammatical information such as part of speech is added (document analysis), and words or word chains appearing in the document and within a certain document range (the same document In the same paragraph, in the same sentence, etc.), and quantifying the conceptual information of the word or word chain based on the co-occurrence of the word or word chain, Since the concept information of all the words or word chains obtained by this quantification is created as a database, the concept information of the words or word chains that appear in a given document is shared with the words or word chains that co-occur with it. It can be quantified based on the degree of occurrence.

  In other words, the above-described embodiment is an embodiment in which a database is created by combining words and word chains, and exists in the document collection means for analyzing a given document set and the given document set. A word extracting means for extracting a word and storing it in a storage device; a word chain extracting means for extracting a word chain existing in the given document set and storing it in the storage device; Detecting the number of co-occurrences with each of the word chains, the co-occurrence number detecting means for storing in the storage device, and detecting the co-occurrence degree according to the number of co-occurrence, based on the detected co-occurrence degree Concept information quantifying means for quantifying the concept information of the word and storing it in a storage device, and concept information database creating means for using as a database the concept information of the word obtained by the concept information quantifying means. Concept information database It is an example of creating apparatus.

  In this case, the co-occurrence degree is a value obtained by normalizing the number of appearances of the word chain related to the focused word among the words. The normalized value is the ratio of the number of appearances of each word chain to the total number of appearances of the word chain related to the focused word. Further, the normalized value is a ratio of the number of appearances of each word chain to the maximum number of appearances among the number of appearances of the word chain related to the word of interest. The word is an independent word, and the word chain is a chain of n words continuous in the document (n is an integer of 2 or more). The conceptual information quantification means quantifies the conceptual information of the word based on the co-occurrence degree of the word existing in the document range for counting the number of co-occurrence in the document set and the word chain. The document range for counting the number of co-occurrence is included in the subset of the given document set, at least one paragraph included in the document, and the one paragraph One of the at least one sentence.

  Further, the above embodiment is an embodiment in which a database is created by a combination of word chain and word chain or word. Document analysis means for analyzing a given document set, and exists in the given document set. The word chain or the word chain and the word are extracted and stored in the storage device, and the number of co-occurrence of each of the word chain and the word chain or the word is detected. The co-occurrence number detection means for storing in the storage device, the co-occurrence degree is detected according to the co-occurrence number, and the concept information of the word chain is quantified and stored based on the detected co-occurrence degree It is an example of the conceptual information database creation apparatus which has the conceptual information quantification means memorize | stored in an apparatus, and the conceptual information database creation means which uses as a database the conceptual information of the said word chain obtained by the said conceptual information quantification means.

  In this case, the co-occurrence degree is a value obtained by normalizing the number of occurrences of the word chain or the word related to the word chain of interest in the word chain. The normalized value is a ratio of the number of appearances of each word chain or the word to the total number of appearances of the word chain or the word regarding the word chain of interest. Further, the normalized value is a ratio of the number of appearances of the individual word chain or the word to the maximum number of appearances in the word chain or the number of appearances of the word regarding the word chain of interest. . The word is an independent word, and the word chain is a chain of n words continuous in the document (n is an integer of 2 or more). The concept information quantifying means is a common word in the document set. A means for quantifying the conceptual information of the word based on the co-occurrence of the word chain and the word or word chain existing in the document range for counting the number of occurrences, and storing it in a storage device, The document range for counting the number of co-occurrence is one of a subset of the given document set, at least one paragraph included in the document, and at least one sentence included in the one paragraph. It is. Further, the concept information quantification means quantifies the concept information of the word chain based on the co-occurrence degree of each of the first word chain and each of the second word chains, and stores the information in the storage device. The first word chain is a chain of n words continuous in the document (n is an integer of 2 or more), and the second word chain is a chain of m words continuous in the document ( m is an integer of 2 or more.

  Further, the above embodiment can be grasped as an embodiment of the method, a document analysis stage for analyzing a given document set, a word existing in the given document set, and a storage device The word extraction stage stored in the above-mentioned document chain, the word chain extraction stage that extracts the word chain existing in the given document set and stores it in the storage device, and each of the above words and each of the above word chains is shared. Detecting the number of occurrences and storing the number of occurrences in the storage device, detecting the degree of co-occurrence according to the number of times of co-occurrence, and based on the detected degree of co-occurrence, the concept information of the word, An example of a conceptual information database creation method having a conceptual information quantification stage to be quantified and stored in a storage device, and a conceptual information database creation stage in which the conceptual information of the word obtained in the conceptual information quantification stage is used as a database It is.

  Further, the above embodiment can be grasped as another embodiment of the method, and a document analysis stage for analyzing a given document set and a word chain existing in the given document set are extracted. Or a word chain and a word are extracted and stored in a storage device, and the number of co-occurrence of each of the word chains and the word chain or each of the words is detected and stored in the storage device. Detecting the number of occurrences and detecting the degree of co-occurrence according to the number of times of co-occurrence, quantifying the concept information of the word chain based on the detected degree of co-occurrence and storing the concept information in a storage device It is an example of the conceptual information database creation method which has a conversion stage and the conceptual information database preparation stage which uses as a database the conceptual information of the said word chain obtained in the said conceptual information quantification stage.

  And the said Example is an example of the program which makes a computer perform each said step in each of both said conceptual information database preparation methods.

The program may be recorded on a recording medium such as a CD, a DVD, or a semiconductor memory. That is, the above-described embodiment is an example of a computer-readable recording medium that records a program that causes a computer to execute the above-described steps in each of the both concept information database creation methods.

It is a block diagram which shows the basic composition of the conceptual information database creation apparatus 10 which is Example 1 of this invention. 4 is a flowchart showing a schematic operation of the concept information database creation device 10. It is a figure which shows the example of the content of the document set 20 used by the specific example of an Example. FIG. 11 is a diagram illustrating an example of the number of times of co-occurrence between each of “words extracted from document set 20 (independent words)” and “word chain extracted from document set 20 (three-word chain)” in specific example 1; is there. FIG. 5 is a diagram showing an example of co-occurrence degrees of “independent words” and “word chains” in the case shown in FIG. FIG. 10 is a diagram illustrating an example of the number of co-occurrence of word chains extracted from a document set 20 in specific example 2. In the specific example 2, in the case shown in FIG. 6, it is a figure which shows the example of the co-occurrence degree of word chains. In the specific example 2, it is a figure which shows the example of the co-occurrence degree of word chains when the number of one word chain of two word chains which investigates the number of times of co-occurrence is reduced.

Explanation of symbols

10 ... Concept information database creation device,
11 ... Document analysis section,
12 ... Word extraction unit,
13 ... word chain extraction part,
14 ... Co-occurrence detection unit,
15 ... Concept information quantification part,
16 ... Concept information database creation part,
20 ... Document set,
30 ... Conceptual information database,
21 ... first document,
22 ... the second document,
2n: nth document (final document).

Claims (17)

A document analysis means for analyzing a given document set;
Word extraction means for extracting a word existing in the given document set and storing it in a storage device;
Word chain extraction means for extracting a word chain existing in the given document set and storing it in a storage device;
A co-occurrence number detecting means for detecting the co-occurrence number of each of the words and each of the word chains and storing it in a storage device;
A concept information quantification unit that detects a co-occurrence degree according to the number of times of co-occurrence, quantifies the concept information of the word based on the detected co-occurrence degree, and stores it in a storage device;
A conceptual information database creating means that uses the conceptual information of the word obtained by the conceptual information quantifying means as a database;
A conceptual information database creation device characterized by comprising: In claim 1,
The said co-occurrence degree is the value which normalized each frequency | count of appearance of the said word chain regarding the focused word among the said words, The conceptual information database creation apparatus characterized by the above-mentioned. In claim 2,
The said normalized value is a ratio of the frequency | count of appearance of each said word chain with respect to the sum total of the frequency | count of said word chain regarding the said attention word, The conceptual information database creation apparatus characterized by the above-mentioned. In claim 2,
The normalized value is a ratio of the number of appearances of each word chain to the maximum number of appearances among the number of appearances of the word chain related to the focused word apparatus. In any one of Claims 1-4,
The above words are independent words,
The word chain is a chain of n words continuous in a document (n is an integer of 2 or more). In claim 1,
The conceptual information quantifying means quantifies the conceptual information of the word based on the co-occurrence degree of the word existing in the document range for counting the number of co-occurrence in the document set and the word chain, Means for storing in a storage device;
The document range for counting the number of times of co-occurrence is one of a subset of the given document set, at least one paragraph included in the document, and at least one sentence included in the one paragraph. A conceptual information database creation device characterized by A document analysis means for analyzing a given document set;
Extraction means for extracting a word chain existing in the given document set or extracting a word chain and a word and storing them in a storage device;
A co-occurrence number detection means for detecting the number of co-occurrence of each of the word chains and the word chain or each of the words and storing it in a storage device;
A concept information quantification means for detecting a co-occurrence degree according to the number of times of co-occurrence, quantifying the concept information of the word chain based on the detected co-occurrence degree, and storing it in a storage device;
A conceptual information database creating means that uses the conceptual information of the word chain obtained by the conceptual information quantifying means as a database;
A conceptual information database creation device characterized by comprising: In claim 7,
The co-occurrence degree is a value obtained by normalizing the number of occurrences of the word chain or the word related to the word chain of interest in the word chain. In claim 8,
The normalized value is a ratio of the number of occurrences of each word chain or the word to the total number of occurrences of the word chain or the word relating to the word chain of interest. Creation device. In claim 8,
The normalized value is a ratio of the number of appearances of the individual word chain or the word to the maximum number of appearances in the word chain or the number of appearances of the word regarding the word chain of interest. Feature information database creation device. In any one of Claims 7-10,
The above words are independent words,
The word chain is a chain of n words continuous in a document (n is an integer of 2 or more). In claim 7,
The conceptual information quantifying means calculates the conceptual information of the word based on the co-occurrence degree of the word chain and the word chain existing in the document range counting the number of times of co-occurrence in the document set. Means for quantifying and storing in a storage device;
The document range for counting the number of times of co-occurrence is one of a subset of the given document set, at least one paragraph included in the document, and at least one sentence included in the one paragraph. A conceptual information database creation device characterized by In any one of Claims 7-10,
The concept information quantifying means is a means for quantifying the concept information of the word chain based on the co-occurrence degree of each of the first word chain and each of the second word chain, and storing it in a storage device. Yes,
The first word chain is a chain of n words consecutive in the document (n is an integer of 2 or more),
The second word chain is a chain of m words continuous in a document (m is an integer of 2 or more). A document analysis stage for analyzing a given set of documents;
A word extraction step of extracting words stored in the given document set and storing them in a storage device;
A word chain extraction step of extracting word chains existing in the given document set and storing them in a storage device;
Detecting the number of co-occurrence between each of the words and each of the word chains and storing it in a storage device;
A co-occurrence degree is detected according to the co-occurrence number, and based on the detected co-occurrence degree, the concept information of the word is quantified and stored in a storage device;
A conceptual information database creation stage in which the conceptual information of the word obtained in the conceptual information quantification stage is used as a database;
A concept information database creation method characterized by comprising: A document analysis stage for analyzing a given set of documents;
Extracting word chains present in the given document set, or extracting word chains and words and storing them in a storage device;
Detecting the number of co-occurrence between each of the word chains and the word chain or each of the words, and storing the number of co-occurrence in a storage device;
A co-occurrence degree is detected in accordance with the number of co-occurrence times, and based on the detected co-occurrence degree, concept information of the word chain is quantified and stored in a storage device;
A concept information database creation stage using the word chain concept information obtained in the concept information quantification stage as a database;
A concept information database creation method characterized by comprising:   The program which makes a computer perform each said step in the database preparation method of Claim 14 or Claim 15.   The computer-readable recording medium which recorded the program which makes a computer perform the said each step in the database preparation method of Claim 14 or Claim 15.

Images