JP2003167894A - Automatic related word extracting method, automatic related word extracting device, plural-important word extracting program and important word vertical hierarchical relationship extracting program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an automatic related word extracting method and an automatic related word extracting device capable of automatically extracting a technical term, a new word, and a buzz term unrecorded in a general existing thesaurus dictionary, and appearing in a specific field designated by a user, and capable of highly accurately and precisely extracting an important word deeply related to a word designated by the user. <P>SOLUTION: A document group of a field designated by the user is stored in a database part 1. The important word being a high significant word is selected from the document group in the database part 1 by an important word analyzing part 2. A count list is made as statistical information to the important word or a pair of important words by a count part 3. A degree of association of the mutual important words is determined by a related word extracting part 4 on the basis of this count list. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention generally relates to a related word automatic extraction method for automatically extracting a word closely related to a word designated by a user based on statistical information of words contained in a database. The present invention relates to a related word automatic extraction method and a related word automatic extraction device capable of extracting a technical term that appears in a specific field designated by a user, a new word, and a buzzword that are not described in an existing thesaurus dictionary.

[0002]

2. Description of the Related Art A conventional related word automatic extraction device has an existing thesaurus dictionary as an internal component thereof, and simply retrieves a word designated by a user from the thesaurus dictionary and outputs the result as a related word extraction result. Generally, it is only displayed.

[0003]

However, in the related-art automatic word extraction device of the related art, technical terms, new words, and buzzwords that are not described in the existing thesaurus dictionary cannot be extracted regardless of their importance. There was a flaw.

Further, when related words for a plurality of fields are required, it is necessary to prepare a thesaurus dictionary for each field individually, which is wasteful in terms of cost.

Further, even in the method of automatically extracting the related words from the statistical information of the database without using the existing thesaurus dictionary, the conventional related word automatic extraction method uses, for example, only the frequency of appearance of words that appear independently. Is common.

Therefore, even if a document database including technical terms, new words, and buzzwords is used, the extraction accuracy of the related word extraction method is defective, and it is difficult to extract an accurate related word desired by the user. there were.

The present invention has been made to solve the above-mentioned problems of the prior art, and its purpose is to provide a specific field specified by the user, which is not described in a general existing thesaurus dictionary. A related word automatic extraction method that can automatically extract technical terms that appear in, new words and buzzwords, and can accurately and accurately extract important words that are deeply related to the words specified by the user. An object is to provide a related word automatic extraction device.

[0008]

In order to achieve the above object, the invention according to claim 1 uses a group of documents in a field designated by a user as a database, and the words of high importance from the documents in the database. It is characterized by using the related word automatic extraction method of selecting the important word which is and calculating the degree of relevance between the important words by using the statistical information for the important word or the pair of important words. Here, the degree of importance refers to the feature of the content indicated by the document, or the degree to which the feature is well represented in the genre of the document.

According to a second aspect of the invention, in addition to the structure of the first aspect, when a group of documents in a plurality of fields is stored in the database, related words for each field can be automatically extracted. It is characterized by having done.

According to a third aspect of the present invention, in addition to the configuration of the first or second aspect, the database can be updated / added at any time, and the differential data is sequentially reflected when the related words are automatically extracted. It is characterized by having done.

The invention according to claim 4 is the invention according to claims 1 to 3.
In addition to one of the above configurations, it is determined whether or not the document group in the database is the same document by using the header information of the document, and one document is included when a plurality of the same documents are included. It is characterized in that other identical documents are removed except for.

The invention according to claim 5 is based on claims 1 to 4.
In addition to any one of the above constitutions, the important word is a compound word created from a morpheme obtained by dividing the document in the database into units of parts of speech.

The invention according to claim 6 is the same as claims 1 to 5.
In addition to any one of the above configurations, it is characterized in that the important word is a part of speech predicted to represent a characteristic for each document in the database.

The invention according to claim 7 is the invention according to claims 1 to 6.
In addition to any one of the above configurations, it is characterized in that words to be excluded from the important words are retained as an exclusion list, and the words in the exclusion list after extracting the important words are excluded from the important words.

The invention according to claim 8 is the invention according to claims 1 to 7.
In addition to any one of the above configurations, it is characterized in that important words having the same meaning are retained as the same word list, and statistical information of words in the same word list is collectively stored when extracting the important word. .

The invention according to a ninth aspect is the first to the eighth aspects.
In addition to any one of the above configurations, the statistical information is characterized in that it is the total number of appearances in the database and the ratio of the number of documents containing important words in the database.

According to a tenth aspect of the present invention, in addition to the configuration of the ninth aspect, the statistical information includes a plurality of important words within a certain range in addition to the number of times the important word included in the document in the database appears independently. The feature is that the number of appearances of words is also used.

According to the invention of claim 11, in addition to the structure of claim 9, in addition to the statistical information, a surface expression included in a document in the database is automatically extracted and automatically constructed from the surface expression. It is characterized by using the hierarchical relationship of important words.

According to a twelfth aspect of the present invention, in addition to the configuration according to any one of the first to eleventh aspects, a plurality of different search condition expressions are created when the statistical information is calculated, and the plurality of different search conditions are created. Conditional expressions are separately set on the plurality of different processors of a massively parallel computer having a plurality of different processors, and a document group accumulated in a database is simultaneously searched in full text by the plurality of different search conditional expressions. , A result that matches the search condition expression is used.

In addition to the structure of claim 1, an apparatus for automatically extracting related words according to claim 13 further comprises a database part according to claim 1 for storing a document group of a field designated by a user, and the database part. An important word analysis unit that extracts and selects included important words, a count unit that acquires statistical information for the important words selected by the important word analysis unit, and hierarchical relationship information of the important words, and a count that is generated by the counting unit A related word extracting unit that calculates the degree of relevance between important words using a list, and the related word automatic extraction method according to claim 1 is used for a series of processes.

According to a fourteenth aspect of the present invention, a plurality of important words are automatically extracted by using the number of times of appearance of a plurality of important words within a certain range in addition to the number of times of occurrence of a plurality of important words included in a document in a database. In the important word extraction program, the documents in the database are read one by one, the important words are searched from the documents, and it is searched whether or not there is another important word within a predetermined range from the searched important words. However, when an important word existing within a certain range from the important word is searched, the important word pair is sequentially stored in the count list, and the important word pair is searched from the already created count list, and the same important word is already searched. If the pair of is present in the count list, the count list is updated by adding 1 to the count of the number of appearances. If it is not present in the count list, the count of the pair of important words is set to 1. Newly saved und list, these processes performed on multiple documents previously specified in the database, based on a count list created, it is characterized in that to determine the importance of the important word pairs.

According to a fifteenth aspect of the present invention, there is provided a program for extracting a relation between upper and lower hierarchical levels of an important word, which automatically extracts a surface layer expression included in a document in a database and uses a hierarchical relation of important words automatically constructed from the surface layer expression. , The documents in the database are read one by one, the surface expressions written in the surface expression list created in advance are extracted from the documents, and the high-order word part and the low-order word part in the extracted surface expression are extracted. It searches for whether the important word extracted by the important word analysis unit 2 is included, and when the important word is searched for in both the upper word part and the lower word part, the searched upper and lower important word pairs are sequentially counted. Saved to the list, if the same important word pair already exists in the count list, the count list is updated by adding 1 to the count of the number of appearances, and it does not exist in the count list. In this case, the count of the pair of upper and lower important words is newly stored in the count list, these processes are performed for a plurality of documents specified in advance in the database, and the upper and lower hierarchical relationships of the important words are based on the created count list. It is characterized by building.

[0023]

BEST MODE FOR CARRYING OUT THE INVENTION The present invention will now be described in detail based on the illustrated embodiments.

FIG. 1 is a block diagram of an apparatus for automatically extracting related words according to an embodiment of the present invention.

That is, the related word automatic extraction device includes a database section 1 for storing a document group of a field designated by a user, an important word analysis section 2 for extracting and selecting important words contained in the database section 1, A count unit 3 that obtains statistical information and important hierarchical relationship information of important words selected by the important word analysis unit 2, and a relationship that calculates the degree of association between important words using the count list generated by the counting unit 3. The word extracting unit 4 is provided, and the important words that are highly important words are selected from the documents in the database unit 1 and the important words are combined using the statistical information for the important words or the important word pairs. Perform a process to calculate the degree of association of.

The database unit 1 determines the same document from the input document group, and when a plurality of the same documents are included, the same document determination function unit 11 that leaves one document and removes other same documents. And a database 12 that stores the document after the same document is removed by the same document determination function unit 11.

The same document determination function unit 11 will be described in detail below.

For example, when it is assumed that the document in the database 12 is a patent document, "name or name of applicant", "name of invention" and "name of inventor" are extracted from the header portion of the patent document. , (1) "Applicant's name or name" is the same (2) "Name of invention" is the same (3) The number of inventors is the same, and each "inventor's" It is determined that all "names" are the same (in any order). All the document groups that meet the above conditions (1) to (3) are regarded as the same document.

The important word analysis unit 2 comprises a morpheme analysis unit 21 and an important word extraction unit 22.

The morphological analysis unit 21 divides the document in the database into POSs by morphological analysis, and acquires POS information.

In the important word extraction unit 22, a compound word is created by subjecting the morphemes divided by the morpheme analysis unit 21 in units of parts of speech to compound word processing such as combining consecutive nouns. Is stored as an important word in the important word list together with part-of-speech information and statistical information. By creating a compound word, it is possible to avoid the word abstraction due to division and improve the accuracy of the related word to be finally extracted.

The term "important word" is not limited to the compound word created by the above-mentioned method. For example, common nouns other than compound words, proper nouns, undefined words, etc. Specifies the part of speech of the words that are considered to characterize the content.

After the important words are extracted, words to be excluded may be retained as an exclusion list in some cases, and a function of excluding the words in the exclusion list from the important words may be added. Specifically, for each genre of document in the database, for example, “inventor” for patent documents, “comparative example”
It is conceivable to add words that cannot characterize the content of the document to the exclusion list.

This exclusion list may include words that are to be excluded as long as they partially match, in addition to words that have an exclusion condition that they completely match each morpheme.

Furthermore, important words having the same meaning are held as the same word list, and when extracting the important words, statistical information of the words in the important word list is collectively stored to save the important words. The extraction accuracy can be improved.

FIG. 2 is a conceptual diagram of the important word list.

Here, as the "statistical information" to be stored in the important word list, the total number of occurrences 25 of the important word 23 in the database and the ratio of the number of documents 24 including the important word in the database are used. . These are pieces of information that are the basis of various statistics used in the counting unit 3 and the related word extracting unit 41 later.

In order to obtain the number of documents containing important words in the database 12, a plurality of different search condition expressions corresponding to each important word are created, and the plurality of different search condition expressions are provided by a plurality of different processors. The database 12 is separately set on the plurality of different processors of the massively parallel computer.
It is possible to simultaneously and in parallel perform full-text search of the documents stored therein by the plurality of different search condition expressions, and use the result that matches the search condition expression. Here, the number of results that match each search condition expression is the number of documents in which each important word is included in the database 12. The accuracy of the statistical information can be maintained by performing the full-text search each time the processing of the important word analysis unit 2 is performed.

The massively parallel computer is a processor of thousands to tens of thousands (hereinafter, these are collectively referred to as a pipeline).
By incorporating, a plurality of different search condition expressions can be set simultaneously in this pipeline. And
By operating a large number of these processors at the same time, a full-text search that matches a plurality of different search condition expressions with a database is executed. As a result of matching, if a document that matches the search condition expression is found, it has a function of assuming that the document is a hit.

The massively parallel computer is a full-text search engine (for example, FDF (registered trademark) 4T T manufactured by Paracel).
A device such as extFinder is desirable, but a device such as a workstation having a function and performance equivalent to this may be used.

The counting unit 3 includes an extracting unit 31 for extracting a plurality of important words within a certain range and an extracting unit 32 for extracting the upper and lower hierarchical relationships of the important words.
Composed of.

In the related word automatic extraction method, the user selects in advance the processing of either the extraction part 31 of a plurality of important words within a certain range or the extraction part 32 of the hierarchical relation of important words, and the user selects it. Only the processing that was done

In the plural important word extraction unit 31 within a certain range, there is a case where another important word exists within a certain range defined in advance based on the important word extracted by the important word analysis unit 2. It is defined as a plurality of important words, and the number of appearances of the plurality of important words is stored as a count list. The procedure for extracting a plurality of important words is shown in the flowchart of FIG. 5, the details of which will be described later.

In the extraction unit 32 for the upper and lower hierarchical relationships of important words,
A surface expression in which the relationship between the upper word and the lower word is clearly expressed is defined in advance, and the surface expression containing the important word extracted by the important word analysis unit 2 is extracted. The important words in the extracted surface expression are defined as upper important words and lower important words, and the counts of the number of occurrences of these are stored as a count list. The procedure for extracting the hierarchical relationship between important words is shown in the flowchart of FIG. 6, the details of which will be described later.

In FIG. 1, the related word extraction unit 4 comprises a related word extraction unit 41. The related word extraction unit 41 determines a related word based on the count list created by the counting unit 3.

For the related word determination, for example, Information Radius for determining the dissimilarity of two words.
(Christopher D. Manning an
dHinrich Schutze, Foundati
ons Of Statistical Nature
l Language Processing, The
MIT Press (MANFH0-262-13
Although a determination index such as 360-1)) can be used,
Not limited to this, for example, when the extraction unit 31 for a plurality of important words within the certain range is selected, a pair of important words having common important words existing within the certain range or a hierarchical relationship of the upper and lower levels of the important words When the extraction unit 32 is selected, a pair of important words having a common lower important word can be determined as a related word.

FIG. 3 is a conceptual diagram of the count list.
A count list is created with the ID 33 of the important word 1, the ID 34 of the important word 2, and the number of appearances 35 of the pair of the important word 1 and the important word 2 as list items.

FIG. 4 shows the count list of FIG. 3 and FIG.
It is a conceptual diagram of the related degree determination list created based on the important word list of.

The words A, B, which are arranged in each column of FIG.
C, D, ... Are a set of related word determination target words (important words) 42 to be subjected to related word determination, and words a, b, c, d, ... It is a related word used for judgment (important word) 43. Basically, each column and each row are important words extracted by the important word analysis unit 2, and one of the important word pairs extracted by the counting unit 3 is arranged in a column and the other is arranged in a row. For example, in an important word pair existing within a certain range in FIG. 5, the important word A is arranged in a column and the important word B is arranged in a row. In the upper and lower important word pairs in FIG. 6, the upper important words are arranged in columns and the lower important words are arranged in rows. In the degree-of-association determination list of FIG. 4, the number of each cell represents the appearance probability. For example, c
The column A row represents “probability that the important word A and the important word c appear within a certain range” or “probability that the important word A is a superordinate word and the important word c is a subordinate word”.

Hereinafter, as an example of the related word determination, the Information is used to determine the dissimilarity of two words.
A determination example using the Radius determination index will be described.

The statistic is "the dissimilarity of two words" calculated using this appearance probability, and is calculated for all pairs of uppercase alphabets arranged in each column (A
And B, A and C, A and D, ..., B and C, B and D ...
., C and D, ...). Taking the determination of the degree of association between the important word A and the important word D as an example, a, b, c, d for A,
... Appearance probability and a, b, c, d for D, ...
The difference in the appearance probability of is calculated as the dissimilarity. If all rows have the same occurrence probability (a row A column = a row D column, b row A column = b row D column, c row A column = c row D column, d row A column = d row D column) , ...), the dissimilarity is 0, that is, the similarity between A and D is the maximum, and therefore the important word A
And the degree of association between the important word D and the key word D are maximum. On the contrary, if there are no words a, b, c, d, ... Of which both appearance probabilities are not 0, the dissimilarity is maximum, that is, the degree of association is minimum. As described above, statistics are calculated for all pairs of uppercase alphabets, and only pairs with a certain threshold value or less are determined to be related words (related words).

FIG. 5 is a flowchart showing a procedure for counting the number of simultaneous appearances of a plurality of important words existing within a certain range in the related word automatic extraction method according to the embodiment of the present invention.

First, the documents in the database are read one by one (step S1), and the important words extracted by the important word analysis unit 2 are searched from the documents (step S2).

The important word to be searched here is not limited to the one extracted by the important word analysis unit 2, but may be a word included in a user-defined important word list defined by the user in some cases. The user-defined important word list may include words that are to be searched as long as they partially match, in addition to words that have a perfect matching as a search condition.

Further, as a criterion for determining the importance of the word to be searched, the total number of appearances in the database, the ratio of the number of documents including the important word in the database, and the number of characters are filtered as necessary for the important word to be searched. May be applied to. By applying these various filters, the important words can be further narrowed down, and as a result, the accuracy of the related words finally extracted can be improved.

When an important word is searched (when YES is determined in step S3), another important word (this is called important word A) is searched within a predetermined range from the searched important word (this is called important word A). Is referred to as an important word B) is searched (step S4).

The term “within a certain range” is defined as, for example, within one sentence (the range from the beginning of one sentence to the punctuation “.”), Which is close to two before and after, within a certain range. Specify the range that is expected to represent the characteristics for each document.

Important word B existing within a certain range from important word A
Is searched (when YES is determined in step S5), the pair of important word A and important word B is sequentially stored in the count list.

When a pair of the important word A and the important word B is searched from the already created count list (step S6), when the same pair already exists in the count list (step S6).
If YES in 7), 1 is added to the count of the number of appearances and the count list is updated (step S8).

If it does not exist in the count list (NO in step S7), the count of the important word A and important word B pair is set to 1 and is newly stored in the count list (step S9). .

As described above, the processes of steps S1 to S9 are performed for a plurality of documents designated in advance in the database (step S10).

After that, the degree of importance of the pair of the important word A and the important word B is determined based on the statistical information in the count list and the important word list created in steps S1 to S10 (step S11). For the step S11, for example, a Dice coefficient or mutual information amount can be used.

FIG. 6 is a flow chart showing a procedure for extracting the hierarchical relationship between important words in the related word automatic extraction method according to the embodiment of the present invention.

First, the documents in the database are read one by one (step S21), and the surface expression written in the surface expression list created in advance is extracted from the document (step S22).

Here, the surface expression to be written in the surface expression list is one in which the relationship between the upper word and the lower word is clearly expressed. For example, "D of A, B, C, etc." (A to A). In the expression "D is an important word", the upper word is D and the lower words are A, B, and C.

Next, the important word analysis unit 2 is added to the high-order word portion and the low-order word portion in the surface expression extracted in the step S22 (when YES is determined in the step S23).
It is searched whether or not the important word extracted in (3) is included (step S24).

The important word to be searched here is not limited to the one extracted by the important word analysis unit 2, but may be a word included in the user-defined important word list defined by the user in some cases. Further, the user-defined important word list may include words that are to be searched as long as they partially match, in addition to words that have a perfect match as a search condition.

By this search, when an important word is searched for in both the upper word part and the lower word part (step S
If YES is determined in 25), the searched upper and lower important word pairs are sequentially stored in the count list. At this time, the database 1 is used as a criterion for determining the importance of the upper and lower important word pairs.
The ratio of the number of documents in which the upper important word and the lower important word are included in 2 is compared, the morphemes of the upper important word and the lower important word are compared, and the upper and lower important word pairs to be excluded are kept as the upper and lower important word pair exclusion list. A function of excluding upper and lower important word pairs in the upper and lower important word pair exclusion list may be applied as necessary.

The upper and lower important word pairs are searched from the already created count list (step S26), and when the same pair already exists in the count list (YE at step S27).
If it is determined to be S), the count list is updated by adding 1 to the count of the number of appearances (step S28).

If it does not exist in the count list (NO in step S27), the count of the pair of upper and lower important words is set to 1 and newly stored in the count list (step S29).

Above, steps S21 to S29
This process is performed for a plurality of documents specified in advance in the database (step S30).

After that, based on the statistical information in the count list and the important word list created in steps S21 to S30, the hierarchical relation of the important words is constructed (step S).
31).

Specifically, for example, when the upper important words A and B having the common lower important word C are extracted, and the upper important word A and the lower important word B are extracted at the same time,
As a whole, the pair that has a direct hierarchical relationship is A
Only the (upper) -B (lower) pair and the B (upper) -C (lower) pair, and the A (upper) -C (lower) pair are redundant. Therefore, the redundant pairs A to C are excluded when constructing the hierarchical relationship of important words.

Further, when constructing the upper and lower hierarchical relations, a threshold may be set for the total number of appearances of the upper and lower important word pairs in the database, and the upper and lower important word pairs less than the threshold may be excluded as necessary.

[0075]

As described above, according to the invention of claim 1, a document group in a field designated by a user is used as a database, and an important word which is a highly important word from the documents in the database. , Which is not included in a general thesaurus dictionary, is selected by the user, and the degree of association between important words is calculated by using the statistical information for the important word or the pair of important words to extract the related words. It is possible to provide a method for automatically extracting a technical term appearing in a specific field, a new word and a buzzword, and an apparatus using the method.

Further, it is possible to provide a method for accurately and accurately extracting an important word that is closely related to a word designated by the user, and an apparatus using the method.

According to the invention of claim 2, claim 1
In addition to the effect of, since the related words for each field can be automatically extracted when documents in multiple fields are accumulated in the database, However, it is possible to extract a field-specific related word that is not a related word in another field. In addition, since the user can set the field independently regardless of the field of the existing thesaurus dictionary, it becomes possible to extract the related words according to the level of the field set.

According to the invention of claim 3, claim 1
In addition to the effect of (2), the database can be updated / added at any time, and by reflecting the difference data sequentially, the latest related words including new words and buzzwords that always reflect the latest database information. Can be extracted.

According to the invention of claim 4, claim 1
In addition to the effect of any one of 1 to 3, it is determined whether the documents in the database are the same document by using the header information of the document, and if a plurality of the same documents are included, one document is selected. Since other identical documents are removed while remaining, it is possible to eliminate unnecessary bias of statistical information that occurs when a specific document has many identical documents, and as a result, the related word extraction accuracy is improved. It becomes possible.

According to the invention of claim 5, claim 1
In addition to any one of the effects 4 to 4, since the important word is a compound word created by dividing the document in the database into parts of speech and created from the divided morphemes, avoiding word abstraction due to the division Therefore, it is possible to improve the accuracy of the related words to be finally extracted.

According to the invention of claim 6, claim 1
In addition to the effect of any one of 5 to 5, since the important word is the part of speech predicted to represent the feature for each document in the database, it is possible to reduce the omission of the important word to be extracted.

According to the invention of claim 7, claim 1
In addition to the effect of any one of to 6, the unnecessary words can be excluded because the words to be excluded from the important words are held as an exclusion list and the words in the exclusion list are excluded from the important words after the important words are extracted.

According to the invention of claim 8, claim 1
In addition to the effect of any one of 7 to 7, because the configuration is such that important words having the same meaning are held as the same word list and statistical information of the words in the same word list is collectively stored when extracting the important words. , It is possible to improve the accuracy of extracting important words.

According to the invention of claim 9, claim 1
In addition to the effect of any one of 8 to 8, since the statistical information is the total number of appearances in the database and the ratio of the number of documents including the important word in the database, the extraction accuracy can be improved.

According to the invention described in claim 10, in addition to the effect of claim 9, the statistical information includes a plurality of important words within a certain range in addition to the number of times the important word included in the document in the database appears independently. Since the number of appearances of a word is also used, it is possible to make the meaning by a pair of a plurality of important words more accurate, and as a result, it is possible to improve the accuracy of extracting the related word.

According to the invention of claim 11, in addition to the effect of claim 9, in addition to the statistical information, the surface expression included in the document in the database is automatically extracted and automatically constructed from the surface expression. Since the upper and lower hierarchical relationships of the important words are used, it is possible to remove noise caused by the accidental appearance of a plurality of unrelated important words, and as a result, it is possible to improve the extraction accuracy of the related words.

According to the twelfth aspect of the present invention, in addition to the effect of any one of the first to eleventh aspects, a plurality of different search condition expressions are created when the statistical information is calculated, and the plurality of different search condition expressions are created. Search condition expressions are separately set on the plurality of different processors of a massively parallel computer having a plurality of different processors, and a group of documents accumulated in a database are simultaneously searched in full text by the plurality of different search condition expressions. However, since the result matching the search condition expression is used, it is possible to use accurate statistical information corresponding to the latest database every time the related word automatic extraction method is applied, and as a result, the related word extraction accuracy is improved. It becomes possible.

According to the thirteenth aspect of the present invention, the related word automatic extraction device stores the database section according to the first aspect for storing the document group of the field designated by the user and the important word contained in the database section. It is important to use an important word analysis unit to extract / select, a count unit to obtain statistical information and important hierarchical relationship information of important words selected by the important word analysis unit, and a count list generated by the counting unit. It consists of a related word extraction unit that calculates the degree of relevance between words,
Since the related word automatic extraction method according to claim 1 is used for the series of processes, the user does not need to be aware of the internal structure of the related word automatic extraction device, and the related terms such as technical terms, new words, and buzzwords desired by the user can be obtained. It becomes possible to extract words accurately.

According to a fourteenth aspect of the present invention, a plurality of important words are automatically extracted by using the number of times of appearance of a plurality of important words within a certain range in addition to the number of times of occurrence of a plurality of important words included in a document in a database. In the important word extraction program, the documents in the database are read one by one, the important words are searched from the documents, and it is searched whether or not there is another important word within a predetermined range from the searched important words. However, when an important word existing within a certain range from the important word is searched, the important word pair is sequentially stored in the count list, and the important word pair is searched from the already created count list, and the same important word is already searched. If the pair of is present in the count list, the count list is updated by adding 1 to the count of the number of appearances. If it is not present in the count list, the count of the pair of important words is set to 1. It is newly stored in the count list, and these processes are performed on multiple pre-specified documents in the database, and the importance of pairs of important words is judged based on the created count list. It is possible to rationalize the meanings of the pairs, and as a result, it is possible to improve the precision of extracting related words.

According to a fifteenth aspect of the present invention, there is provided an important word upper / lower hierarchical relation extracting program which automatically extracts a surface expression included in a document in a database and uses an upper / lower hierarchical relationship of important words automatically constructed from the surface expression. , The documents in the database are read one by one, the surface expressions written in the surface expression list created in advance are extracted from the documents, and the high-order word part and the low-order word part in the extracted surface expression are extracted. A search is performed to determine whether or not the important word extracted by the important word analysis unit is included. When both the upper word portion and the lower word portion are searched for the important word, the searched upper and lower important word pairs are sequentially counted in a list. If the same important word pair already exists in the count list, the count list is updated by adding 1 to the count of the number of appearances, and the word does not exist in the count list. In this case, the count of the pair of upper and lower important words is set to 1 and newly stored in the count list, these processes are performed for a plurality of documents specified in advance in the database, and the upper and lower hierarchical relationships of the important words are based on the created count list. By constructing, it is possible to rationally remove noise caused by the accidental appearance of multiple unrelated important words.

[Brief description of drawings]

FIG. 1 is a block diagram of a related word automatic extraction device according to an embodiment of the present invention.

FIG. 2 is a conceptual diagram of an important word list used in the related word automatic extraction device according to the embodiment.

FIG. 3 is a conceptual diagram of a count list used in the related term automatic extraction device according to the embodiment.

4 is a conceptual diagram of a degree-of-association determination list created based on the count list of FIG. 3 and the important word list of FIG.

FIG. 5 is a flowchart showing a procedure for extracting a plurality of important words within a certain range in the related word automatic extraction method according to the embodiment.

FIG. 6 is a flowchart showing a procedure for extracting a hierarchical relationship of important words in the related word automatic extraction method according to the embodiment.

[Explanation of symbols]

1 database department 2 Important word analysis section 3 counting section 4 Related term extractor

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Naoko Yoshino             Mitsubishi Spa, 792 Kamimachiya, Kamakura City, Kanagawa Prefecture             Software Co., Ltd. in Kamakura Division (72) Inventor Kazuko Adachi             Mitsubishi Spa, 792 Kamimachiya, Kamakura City, Kanagawa Prefecture             Software Co., Ltd. in Kamakura Division F-term (reference) 5B075 ND03 NK33 NK35 NR05 PQ32                       PR04 PR08 QP03

Claims (15)

[Claims] 1. A document group in a field designated by a user is used as a database, important words that are highly important words are selected from documents in the database, and the important words or pairs of important words are stored in the database. A method for automatically extracting a related word, characterized in that the degree of relevance between important words is calculated using statistical information of included words to extract the related words. 2. The related word automatic extraction according to claim 1, wherein when the database stores documents in a plurality of fields, related words for each field can be automatically extracted. Method. 3. The database is updated / updated at any time.
The related word automatic extraction method according to claim 1 or 2, wherein addition is possible, and the differential data is sequentially reflected in the automatic extraction of related words. 4. The group of documents in the database is judged whether or not they are the same document by using the header information of the documents, and when a plurality of the same documents are included, one document is left and other documents are the same. The related word automatic extraction method according to claim 1, wherein the document is removed. 5. The related word automatic extraction method according to claim 1, wherein the important word is a compound word created by dividing a document in a database into parts of speech and created from the divided morphemes. 6. The related word automatic extraction method according to claim 1, wherein the important word is a part of speech predicted to represent a feature for each document in the database. 7. The related word automatic according to claim 1, wherein words excluded from important words are retained as an exclusion list, and words in the exclusion list are excluded from important words after the important words are extracted. Extraction method. 8. The method according to claim 1, wherein important words having the same meaning are held as a same word list, and statistical information of words in the same word list is collectively stored when the important words are extracted. The related word automatic extraction method described in. 9. The method for automatically extracting a related word according to claim 1, wherein the statistical information is a total number of appearances in a database and a ratio of the number of documents including an important word in the database. . 10. The statistical information includes the number of appearances of a plurality of important words included in a document in the database, as well as the number of appearances of a plurality of important words within a certain range. Automatic extraction method of related words. 11. A method for automatically extracting a surface expression included in a document in the database in addition to the statistical information and using a hierarchical relation of important words automatically constructed from the surface expression. 9. The related word automatic extraction method according to item 9. 12. When calculating the statistical information, a plurality of different search condition expressions are created, and the plurality of different search condition expressions are separately provided on the plurality of different processors of a massively parallel computer having a plurality of different processors. 12. A set of documents, which are stored in a database, are simultaneously searched for in full text by the plurality of different search condition expressions, and a result that matches the search condition expression is used. A method for automatically extracting a related word according to any one of 1. 13. The database unit according to claim 1, which stores a group of documents in a field designated by a user, an important word analysis unit for extracting and selecting important words contained in the database unit, and the important word analysis unit. It consists of a counting unit that acquires statistical information and important hierarchical relation information of the important words selected in 1., and a related word extracting unit that calculates the degree of association between the important words using the count list generated by the counting unit. ,
An apparatus for automatically extracting related words, wherein the method for automatically extracting related words according to claim 1 is used for a series of processes. 14. A multi-important word extraction program for automatically extracting a plurality of important words by using the number of appearances of a plurality of important words within a certain range in addition to the number of single occurrences of the important words included in a document in the database. Each document is read one by one, an important word is searched from the document, and it is searched whether another important word exists within a predetermined range defined from the searched important word. When an important word existing inside is searched for, the important word pairs are sequentially stored in the count list, the important word pairs are searched from the already created count list, and the same important word pair already exists in the count list. If you do, add 1 to the count of the number of appearances and update the count list,
If it does not exist in the count list, the count of the pair of important words is set to 1 and newly stored in the count list, and these processes are performed for a plurality of documents specified in advance in the database, and based on the created count list. A multi-important word extraction program characterized by determining the importance of a pair of important words. 15. A key word upper / lower hierarchical relation extraction program that automatically extracts a surface expression included in a document in a database and uses the upper / lower hierarchical relation of important words automatically constructed from the surface expression. The document is read one by one, the surface expression written in the surface expression list created in advance is extracted from the document, and the important word analysis unit 2 extracts the upper word part and the lower word part in the extracted surface expression. Search whether the extracted important word is included or not, and if the important word is searched for in both the upper word part and the lower word part, the searched upper and lower important word pairs are sequentially saved in the count list and are already the same. If a pair of important words of is present in the count list,
The count list is updated by adding 1 to the count of the number of appearances, and if it does not exist in the count list, the count of the pair of upper and lower important words is set to 1 and newly stored in the count list, and these processes are stored in the database. An important word upper / lower hierarchical relation extraction program, which is performed for a plurality of documents designated in advance, and constructs an upper / lower hierarchical relation of important words based on a created count list.