JPH0776971B2 - Document abstract creation device - Google Patents

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document abstract creating apparatus for automatically creating an abstract from a machine-readable document.

(Prior Art) Conventionally, as a technology for creating an abstract of a document, there is a method of extracting a keyword to facilitate a search, and a method of extracting an important part using surface information of a sentence. It was

The former keyword method divides the document into words, records the frequency and appearance location of the words, applies an evaluation function to each word to obtain an evaluation value, and assigns words with high evaluation as the keywords of the document. To do. The main purpose of this is to facilitate the search for a desired document from a large number of documents. Furthermore, in this keyword method, a method of evaluating the importance of a sentence based on the length of the sentence and the number of keywords included in the sentence has been proposed (Reference: "The Automatic Creation of Literature Abstr
acts), IBM Journal, April 1958 ").

Several methods have been proposed for extracting important parts using the surface information of the latter document, but by focusing on the main verbs of a sentence and analyzing their relationships, the structure of the document Decide what, and then take out the main part,
Some examples are those that focus on the connection relationship.

(Problems to be Solved by the Invention) In such a conventional method, the importance is evaluated on a word-by-word basis or on a sentence-by-sentence basis, so that it is not possible to extract only an important part of one sentence. .

The keyword method is useful for searching documents, but since it does not check the contents, there is a problem that it is necessary to see the contents of the document without knowing which part is particularly important. In addition, the evaluation method used in this case has a problem that it cannot be used to determine which key sentence is because the unit is a word.

In addition, in the method of extracting important parts using surface information, there is no problem in the keyword method, but when only surface information is used, unimportant parts are often extracted. there were. For example, a part including enumeration will be extracted. Moreover, since the method of evaluating the importance is fixed, there is a problem that the evaluation cannot be performed from different viewpoints.

An object of the present invention is to provide a document abstract creating apparatus capable of performing a more accurate evaluation than the conventional method and an evaluation of importance according to a request.

(Means for Solving the Problem) In order to achieve this object, a document abstract creation apparatus of the present invention analyzes an input device for reading a sentence of an input document and a context structure of the read document, and at least For the context analysis device that outputs information including the relationship between sentences and sentences and the direction thereof and information including anaphora and omission information that appears in a document as an analysis result of the document structure, and for the importance evaluation of sentences and words, A database of rules including a vocabulary level rule for calculating a word and a sentence evaluation value from a vocabulary included in a sentence and a document structure level rule for calculating a sentence evaluation value from an analysis result of the context structure. A database device for storing and a vocabulary level are evaluated using the rules, and the importance of sentences and words is evaluated from the analysis result of the document structure output by the context analysis device. And an evaluation device for performing.

(Operation) With respect to the first problem described above, since not only words but also words and sentences are evaluated, only important sentences can be picked up. Moreover, in order to make up for the shortcomings of the method using only surface information, the structure of the document is analyzed, and the important part is extracted using the result. This makes it possible to more accurately extract the part of the document that the author wants to convey. With respect to the third problem, the rule used for evaluation can be changed, so that evaluation can be performed with particular emphasis on a certain point of interest. For example, it is possible to perform an evaluation that emphasizes the structure of a document and an evaluation that emphasizes a certain keyword.

(Embodiment) An embodiment of the document abstract creation apparatus of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing an outline of the present invention, 10 is an input document, 12 is an input device for reading a sentence of the input document 10, and 14 is a context analysis device for performing a context analysis of the read document. , 16 is an evaluation device that evaluates the importance of sentences and words in the read document, and 18 is a generation device that generates an abstract of the document,
Reference numeral 20 is a database device that records a database of rules used in the evaluation device 16 to evaluate importance.

First, the input device 10 reads the input document 10. At this time, the morpheme analysis is performed by referring to the dictionary, the document is divided into words, and the part-of-speech information and the like are added and temporarily stored in the storage device 12a. Next, receiving the output from the input device 12, the context analysis device 14 analyzes the document structure, and stores the result in an appropriate storage device 14a provided in the context analysis device 14. The analysis result of the document structure includes the information of “sentence-to-sentence connection relation and its direction” and the information of “anaphora and omission appearing in document”.

Next, the evaluation device 16 evaluates the importance of sentences and words.

In order to evaluate the importance of sentences and words, this device reads and uses the result of morphological analysis output from the input device 12, the result of syntactic analysis once stored in the storage device 14a, and the result of context analysis. In addition, a database of evaluation rules is read from the database device 20 shown in FIG.
This gives rules for giving scores to sentences or words and weighting for each rule. Therefore, adding rules,
The weighting can be changed easily.

The rules database used here consists of two parts.

The first part of the rule database uses the result of the morphological analysis output from the input device 12 to evaluate the importance based on the vocabulary level information. It has an importance evaluation rule based on a word pattern for measuring this importance, and the importance is calculated by the rule using this.

An example of this rule is as follows. The following are given as keywords.

Frequent words: Divide the frequency of words in the document by the number of words in the document,
A word with a certain level or more is a high-frequency word. However, functional words such as particles and auxiliary verbs are not included in the number of words.

Emphasized expression: Emphasized words are words emphasized or enclosed in parentheses.

Important word: A word that clearly indicates that the part is important (eg, after all, in short, etc.) Important word by field: Gives a set of words indicating the degree of importance for each field and type of document.

Each of these words is given a weight. The score of the sentence is the sum of the scores of the keywords in the sentence.

The second part of the rule database uses the parsed document structure information. This information depends on the dependencies between the statements,
And anaphoric relationships.

An example of the evaluation criteria is given below.

Number of sentences that depend on the sentence: Check the dependency of sentences and count the number of sentences that directly or indirectly depend on the sentence to obtain the score. Here, the sentence 1 spans the sentence 2 means that there is a dependency between the two sentences and the direction is from 1 to 2.

Number of times a word is anaphorated: The number of times a word is anaphorated is counted and used as the score for that word. For sentences, the total score of each word in the sentence is calculated and used as the sentence score.

Embedded sentence check: A negative evaluation point is given to the embedded sentence.

Evaluation by sentence type: A function depending on the sentence type and the document type is determined, and the score is determined accordingly. For example, in editorials, a high score is given to the part in which an opinion is stated.

The evaluation in the evaluation device 16 is performed independently for each rule. Since a weight is given to each rule, the sum of products of the evaluation value and the weight is used as the score.
Expressed as a formula, Becomes Where Ei is the evaluation value for the i-th rule,
Wi is a weight for the i-th rule, and E is the final evaluation value.

When the evaluation value is obtained for each sentence, the result is generated by the generator 18
Hand over. The generator 18 extracts the high-scoring part,
Output as a document abstract.

The invention is not limited to the embodiments described above. For example, the storage device 14a may be provided independently of the context analysis device 14. Further, although the input device 12 is provided with the storage device 12a for temporarily storing the result of the morphological analysis, it may be provided separately from the input device 12. Further, the database device 20 may be provided in the evaluation device 16.

(Effects of the Invention) As is clear from the above description, according to the document abstract creation apparatus of the present invention, it is possible to browse a document and search for a necessary document by automatically extracting an important part from a sentence. It will be easier.

[Brief description of drawings]

FIG. 1 is a block diagram showing an example of a document abstract creating apparatus of the present invention. 10 …… input document, 12 …… input device 12a …… storage device, 14 …… context analysis device 14a …… storage device, 16 …… evaluation device 18 …… generation device, 20 …… database device.

Claims (1)

[Claims] 1. An input device for reading a sentence of an input document and an analysis of a context structure of the read document, and at least information of connection relation between the sentence and the sentence and its direction, and anaphora and omission which appear in the document. A context analysis device that outputs information including information as an analysis result of a document structure, and a vocabulary level rule for calculating an evaluation value of a word and a sentence from a vocabulary included in the sentence to evaluate the importance of the sentence and the word And a database device for storing a rule database including a document structure level rule for calculating a sentence evaluation value from the context structure analysis result, and a vocabulary level evaluation using the rule and the context analysis An apparatus for creating a document abstract, comprising: an evaluation device that evaluates the importance of sentences and words from the analysis result of the document structure output by the device.