JPH0474259A - Document summarizing device - Google Patents

Abstract

PURPOSE:To improve the accuracy of recognition for decision of the importance of the sentences forming a document by recognizing and deciding the important sentences of the document based on the analyzing results of the component sentences of the document and the knowledges prepared for recognition of the important parts of the document which are stored in a knowledge storage means. CONSTITUTION:A sentence analyzing part 2 extracts the component sentences of a document out of the document information inputted from a sentence input part 1 and analyzes the morphemes of these sentences. At the same time, the character modifying information added to the document information is also detected and given to an important sentence deciding part 3 together with the morpheme analyzing results to be compared with the character modifying information 5 stored in a knowledge storage part 4. Then the part 3 extracts the sentences including the characters added with the character modifying information as the highly important sentences and at the same time applies properly a general important sentence extracting method to extract the highly important sentences out of a document. Thus it is possible to improve the recognizing performance of the sentence important and to produce a summary sentence of high quality.

Description

DETAILED DESCRIPTION OF THE INVENTION [Object of the Invention] (Industrial Application Field) The present invention relates to a document summarization device that can accurately and efficiently create a summary text that extracts important parts of a document.

(Conventional technology) With the development of the information society, various types of document information are overflowing.

Various attempts have been made to automatically create document summaries with the aim of efficiently organizing such large amounts of document information into categories.

Generally, summarizing a document is achieved by extracting important parts of the document and connecting them together in the form of a summary sentence. Therefore, it can be said that the technology for recognizing important parts of a document is a very important basic technology for creating a summary sentence.

By the way, the conventional methods for recognizing important parts of a document are: ■ A method that focuses on the frequency and distribution of words. ■ A method that focuses on words that express claims, evaluations, and opinions. ■ A method that focuses on words that express assertions, evaluations, and opinions. Techniques that utilize knowledge ■ Techniques that focus on the document structure indicated by conjunctions, etc. have been proposed.

■ “Method that focuses on the frequency and distribution of words”
Regarding important matters to be asserted in the document, for the purpose of helping the reader understand, etc.
Based on the premise that words are often used repeatedly or paraphrased using similar words, it is possible to simply aggregate and process sentences containing words that are distributed and appear in large numbers over a wide range of documents. Since the summarization process is good, it is relatively easy to implement the summarization process. However, on the other hand, sentences that do not include words determined to be important are excluded from extraction targets for creating a summary sentence. As a result, if an important word is referred to by a pronoun, for example, even if the sentence has an important meaning, it will not be used to create a summary sentence.

Therefore, there is a problem in that it is often impossible to create an appropriate summary sentence.

In addition, ■ ``8 methods that focus on words that express claims, evaluations, and opinions'' is a method that is often used to summarize editorial-style texts (opinionary texts), and is a method that is often used to summarize editorial-style texts (opinionary texts). It is based on the premise that the arguments, evaluations, and opinions of document creators are important. However, with this method, important sentences include sentences that include colloquialisms or specific expressions used when expressing assertions, opinions, speculations, etc., such as "I think..." or "U..." It is judged and extracted as such, and a summary sentence is created.

Furthermore, ``techniques that utilize non-linguistic knowledge'' in ■ are not only based on linguistic knowledge such as part-of-speech information and grammar of each word used to analyze sentences, but also how concepts attached to words are used in real society. This method uses knowledge other than linguistic knowledge, such as knowledge about whether a document has certain properties, to summarize the document. Specifically, rather than literally summarizing the text,
Rather, it is used for extracting and organizing information about a single matter from a document and outputting it in a table format, for example, depending on the user's purpose or interest.

Such a method is very effective for document summarization in a specific field for which knowledge is prepared in advance.

However, it is not only extremely difficult to prepare a large amount of such non-linguistic knowledge, but it is also extremely difficult to retain that knowledge.

Moreover, the time required for this processing is extremely large, and there is a problem in that it is extremely difficult to realize it at the current technological level.

For these, we will find out which sentence indicated by the conjunction, etc. in ■ is more important.Specifically, the conjunction "therefore" connects two sentences in a "conjunctive" relationship. , the part (sentence) located after is detected as an important part. Further, when two sentences are connected in a relationship of "reason" by a conjunction "because", the preceding sentence is detected as an important part.

By determining the sentence structure based on the rhetorical relationships of the document, we can find sentences with higher importance based on the structure (
parts) in order. Furthermore, in order to obtain the above-mentioned structure, it is necessary to recognize the rhetorical relationships between sentences, but in order to find rhetorical relationships, it is sufficient to focus primarily on conjunctions, and the number of these conjunctions is relatively limited. Therefore, it is possible to extract important parts of a document with higher precision than with the other methods mentioned above.

However, depending on the text, there are cases where there are almost no conjunctions that indicate the rhetorical relationship between words, and as a result, there is a problem that it is not possible to make a synopsis.In the end, only this method is used to create a highly accurate summary sentence. There is a problem.

(Problem to be Solved by the Invention) As described above, various methods of creating summaries that have been proposed in the past have poor recognition accuracy for extracting important sentences from documents, so The content of the text often differs significantly from the content of the original document, making it extremely difficult to obtain a satisfactorily accurate summary text.

The present invention was developed in consideration of these circumstances, and its purpose is to improve the recognition accuracy for determining the importance of the sentences that make up a document, and to provide a highly accurate summary that is rich in content. An object of the present invention is to provide a document summarization device that can create sentences.

[Configuration of the Invention (Means for Solving the Problems) The document summarization device according to the present invention analyzes each sentence constituting a document to be summarized, and stores the results of this analysis and the knowledge storage means. This feature is characterized in that a summary sentence is created by extracting sentences from a given document.

(Function) According to the present invention, character modification information attached to characters in a document is used as a clue for determining sentences with high importance in a document, so it can be used, for example, in combination with the conventional method described above. By recognizing and determining sentences with high importance in a document,
It becomes possible to extract important sentences with high precision and connect these sentences to create a rich summary sentence.

(Embodiment) A document summarization device according to an embodiment of the present invention will be described below with reference to the drawings.

FIG. 19 is a block diagram showing a schematic configuration of the embodiment device, and l is inside the Bunjin. This human power section I includes a buffer memory function for storing document information inputted from a keyboard or the like to create a document, or a memory function for inputting document information created by another device and storing and holding it. In addition, a document image is input, and it is recognized and processed, and the sentences that make up the sentence are extracted using clues such as document paragraphs, and each of these sentences is detected in this way. The character modification information is given to the important sentence determination unit 3 together with the morphological analysis result of the sentence, and the knowledge information regarding the important sentence stored in advance in the knowledge storage unit 4, that is, the character modification information that can identify the important sentence. It is used for comparison with information 5. The important sentence determination unit 3 extracts sentences including characters to which character modification information is added as sentences with high importance according to the comparison with such knowledge information, and also extracts sentences that include characters with character modification information added as sentences with high importance. Sentences with high importance from the document are extracted by appropriately employing an extraction method.

The summary sentence creation unit 6 inputs the sentences with high importance extracted by the important sentence determination unit 3 as described above, connects these sentences in accordance with the extraction order, etc., creates a required essay, and outputs this. do.

The character modification information added to the characters of sentences constituting the document is given in the format shown in FIG. 2, for example. For example, the upper 4 bits are used to specify the type of character modification such as underlining, shading 1 emphasis, hollowing, etc., and the remaining lower 4 bits are
They are placed in a document so that the range of character modification can be specified using bits, up to a maximum of 16 characters.

That is, the document information is given in an internal representation format as shown in FIG. 3(a), for example. This example shows the character code in hexadecimal notation, and the sentence ``This is an underline.'' is given as a code string ra4b3a4ec-.-ala3PPJ.

Therefore, the character modification information is the character modification information designation symbol "Ol".
" followed by "82" etc., which are inserted into the character code string described above. With such character modification information, the characters "bottom, line" indicated by the character codes r b2bc and cOreJ in the sentence are each underlined. As a result, a sentence with such character modification information, for example, as shown in Figure 3 (
As shown in b), the "underlined" portion of the character is displayed or printed out with an underline attached.

The sentence analysis unit 2, which analyzes electronic document information including character modification information as described above, morphologically analyzes each sentence in the document one by one. At this time, the number of the baragraph in which the sentence appears and the sentence number within the baragraph are registered for each pointer clause in the sentence analysis section 2.

At this time, the upper four bits of the character modification information in the format shown in FIG. 2 described above are used as the character modification information in the result buffer 2b.

Also, even if the character modification is applied to only part of the word,
Here, it is assumed that all of the word has undergone character modification, and character modification information is attached to the word on the result buffer 2b.

In addition, the items of importance in pointer table 2a are as follows:
The importance determination results for each sentence determined by the important sentence determining section 3 are stored.

Now, the knowledge information 5 for recognizing important sentences registered in advance in the knowledge storage section 4 includes items such as "type", "content", "location information", and "importance" as shown in FIG. It is given as information that you have.

The content of knowledge described in the ``Content'' item is r type.''
If the item is “0”, it is described as importance judgment knowledge based on expressions such as various quantities, and if the item “type” is “1”, importance judgment is based on character modification information. It is given as hexadecimal information corresponding to information described as knowledge.

In addition, the content described in the "location information" item specifies whether to focus on the part in the sentence when determining the condition described in the "content" item. For example, if 0'' The specification is such that ``1'' indicates the beginning of the sentence, ``1'' indicates the end of the sentence, ``2'' indicates any position within the sentence, and ``3'' specifies the entire sentence.

Further, the "importance" item stores information indicating the degree of importance expressed as a numerical value ranging from [-1 to [1], for example. For example, the closer the value is to 1, the higher the importance is.
-1] indicates that the importance level is lower.

Knowledge information regarding importance described in such a format is stored in the knowledge storage unit 4 in a table format as shown in FIG. 6, for example.

In the example shown in Figure 6, the information in the first column indicates that ``If all words in a sentence are underlined.

By comparing such knowledge information with the analysis results described above, the important sentence determination unit 3 determines the importance of each sentence found from the input document, and uses the information on the importance as described above. It is described in the "importance" item of the pointer table 2a. Then, the summary sentence creation unit 6 extracts sentences with high importance by paying attention to the "importance" item in the pointer table 2a, and combines these sentences to create a summary sentence.

This processing procedure starts with initializing the control parameter I to [1] for sequentially referencing the pointer table 2a for each sentence (step a). After that,
Pointer table 2 specified by control parameter I above
Set the importance of a to [0], set the pointer value specified by the above control parameter I to variable P1, and set the pointer value specified by the control parameter I to variable P2.
A pointer value specified by the control parameter (1+1) is set for each (step b).

The above variable pt is the variable P2 in which the sentence analysis result is registered.
specifies the first element (clause) of the analysis result for the sentence next to the sentence specified by control parameter I.

By comparing the values of the variables PI and P2 set in this way with each other, for example, if the variables PI and P2 are equal, the importance of the sentence can be determined due to reasons such as a failure in the sentence analysis process. It is detected that the element to be determined does not exist (step C). In this way, if it is determined that there is no element for determining the importance in the sentence specified by the control parameter l, the control parameter 1 is incremented and step d), the sentence specified by the control parameter I is After confirming that the value of the paragraph in the pointer table 2a is not "0", the process returns to step b and starts the next process based on the control parameter I in the same way (step e).

If it is confirmed in the above-mentioned variable check that there is an element for determining importance in the sentence, then control is executed to sequentially refer to the knowledge 5 registered in the knowledge storage section 4. After performing the initial settings for knowledge reference such as parameters, the value of the above-mentioned position information shown in the variable K has the meaning as described above [0 [1] [2]
] [3] (step h
, i, j).

However, if the value indicated by the variable K is [0], the content of the knowledge is valid only at the beginning of the sentence, so the result buffer 2b specified by the variable P1
Check only the contents of (step k). If the content of the result buffer 2b does not satisfy the condition indicated in the important sentence judgment knowledge, the control parameter J is incremented in step ``)'' to check whether the condition is similarly satisfied for the next knowledge. This is repeated until the check for knowledge of is completed (step S).

On the other hand, if the content of the result buffer 2b satisfies the conditions indicated by the important sentence determination knowledge, the importance of the important sentence determination knowledge specified by the control parameter J and the importance of the important sentence determination knowledge specified by the control parameter I are determined. If the value shown in the variable K is 1, it is shown that the content of the knowledge consists only of the last clause of the sentence. Then, only the contents of the result buffer 2b specified by the value [P2-1] obtained by subtracting [1] from the value specified by the variable P2 are examined (step m). Then, according to the contents of the result buffer 2b, the same processing procedure as in the case where the value indicated by the variable K is [0] is performed to obtain the importance value for that sentence.

Similarly, if the value indicated by the variable K is [2], it is checked whether or not there is a clause in which the knowledge is established in the sentence specified by the control parameter ■ (step n); If the value indicated by the variable K is [3], it is checked whether the knowledge holds true for all clauses forming the sentence specified by the control parameter I (step O).

In this way, whether or not the sentence specified by the control parameter I satisfies the conditions indicated in the important sentence determination knowledge is sequentially checked according to the determination conditions of each knowledge, and the importance value for each sentence is determined. A summary sentence is created by concatenating them in the order in which they appear.

According to the present device configured in this way, in addition to the various methods of determining important sentences that have been proposed in the past, it also focuses on character modification information added to the characters that make up a sentence, and determines which characters are modified. Since the important sentences are extracted from the document while recognizing the sentences containing the characters in the text as sentences with high importance, it becomes possible to select and extract sentences with high importance easily and effectively. Moreover, since the importance of each sentence is recognized and judged while weighting the importance according to the knowledge information, sentences with high importance can be accurately selected and extracted.

As a result, it becomes possible to effectively and automatically create a highly accurate summary sentence that is rich in content.

Note that the present invention is not limited to the embodiments described above. For example, it goes without saying that the various techniques for determining important sentences that have been proposed in the past may be appropriately used in conjunction with each other, and the important sentences may be extracted by comprehensively evaluating them.

Further, the knowledge information used to recognize and determine important sentences is not limited to the above-mentioned example, and its conditions and contents may be determined according to the specifications. In addition, various modifications can be made to the internal representation format of the document to be summarized, and in other words, various modifications can be made without departing from the gist of the document.

[Effects of the invention] Figure 4 shows an example of sentence analysis results, Figure 5 shows an example of the expression format of importance knowledge used for determining important sentences, and Figure 6 shows important sentence determination. FIG. 7, which is a diagram showing an example of processing, is a diagram showing the flow of important sentence determination processing.

1...Inside literary figures, 2...Sentence analysis section, 3...Important sentence judgment section, 5...Knowledge storage section, 6...Knowledge information regarding character modification information, 6...Summary sentence creation Department.

Since the degree of importance of the sentence is recognized by using the system, the recognition performance of the degree of importance of the sentence can be greatly improved. As a result, great practical effects can be achieved, such as making it possible to effectively create a high-quality summary with rich content.

[Brief explanation of drawings]

Claims (2)

[Claims] (1) A sentence analysis means for analyzing the sentences that constitute the document to be summarized, a knowledge storage means for storing knowledge for recognizing important parts in the document, and an analysis result by the sentence analysis means and the knowledge. a determining means for recognizing and determining important sentences in the document to be summarized using the knowledge stored in the storage means; 1. A document summarizing device comprising: a summary sentence creation means for creating a summary sentence. (2) The document summarizing device according to claim (1), wherein the knowledge for recognizing important parts in a document consists of character modification information.