JPH02112069A - Automatic summarizing system - Google Patents

Abstract

PURPOSE:To extract a summarized sentence only with a word dividing effect by providing a means to accumulate a specified phrase, which already appears, in a natural word processing system equipped with a dictionary and a word dividing means. CONSTITUTION:A central processing unit 2 executes all processing from the reading of a Japanese sentence to the output of a result. A storage device 4 to accumulate information, which are needed for the processing, is equipped with a dictionary data base, a word divided result data base, a document data base and a key word data base as a storing area. A condition for a key word or a key sentence is integrated into the natural word processing system in advance. While the system reads a sentence to be processed one by one, the system decides whether the sentence includes the key word or goes to be the key sentence or not. Based on such a result, only the partial sentence is extracted and the summarized sentence is automatically made. Thus, the substance of the sentence can be automatically extracted and the substance of the sentence can be grasped in a short time.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention relates to a method for automatically extracting a summary sentence from a text. For example, a machine translation system automatically extracts a summary sentence to be translated. Regarding the extraction method.

[Prior Art] As a conventional method for automatically obtaining the gist of two sentences, for example, as shown in Japanese Patent Laid-Open No. 117658/1983, keywords are automatically extracted based on the frequency of words appearing in the sentences. There is a method of determining the most important sentence from each paragraph based on the keyword and using it as a summary sentence.

On the other hand, in conventional machine translation systems, all sentences in a text are subject to translation processing, and when a person wants to specify a sentence to be translated, a human has to give an instruction. In addition, from the perspective of improving the efficiency of machine translation processing, for example,
As described in Japanese Patent No. 1-255468, there is a method in which an input sentence is translated into a partial translation or word translation after a predetermined period of time has elapsed. However, there was no way to automatically extract and translate a portion of the five sentences.

[Problem to be solved by the invention]

However, with the above-mentioned summarization method, it is difficult to decide how frequently words should appear as keywords.
Furthermore, it is possible that one sentence per paragraph may not be enough to grasp the meaning of the text.

The purpose of the present invention is to provide a new method for extracting summary sentences, and to extract enough summary sentences to understand the meaning of sentences using only the word segmentation effect without performing syntactic or semantic analysis. . It is also possible to apply this to machine translation to make translation processing more efficient.

(Means for solving iM) In the present invention, conditions for becoming keywords and key sentences are incorporated into the natural language processing system in advance.The system reads each sentence to be processed one by one, and then identifies the sentences as keywords. It determines whether it contains a key sentence or whether it is a key sentence.Based on this result, only a part of the sentence is output and a summary sentence is automatically created.

[Effect]

The automatically generated summary document has fewer sentences than the original, so
You can understand the contents of a document in a short time.

Also, since the same words appear many times, it is possible to create an easy-to-read summary document.

〔Example〕

Hereinafter, two examples in which the present invention is applied to a Japanese document processing system will be described in detail with reference to the drawings.

FIG. 2 is a hardware configuration diagram of the Japanese document processing system. 1 is a device for inputting Japanese sentences.

Reference numeral 2 is a central processing unit of the system, and 1 performs all processing from reading a Japanese sentence to outputting the result. Reference numeral 3 is a device that outputs the result. Reference numeral 4 denotes a storage device for storing information necessary for processing, and here it is assumed that its storage area includes a dictionary database, a word division result database, a document database, and a keyword database.

FIG. 3 is an example of records of a word division result database that stores word division results. The record is composed of word spelling 3019, part of speech 302, and dictionary type 303.

FIG. 4 is an example of records of a document database that stores documents. The record is sentence number 401° sentence 402. It consists of a dependent sentence number 403 and a summary sentence flag 404.

FIG. 5 is an example of a record in a keyword database that stores important words that have already appeared. The record is composed of a sentence number 501 and a word 502.

Next, a processing method of the present invention will be explained according to the flowchart shown in FIG. First, set the initial value 1 to the first sentence number SN and the record number M of the keyword database 101), read the input Japanese sentence from the input device 1, and register the sentence number and sentence in the document database 102)
, word division is performed (103). Many methods of word segmentation are already known, for example,
The method shown in No. 169262 may be used. The word segmentation results are stored in a pre-cleared word segmentation result database as word spelling 301, part of speech 302, and dictionary type 303.

After reading the Japanese sentence, the values of the record number N of the word division result database and the final record number J of the keyword database are set (104).

Read the contents of the word segmentation result database in order (10
5).

Next, the dependent sentence of the input Japanese sentence is identified by determining whether or not the read word includes a word that has already appeared. The details are described below. Here, already mentioned words are limited to nouns. First, it is determined whether the part of speech 302 of the read word is a noun or not (106), and if it is not a noun, move to the next record in the word segmentation result database.6) If it is a noun, the record number M in the keyword database is O. Determine whether or not. If M is not O, since an unprocessed word exists in the keyword database, it is determined whether the word in the word segmentation result database matches the Mth record in the keyword database (108).

If they do not match, the record number M of the keyword database is decreased by 1 (109), and the same process is repeated. If they match, the sentence number of the Mth record in the keyword database is compared with the dependent sentence number of the SNth record in the document database (110). At this time, if the sentence number is blank, it is assumed to be O. If the sentence number of the Mth record in the keyword database is larger, the dependent sentence number of the SNth record in the document database is changed to the sentence number of the Mth record in the keyword database (111). In other cases, the 5-document database is not updated, and the process moves to the next word in the word segmentation result database.

If the record number M of the keyword database is O, the read word does not match any word in the keyword database, that is, it is a new word, and the process moves to the next word I\ in the word division result database.

When moving to the next word in the word segmentation result database, add 1 to the record number N in the word segmentation result database (1
12) Check if the word exists in the record11
3) If it exists, the process of reading it and determining whether it matches the word in the keyword database is repeated. If no word exists in the Nth record of the word segmentation result database, all words in the word segmentation result database have been processed.

Next, it is checked whether the dependent statement number is registered in the SNth record of the document database (114). If it is not registered, it means that the dependent statement number could not be specified, and the default value is ''. 5N-1' is registered in the dependent bill number column of the document database (115).

Next, the words in the word segmentation result database are added to the keyword database. The details will be described below. first,
Set the values of the record number N of the word division result database and the record number M of the keyword database (11
6). Next, the words in the word division result database are read in order (117), and it is determined whether or not they are nouns (118). If it is a noun, the word and sentence number are registered in the keyword database (119). Anything other than a noun will not be registered.

Then, set the record number of the word segmentation result database to 1.
It advances (120), and it is determined whether or not there are any unprocessed words left (121). If there are, the record number of the keyword database is incremented by one (122).

The same process is repeated for words in the word segmentation result database. After processing all the words in the word segmentation result database, it is determined whether the next Japanese sentence exists or not (123).
If there is, the sentence number is incremented by 1 (124) and the same process is repeated for the next Japanese sentence.

After processing all input Japanese sentences, select and output sentences to be extracted as summary sentences. Here, we will extract and output dependent statements in order from the last statement. First, a flag is set to output the SNth sentence of the document database (125).

Next, the statement number on which the SNth statement depends is assigned to SN (126), and it is determined whether SN is 0 (127).

If it is not 0, continue flagging the dependent statement. When SN becomes 0, it means that all sentences to be output have been selected, and the sentences are output in order starting from the first sentence.

1 is sequentially added to the sentence number SN (128), and it is determined whether there is a Japanese sentence (129). If there is Japanese text.

Determine whether the flag is set 130), if the flag is set, output the Japanese text 131), if the flag is not set, do not output it, then add 1 to the statement number SN and continue the same process. . Once the last Japanese sentence has been output, the entire summary sentence generation process is complete.

As an example, a method of summarizing documents written in the document database of FIG. 4 will be shown. The dependent bill number obtained by the method described above is written in 403. What is shown in FIG. 6 is a tree structure showing the dependency relationships between sentences. In this method,
A series of sentences leading from the first sentence to the last sentence, that is, sentences with numbers 1, 3, 4, 9, and 10 are extracted as summary sentences.

The summary results are shown in Figure 7.

In order to extract some sentences and use them as a summary sentence, it is conceivable to specify a specific keyword and extract only the sentences that include the keyword, without examining the relationship between the sentences. This method is the 8th
The explanation will be given according to the flowchart shown in the figure. first,
Read the input Japanese text from input device 1 (801)
, word segmentation (802), the word segmentation results are cleared in advance and stored in the word segmentation result database as word spelling 301, part of speech 302, . It is stored as the dictionary type 303.

After reading the Japanese sentence, set the record number N of the word division result database to an initial value of 1 (803), and determine whether the read sentence is a title sentence (804).
To determine whether it is a title sentence or not.

For example, if there is no period at the end of the second sentence and there is a line break code immediately after the sentence,
Conditions such as consecutive blank spaces can be used.

If the read sentence is a title sentence, clear the keyword database (805) and set the initial value 1 to the record number M of the keyword database (806).
. Next, the Nth record of the word division result database is read (807), and it is determined whether the word type 302 is a content word (80g).

If it is a content word, first register the full word 301 in the Mth record of the keyword database 809),
The record number M of the keyword database is increased by 1 (810). Next, the record number N of the word division result database is incremented by 1 (810). If the Nth code in the word division result database is not a content word, do nothing and increment N by 1 (s i o). Then, it is determined whether there are any unprocessed words in the Nth record of the word division result database (812). If there are any unprocessed words, the same process is repeated. If there are no unprocessed words, it means that all words in the word division result database have been processed, the title sentence is output (820), and the process moves to the next sentence.

If the read Japanese sentence is not a title sentence, read the Nth record of the word segmentation result database (81
3) Set the record number M of the keyword database to an initial value of 1 (814). Then, it is checked whether the word in the Mth record of the keyword database matches the word in the Nth record of the word division result database (815).

If they match, it means that the keyword is included in the read Japanese text, and the input text is considered to be a key text, and this text is output to the output device 3 (820). ,
Move on to the next sentence. If there is no match, 1 is added to the record number M of the keyword database (816), and it is determined whether there are any words left in the Mth record of the keyword database (817). It is checked whether the word of the th record matches the word of the Nth record of the word segmentation result database (815). If there is no word left, the record number N of the word segmentation result database is increased by 1 818
), it is determined whether there is still an unprocessed word in the Nth record of the word segmentation result database (819). If there is an unprocessed word, the N#rth record of the word division result database is read and the same process is repeated. If there are no unprocessed words, it means that the words in the word segmentation result database do not contain the keyword, and the input Japanese sentence is considered not to be important, and the process moves to the next sentence without outputting this sentence. .

After moving on to the next sentence, it is first checked whether there is a next sentence (821); if there is no next sentence, all sentences have been processed. If there is a next sentence, the first sentence is read again (801) and the same process is repeated until the sentence ends.

Keywords can be determined not only by using content words in the title sentence, but also by using content words that appear in the first or last sentence, underlined words, bold words, etc.
It is also possible to use words in different fonts, such as Gothic or italic. in this case.

The algorithm other than the part for determining keywords is the same as in FIG. 8.

If the dictionary built into the system is separated into a basic language dictionary and a specialized language dictionary, the following methods are used to extract summary sentences:
The following may also be considered.

This method will be explained according to the flowchart shown in FIG. First, an input Japanese sentence is read from Kugata device 1 (901), the read sentence is divided into words (902),
The word segmentation results are stored in the word segmentation result database as full word 301 and part of speech 3o2° dictionary type 303. Next, the words in the word division result database are read in order (903), and it is checked based on the dictionary type 303 whether the read word is a word described in the technical language dictionary (903).
904), if the word is described in the technical language dictionary, the input Japanese sentence is considered to be an important sentence, and this sentence is sent to the output device 3.
(905) and moves on to the next sentence. If the word in the read word segmentation result database is not a word described in the technical language dictionary, it is determined whether all the words in the word segmentation result database have been processed (906), and if there are unprocessed words,
Read the next word from the word division result database (9
03).

Continue the same process. Once all the words in the word segmentation result database have been processed, the input Japanese sentence is considered not to be an important sentence, and the process moves to the next sentence without outputting this sentence. First, it is checked whether the next sentence exists or not (907).

If there is no next sentence, it means that all sentences have been processed, and if there is a 0th sentence to finish the summary sentence extraction process, read the 1st sentence again (901) and perform the same process at the end of the sentence. Repeat until.

If the system has a user dictionary, the user dictionary can be used instead of the technical language dictionary, and the algorithm shown in FIG. 9 can also be used.

There is also a crude automatic summarization method that extracts only the first and last paragraphs of a single sentence. Although this method cannot be said to be very effective as an automatic summary, it is considered to be effective when combined with a machine translation system.

The automatic summarization method described above can be applied to determine whether a certain document is important or not. From multiple documents,
An example of a method for selecting and displaying important documents will be explained with reference to the flowchart shown in FIG.

First, read one document (1001), read the sentences in the document in order (1002), then judge whether the read sentence becomes a summary sentence (10()3), and if it becomes a summary sentence, , adds 1 to the number of summary sentences (1004).

Add 1 to the number of full sentences (1005); if the read sentence does not become a summary sentence, add 1 only to the number of full sentences (1005)
), then it is determined whether all the sentences in one document have been read (1006), and if not.

Read the next sentence (1002), if finished, determine whether (number of summary sentences)/(number of full texts) exceeds a certain value (1007), and if it does, output the document. (
1008), if the limit is not exceeded, nothing is output, and it is determined whether there is a next document (1009), if there is a next document, it is continued (1001), otherwise all Finish processing.

〔Effect of the invention〕

According to the present invention, the gist of a sentence can be automatically extracted, and the gist of the sentence can be grasped in a short time. Furthermore, when one method is applied to machine translation, the translation process can be made more efficient.

[Brief explanation of the drawing]

FIG. 1 is a flowchart showing one embodiment of the present invention, FIG. 2 is a hardware configuration diagram of a Japanese document processing system according to the present invention, FIG. 3 is an example of records of a word segmentation result database, and FIG. An example of a document database record, Figure 5 is an example of a keyword database record, Figure 6 is an example of a tree structure showing a sentence dependency structure, Figure 7 is an example of a summary result, Figures 8 and 9. and FIG. 10 are flowcharts showing a modification of the embodiment. 1... Input device, 2... Central processing unit, 3... Output device, 4... Storage device. Figure D

Claims (1)

[Claims] 1. In a natural language processing system having a dictionary and a word segmentation means, an automatic system characterized in that a specific sentence is automatically extracted by providing a means for accumulating specific already-used phrases. Summary method. 2. The automatic summarization method according to claim 1, wherein the extraction means analyzes connections between sentences in the document. 3. In a natural language processing system that has a dictionary and word segmentation means, it is possible to automatically determine which sentences should be extracted by checking whether each sentence in a document contains a specific word. Features an automatic summary method. 4. The automatic summary system according to claim 3, wherein the extraction means extracts sentences including words appearing in the title. 5. The automatic summarization system according to claim 3, wherein the extraction means extracts sentences including underlined words or words with different fonts. 6. A patent characterized in that, when the dictionary of the natural language processing system is separated into a technical term dictionary and a basic term dictionary, the extraction means extracts sentences that include words registered in the technical term dictionary. Automatic summarization method according to claim 3. 7. Claim 3, characterized in that when the dictionary of the natural language processing system includes a user dictionary, the extraction means extracts sentences that include words registered in the user dictionary. Automatic summary method for section descriptions. 8. In a natural language processing system having means for determining titles and paragraphs, an automatic system is characterized in that it automatically determines sentences to be extracted by examining the positional relationship of each sentence in a document. Summary method. 9. The automatic summary system according to claim 8, wherein the extraction means extracts a title, a first paragraph, and a last paragraph. 10. The automatic summary system according to claim 8, wherein the extraction means extracts a title and a first sentence and a last sentence in each paragraph. 11. A machine translation method characterized in that a summary sentence created by the methods described in claims 1, 3, and 8 is to be translated. 12. Important documents are recognized using the ratio between the number of summary sentences extracted by the methods described in claims 1, 3, and 8 and the number of full sentences in the document. Natural language processing system.