JP6902945B2 - Text summarization system - Google Patents

Description

The present invention relates to a text sentence summarization technique, and is particularly applied to a text summarization system that creates a summarization based on a learning model generated by machine learning.

A technique for automatically generating a summary of text sentences using machine learning is being studied. The methods for generating summaries are roughly classified into extraction type and generation type. In the extraction type, for example, words and sentences with high importance included in the sentence to be summarized are extracted, and a summary is created by combining these. On the other hand, in the generation type, for example, a sentence is converted into a predetermined intermediate representation, and a summary is created based on this using a technique of natural language generation. At present, in general, extraction-type methods have been widely studied, and various methods for improving accuracy have been proposed.

For example, Japanese Patent Application Laid-Open No. 2016-186772 (Patent Document 1) expresses a tree structure in which components included in a sentence for which a summary (abbreviated sentence) is created are connected based on a grammatical or conceptual relationship. At the same time, by adding the degree of coupling obtained as the degree of remaining in the abbreviated sentence to each connection between the components, and extracting the components to be included in the abbreviated sentence based on the degree of coupling, the abbreviated sentence is created. , It is stated that a natural abbreviation is generated.

In the extraction type method, the words and sentences contained in the abstract are, in principle, restricted to the words and sentences contained in the original sentence. Therefore, shortening the sentence may result in unnatural expressions or grammatically inappropriate summaries. On the other hand, although it is possible to use natural expressions for the generation-type method, there are problems such as the need for a highly accurate natural language generation technique. However, in recent years, for example, research as described in Non-Patent Document 1 has been conducted, and programs, libraries, and the like have become generally available.

Japanese Unexamined Patent Publication No. 2016-186772

"Research Blog: Text summarization with TensorFlow (registered trademark)", [online], August 24, 2016, Google (registered trademark, the same applies hereinafter), [Search on June 27, 2017], Internet <URL: https //research.googleblog.com/2016/08/text-summarization-with-tensorflow.html ＞

Regardless of the extraction type or the generation type, the conventional automatic summarization technique for text sentences can automatically generate a summary with a certain degree of accuracy by using a learning model generated by machine learning. However, there are many cases where the accuracy is not sufficient. For example, among the words contained in the original text, there are cases where some important words that you want to appear in the abstract do not appear in the abstract. There is a lot of room for improvement.

Therefore, an object of the present invention is to provide a text summarization system capable of further improving the accuracy of automatic summarization of text sentences.

The above and other objects and novel features of the present invention will become apparent from the description and accompanying drawings herein.

A brief description of typical inventions disclosed in the present application is as follows.

The text summarization system according to a typical embodiment of the present invention is a text summarization system that creates a summary from a text sentence, and performs predetermined preprocessing on a plurality of learning texts including the original text and the summary. A pre-processing unit for learning that creates training data, a learning model creation unit that creates a learning model related to summarization by machine learning based on the training data, and a pre-processing that performs predetermined preprocessing on the text to be summarized. A summary creation unit that creates a summary text based on the learning model for the unit and the summary target text that has been preprocessed by the preprocessing unit, and a predetermined post-processing for the summary text. It has a post-processing unit that outputs data.

Then, in the predetermined pre-processing in the learning pre-processing unit and the pre-processing unit, predetermined processing is performed on the words and phrases included in the learning text and the summary target text, and the predetermined post-processing in the post-processing unit is performed. In the processing, the content of the predetermined processing performed by the preprocessing unit is restored with respect to the summary text.

Among the inventions disclosed in the present application, the effects obtained by representative ones will be briefly described as follows.

That is, according to a typical embodiment of the present invention, it is possible to further improve the accuracy of automatic summarization of text sentences.

It is a figure which showed the outline about the structural example of the text summarizing system which is one Embodiment of this invention. It is a flowchart which showed the outline about the example of the flow of the pre-learning process in one Embodiment of this invention. It is a flowchart which showed the outline about the example of the flow of the summary making process in one Embodiment of this invention. It is a figure which showed the outline about the example of the pretreatment and the posttreatment in one Embodiment of this invention. It is a figure which showed the outline about the other example of the pretreatment and the posttreatment in one embodiment of the present invention. It is a figure which showed the outline about the other example of the pretreatment and the posttreatment in one embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In addition, in all the drawings for explaining the embodiment, in principle, the same reference numerals are given to the same parts, and the repeated description thereof will be omitted. On the other hand, the parts described with reference numerals in one figure may be referred to with the same reference numerals in the explanation of other figures, although they are not shown again.

<System configuration>
FIG. 1 is a diagram showing an outline of a configuration example of a text summarizing system according to an embodiment of the present invention. The text summarization system 1 of the present embodiment is a server system having a function of automatically generating and outputting a summary of text sentences such as news articles based on a learning model generated by machine learning.

The text summarization system 1 is composed of, for example, a server device, a virtual server built on a cloud computing service, or the like. Then, a CPU (Central Processing Unit) (not shown) is used to develop middleware such as an OS (Operating System), a DBMS (DataBase Management System), and a Web server program developed on a memory from a recording device such as an HDD (Hard Disk Drive), and the software thereof. By executing the software running on the above, various functions related to automatic summarization, which will be described later, are realized.

The text summarization system 1 includes, for example, a learning pre-processing unit 11, a learning model creation unit 14, a pre-processing unit 16, a summarization creation unit 18, a post-processing unit 19, and the like implemented as software. In addition, it has each data such as learning data 12, vocabulary data 13, learning model 15, and replacement data 17 implemented as a database, a file, or the like. The learning preprocessing unit 11 and the learning model creation unit 14 have a function of generating a learning model 15 in advance by machine learning based on a learning text 2 which is teacher data in machine learning. Further, the pre-processing unit 16, the summary creation unit 18, and the post-processing unit 19 have a function of generating and outputting the summary text 4 based on the learning model 15 for the summary target text 3 to be the target of the summary creation.

The learning text 2 which is the teacher data includes the original text to be summarized and the summary result which is the correct answer. Examples of such sentences include news articles. In this case, the title, headline, etc. of the news can be used as the correct summary result for the original news text to be summarized. In the present embodiment, the news article is described as the learning text 2 and the summary target text 3, but the present invention is not limited to this and can be applied to various sentences.

The learning preprocessing unit 11 performs various preprocessing on the learning text 2 to be input, and prepares the normalized learning data 12 (and vocabulary data 13) for input to the machine learning engine. Has a function to do. The specific content of the preprocessing will be described later. The learning model creation unit 14 receives the learning data 12 and the vocabulary data 13 created by the learning preprocessing unit 11 as inputs, performs machine learning using a predetermined algorithm of natural language processing, and learns for summarization generation. It has a function of generating a model 15.

Known techniques can be appropriately used for machine learning engines and natural language processing algorithms. In this embodiment, referring to the technique described in Non-Patent Document 1 described above, for example, as a machine learning engine, Tensorflow (registered trademark, the same applies hereinafter), which is an open source machine learning library provided by Google Inc., is used. Use. Further, as an algorithm for natural language processing (automatic sentence summarization) used on this, a Textsum program provided as open source is used. This makes it possible to apply technologies such as deep learning, RNN (Recurrent Neural Network), RSTM (Long Short-Term Memory), Sequence to Sequence model, Sequence to Sequence with attention model, etc. at the time of automatic summarization. It becomes.

The preprocessing unit 16 performs various preprocessing similar to a part of the above-mentioned learning preprocessing unit 11 on the summarization target text 3 to be summarized, and is normal for applying the learning model 15. Has the function of becoming. The specific contents of this preprocessing will be described later, but at this time, the predetermined words and the like are replaced with predetermined words and symbols, and the information related to the results and contents is used as the replacement data 17. Recorded.

The summary creation unit 18 has a function of applying the learning model 15 by the machine learning engine to the summary target text 3 normalized by the preprocessing unit 16 to create a summary. At this time, the vocabulary data 13 created by the learning preprocessing unit 11 is also used. As the machine learning engine, for example, Tensorflow is used in the same manner as described above. The summary here is created in a state including words, symbols, and the like related to the replacement result performed by the preprocessing unit 16.

Based on the replacement data 17 recorded in the preprocessing unit 16, the post-processing unit 19 replaces / restores words, symbols, etc. related to the replacement result in the summary created by the summary creation unit 18 with the original words, etc. , It has a function of shaping the appearance of a sentence as needed and outputting it as a summary text 4. The post-processing unit 19 replaces / restores the replacement result for the summary target text 3 by the pre-processing unit 16 with the original word or the like and outputs the summary text 4. In order to verify the summary result, the post-processing unit 19 outputs the summary text 4. The same post-processing may be performed in the learning model creation unit 14 described above to replace / restore the replacement result of the learning text 2 by the learning pre-processing unit 11 with the original word or the like.

<Processing flow (learning process)>
FIG. 2 is a flowchart showing an outline of an example of the flow of the prior learning process in the present embodiment. In the learning process, first, the learning preprocessing unit 11 reads the learning text 2 such as a news article, performs morphological analysis on all articles, and performs part-speech decomposition (S01). The morphological analysis can be carried out by appropriately using various programs and libraries including generally available ones such as ChaSen and MeCab. The method of retaining the information of words and phrases divided by part-speech decomposition is not particularly limited, but for example, in the learning text 2, a space is inserted at the delimiter of the target word or phrase in the learning text 2. It may be reflected.

Next, the learning text 2 divided into words and the like is subjected to preprocessing such as various replacement processes for improving the accuracy of the learning model 15 (S02). The content of the preprocessing will be described later, but for example, the description of the numerical value included in the learning text 2 is replaced with a symbol such as "#" in consideration of the number of digits. Further, a dictionary database or the like (not shown) may be used to replace the synonyms so as to unify the notation. In addition, uppercase and lowercase letters and full-width and half-width characters may be converted and replaced so as to be unified. By normalization by unifying the notation in this way, words and the like having the same content can be aggregated in the learning text 2, and the accuracy as a sample can be improved. Not limited to the replacement of words or the like, a label indicating information such as importance may be added to each word or the like.

After that, each sentence included in the normalized learning text 2 is subjected to a data cleansing process for excluding inappropriate sentences such as noise based on a predetermined selection criterion (S03). For example, when the learning text 2 is a news article, the title of the article corresponds to the correct summary text for the article body which is the sentence to be summarized, but the noun included in this title is included in the article body. If all of them are not, it can be judged that the title is not appropriate for the article body. Therefore, for example, only sentences in which a certain percentage (for example, 80%) or more of nouns included in the title are included in the article body may be selected and used as the learning text 2. Regarding the learning text 2 remaining by the data cleansing, this is recorded as the learning data 12 (S04).

At this time, based on the learning data 12, vocabulary data 13 that holds various information such as metadata related to each word or the like included in the learning data 12 is also generated (S05). When Tensorflow is used as the machine learning engine and the Textsum program is used as the algorithm for automatic sentence summarization, this vocabulary data 13 is created as a "vocab" file.

Then, using the learning data 12 and the vocabulary data 13 obtained in steps S04 and S05 as inputs, the learning model creation unit 14 performs machine learning by the machine learning engine based on predetermined setting conditions to create the learning model 15. (S07), the learning process is terminated. As a predetermined setting condition, for example, it is possible to set a condition such that only a few sentences at the beginning of each sentence of the learning data 12 are to be summarized.

<Process flow (summary creation process)>
FIG. 3 is a flowchart showing an outline of an example of the flow of the summary creation process in the present embodiment. In the summary creation process, first, the preprocessing unit 16 reads the summary target text 3 such as a news article, performs morphological analysis, and performs part-speech decomposition (S11). This process is the same as the process of step S01 in the learning process of FIG. Next, the summary target text 3 divided into words and the like is subjected to the same preprocessing as the process of step S02 in the learning process of FIG. 2 (S12). At this time, when the word or the like included in the summary target text 3 is replaced, the content of the replaced word or the like and information such as the appearance order, position, number of digits, and unit in the sentence are replaced with the replacement data 17. Record as.

After that, the summarization target text 3 normalized by the preprocessing is input, and the summarization creation unit 18 applies the learning model 15 created by the learning process of FIG. 2 to create a summarization by the machine learning engine. (S13). At this time, the vocabulary data 13 generated in step S05 of FIG. 2 described above is also used. As with the learning process, Tensorflow can be used as the machine learning engine. After that, the content of normalization such as replacement and label addition in the created summary is replaced / restored with the original word or the like by referring to the replacement data 17 by the post-processing unit 19, and a sentence is required. The post-processing for shaping the appearance of the above is performed, the output is output as the summary text 4 (S14), and the summary creation process is completed.

<Example of pre-processing / post-processing>
FIG. 4 is a diagram showing an outline of an example of pretreatment and posttreatment in the present embodiment. Here, an example in which various numerical values are included in the learning text 2 and the summary target text 3 is shown. Texts such as news articles may contain many types of numerical values with different values and digits, but if all of these terms are treated as different terms, samples will diverge in the learning text 2 and become enormous. Since the number of vocabulary data 13 increases, the accuracy of the learning model 15 decreases and the load of the learning process also increases.

Therefore, in the present embodiment, the numerical value is replaced with a symbol such as "#" in the preprocessing (step S02 in FIG. 2 and step S12 in FIG. 3) at the time of learning and at the time of summarizing. For example, the uppermost sentence of FIG. 4 (divided learning text 2 and summary target text 3) contains words and phrases including the numerical values of "December" and "0.3%". Replace each of these numerical parts with "#" such as "## month" and "#. #%" As in the second sentence from the top of FIG.

In the processes of creating the learning model (step S07 in FIG. 2) and creating the summary (step S13 in FIG. 3), each of the sentences in the state replaced by "#" is processed to create the learning model 15. And create a summary. At this time, the words "## month" and "#. #%" Remain in the abstract sentence, as in the sentence in the third row from the top of FIG.

In the present embodiment, these words and phrases are returned to the original words and the like in the post-processing at the time of summarization (step S14 in FIG. 3). Therefore, when a numerical value is replaced with a symbol such as "#" in the preprocessing at the time of summarization (step S12 in FIG. 3), the replaced numerical value and the appearance position, the number of digits, the unit, etc. in the sentence are restored. Information is recorded in the replacement data 17, and this is referred to at the time of post-processing. As a result, it is possible to create a summary text 4 including the original numerical values as in the lowermost sentence of FIG.

FIG. 5 is a diagram showing an outline of other examples of pretreatment and posttreatment in the present embodiment. Here, an example in which importance information is added as a label to words and the like included in the learning text 2 and the summary target text 3 is shown. Each word, etc. contained in a news article, etc. has a different importance from the viewpoint of creating a summary. Therefore, in the present embodiment, the importance is taken in the pre-processing (step S02 of FIG. 2 and step S12 of FIG. 3) at the time of learning and at the time of summarization so that the importance of each word or the like can be considered in the summarization. A label indicating the above information is added to each word or the like to form an integrated word or the like, and machine learning processing is performed on this as a target.

For example, the importance of the uppermost sentence in FIG. 5 (the learning text 2 and the summary target text 3 as a result of the numerical replacement described above) is as shown in the second sentence from the top in FIG. A symbol of "_X" (X = 0 to 2) is added to the end of each word or the like as a label indicating. In the example of FIG. 5, for example, the word "European Central Bank" is labeled with "_0" of importance X = 0 and converted into the phrase "European Central Bank_0". Similarly, for example, the word "euro" is labeled with "_2" of importance X = 2 and converted to "euro_2". In the example of FIG. 5, X indicating the importance is divided into three types such as 0 (not important) <1 (normal) <2 (important), but the present invention is not limited to this. , Other values and classification methods may be used.

The importance of each word or the like can be used by calculating a TF-IDF (Term Frequency --Inverse Document Frequency) value that is generally used as an index indicating the importance of a word in a sentence. For example, the calculated TF-IDF value is divided into predetermined ranges, and the importance of 0 to 2 is set.

In the processes of learning model creation (step S07 in FIG. 2) and summary creation (step S13 in FIG. 3), each process is performed on a sentence in a state in which a label indicating the importance of a word or the like is attached, that is. , The learning model 15 is created and the summary is created in consideration of the importance of each word and the like. At this time, words and phrases with labels such as "_0" and "_2" remain in the summary sentence, as in the sentence in the third row from the top of FIG.

In the present embodiment, these words and phrases are returned to the original words and the like in the post-processing at the time of summarization (step S14 in FIG. 3). That is, all the label portions indicating the importance such as "_0" and "_2" are deleted from each word or the like in the sentence. As a result, it is possible to create a summary text 4 using the original word or the like as in the sentence at the bottom of FIG.

FIG. 6 is a diagram showing an outline of other examples of pretreatment and posttreatment in the present embodiment. Here, an example is shown in the case where the learning text 2 and the summary target text 3 include nouns including various proper nouns. In sentences such as news articles, there may be a plurality of different words but the same part of speech. For example, the sentence "Mr. Nomura wants to live in France rather than the United States" contains the proper nouns "Nomura," "America," and "France." Here, "Nomura" is a person's name, but "America" and "France" are both country names. If all of these words and phrases are treated as different words and phrases, the number of samples diverges in the learning text 2 and becomes enormous, as in the case where a numerical value is included in the sentence, the accuracy of the learning model 15 is lowered, and the load of the learning process is reduced. Will also increase.

Therefore, in the present embodiment, nouns (particularly proper nouns) are replaced with words and phrases indicating part-speech information in the preprocessing during learning and summarization (step S02 in FIG. 2 and step S12 in FIG. 3). That is, in the above example, for example, "Nomura" is "_proprietary noun person's first and last name * 1", "America" is "_proprietary noun region country * 1", and "France" is "_proprietary noun region country * 2". Replace each as follows. Thus, for example, "America" and "France" are different words, but they can be treated as the same part of speech ("proper noun region country").

In this embodiment, even if the "country" such as "America" or "France" is different, it is treated as the same proper noun at the "country" level, but the unit is not limited to this and is treated in the same way. -The level can be set as appropriate. For example, it may be treated as the same proper noun at the "region" level, "city" level, etc., but it is desirable to stop the subdivision at an appropriate level because the accuracy of summarization will decrease if the subdivision is excessive. ..

In the example of FIG. 6, for example, the uppermost sentences (learning text 2 and summary target text 3 as a result of the above numerical substitution) include "Ministry of Finance", "Trade Statistics", and "Trade Surplus". Contains words and phrases of proper nouns such as. Each of these proper noun parts is part of speech such as "_proprietary noun organization * 1", "_proprietary noun general * 1", and "_proper noun general * 2" as shown in the second sentence from the top of FIG. Replace with a phrase that indicates information. Here, in addition to part-speech information such as "proper noun organization" and "general nouns", information on the order and position of appearance such as "* 1" and "* 2" is also included.

In the processes of creating the learning model (step S07 in FIG. 2) and creating the summary (step S13 in FIG. 3), the sentences in the state of being replaced by the words indicating the part-of-speech information are processed, respectively, and the learning model 15 Create and create a summary. At this time, words such as "_proprietary noun organization * 1" and "_proprietary noun general * 1" remain in the summary sentence, as in the sentence in the third row from the top of FIG.

In the present embodiment, these words and phrases are returned to the original words and the like in the post-processing at the time of summarization (step S14 in FIG. 3). Therefore, when the proper noun or the like is replaced with a symbol such as "_proprietary noun organization * 1" in the preprocessing at the time of summarization (step S12 in FIG. 3), the content of the replaced proper noun and the appearance position in the sentence. Information for restoration such as is recorded in the replacement data 17, and this is referred to at the time of post-processing. As a result, it is possible to create a summary text 4 including the original proper noun and the like as in the sentence at the bottom of FIG.

As described above, according to the text summarization system 1 according to the embodiment of the present invention, the learning text 2 and the text to be summarized 3 before the machine learning is subjected to numerical values at the time of preprocessing. Is replaced with a predetermined symbol, a label indicating the importance is added to each word, etc., a proper noun, etc. is replaced with a predetermined phrase, etc., and the processed sentences are learned. Processes the creation of model 15 and the creation of summaries. Then, the final summary text 4 is obtained by returning the processed contents (replaced / added symbols and phrases) included in the created summary to the original words and the like. As a result, it is possible to avoid the divergence of the learning data 12 as a sample and to generate the learning model 15 in consideration of the importance of the words, thereby improving the accuracy in creating the summary.

The present inventors target news articles as learning texts 2 and abstract texts 3, and after actually creating abstract texts 4, Rouge (Rouge), which is widely used as an evaluation index for automatic summarization of newspaper articles and the like. Recall-Oriented Understudy for Gisting Evaluation) -1 was calculated and compared. According to this, while the average value of Rogue-1 when the above-mentioned normalization is not performed in the pretreatment is about 0.3 to 0.4, the above-mentioned normalization is performed individually or. It was found that the average value of Rouge-1 can be improved to about 0.6 to 0.7 at the maximum by applying one or more in combination.

Although the invention made by the present inventor has been specifically described above based on the embodiments, the present invention is not limited to the above embodiments and can be variously modified without departing from the gist thereof. Needless to say. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to the one including all the described configurations. Further, it is possible to add / delete / replace a part of the configuration of the above-described embodiment with another configuration.

Further, each of the above configurations, functions, processing units, processing means and the like may be realized by hardware by designing a part or all of them by, for example, an integrated circuit. Further, each of the above configurations, functions, and the like may be realized by software by the processor interpreting and executing a program that realizes each function. Information such as programs, tables, and files that realize each function can be placed in a memory, a hard disk, a recording device such as an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, or a DVD.

Further, in each of the above figures, the control lines and information lines are shown as necessary for explanation, and not all the control lines and information lines in the implementation are necessarily shown. In practice, it can be considered that almost all configurations are interconnected.

The present invention can be used in a text summarization system that creates summaries based on a learning model generated by machine learning.

1 ... text summarization system, 2 ... learning text, 3 ... summarization target text, 4 ... summarization text,
11 ... Learning preprocessing unit, 12 ... Learning data, 13 ... Vocabulary data, 14 ... Learning model creation unit, 15 ... Learning model, 16 ... Preprocessing unit, 17 ... Replacement data, 18 ... Summary creation unit, 19 ... After Processing unit

Claims (6)

A text summarization system that creates summaries from text sentences.
A learning pre-processing unit that creates learning data by performing predetermined pre-processing on a plurality of learning texts including the original text and its summary.
A learning model creation unit that creates a learning model related to summarization by machine learning based on the learning data,
A pre-processing unit that performs predetermined pre-processing on the text to be summarized,
A summary creation unit that creates a summary text based on the learning model for the summary target text that has been preprocessed by the preprocessing unit.
It has a post-processing unit that performs predetermined post-processing on the summary text and outputs it.
In the pre-processing unit for learning and the predetermined pre-processing in the pre-processing unit, the words and phrases included in the learning text and the summary target text are subjected to predetermined processing.
In the predetermined post-processing in the post-processing unit, the content of the predetermined processing performed by the pre-processing unit is restored with respect to the summary text.
The predetermined processing adds information indicating the importance indicating the frequency or rarity of the phrase to each of the words and phrases contained in the learning text and the summary target text, respectively.
A text summarization system in which the post-processing unit deletes information indicating the importance in the summarization text. In the text summarization system according to claim 1.
The importance is a value based on the TF-IDF value, a text summarization system. In the text summarization system according to claim 2.
The importance is a value obtained by dividing the TF-IDF value into predetermined ranges, and is a text summarization system. In the text summarization system according to claim 1.
The predetermined processing replaces the numerical values included in the learning text and the summary target text with predetermined symbols.
At the time of the replacement, the pretreatment unit records information related to the content of the replacement as replacement data.
The post-processing unit is a text summarization system that replaces the predetermined symbol in the summarization text with the corresponding numerical value based on the content of the replacement data. In the text summarization system according to claim 1.
The predetermined processing replaces a phrase of a predetermined part of speech included in the learning text and the summary target text with a predetermined character string.
At the time of the replacement, the pretreatment unit records information related to the content of the replacement as replacement data.
The post-processing unit is a text summarization system that replaces the predetermined character string in the summary text with a corresponding phrase of the predetermined part of speech based on the content of the replacement data. In the text summarization system according to claim 1.
The learning preprocessing unit is a target for creating the learning data only for words and phrases contained in the summary in the learning text and in which a predetermined ratio or more of the words and phrases are included in the original text in the learning text. Text summarization system.

Images