JP2019133529A - Learning apparatus, generation apparatus, learning method, generation method, learning program, generation program, and model - Google Patents

Abstract

To generate a text with high accuracy.SOLUTION: A learning apparatus includes a generation unit and a learning unit. The generation unit generates a fourth text corresponding to a third text from the third text by use of a model which is generated by learning based on a relationship between a first text and a second text corresponding to the first text. The learning unit executes learning based on a relationship between the third text and the fourth text.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a learning device, a generation device, a learning method, a generation method, a learning program, a generation program, and a model.

  Techniques for generating text on a device are known. For example, a technique for causing a device to generate a summary sentence of a document is known. By causing the device to generate text, it is possible to save the person from generating text.

JP 2005-115628 A JP 2005-174173 A

  In recent years, machine learning has attracted attention. For example, the text is generated by using a learning model (hereinafter simply referred to as a model) obtained by learning a plurality of sets of text (for example, a plurality of sets of texts including a document and its summary sentence) as learning data. It is feasible.

  If a large number of text sets for learning can be prepared, a highly accurate model can be realized. However, since text sets are created manually, the number of text sets that can be prepared is limited. When the number of text sets is small, it is difficult to realize a highly accurate model. When the accuracy of the model is low, the text generated by the device is low in accuracy (eg, accuracy).

  The present application has been made in view of the above, and an object thereof is to enable generation of highly accurate text.

  The learning device according to the present application supports a third text to a third text using a model generated by learning based on the relationship between the first text and the second text corresponding to the first text. A generation unit that generates a fourth text to be performed, and a learning unit that performs learning based on the relationship between the third text and the fourth text.

  According to one aspect of the embodiment, it is possible to generate text with high accuracy.

FIG. 1 is a diagram illustrating an example of processing executed by the information processing apparatus according to the first embodiment. FIG. 2 is a diagram illustrating a terminal device on which content is displayed. FIG. 3 is a diagram illustrating an example of learning data registered in the learning data database. FIG. 4 is a diagram illustrating an example of learning data in which pseudo headings are stored. FIG. 5 is a diagram illustrating an example of a model. FIG. 6 is a diagram illustrating an example of processing executed by the information processing apparatus according to the second embodiment. FIG. 7 is a diagram illustrating a configuration example of the information processing apparatus according to the embodiment. FIG. 8 is a diagram illustrating an example of information registered in the learning data database. FIG. 9 is a diagram illustrating an example of information registered in the model database. FIG. 10 is a diagram illustrating an example of information registered in the content information database. FIG. 11 is a flowchart illustrating an example of the learning process. FIG. 12 is a flowchart illustrating an example of the generation process. FIG. 13 is a diagram illustrating another example of processing executed by the information processing apparatus according to the embodiment. FIG. 14 is a hardware configuration diagram illustrating an example of a computer that implements the functions of the information processing apparatus.

  Hereinafter, a learning device, a generation device, a learning method, a generation method, a learning program, a generation program, and a form for implementing a model (hereinafter referred to as “embodiment”) according to the present application will be described in detail with reference to the drawings. explain. Note that the learning device, the generation device, the learning method, the generation method, the learning program, the generation program, and the model according to the present application are not limited to the following embodiments. In the following embodiments, the same portions are denoted by the same reference numerals, and redundant description is omitted.

[Embodiment 1]
[1-1. Example of information processing apparatus]
First, an example of learning processing executed by an information processing apparatus that is an example of a generation apparatus and a learning apparatus will be described. FIG. 1 is a diagram illustrating an example of processing executed by the information processing apparatus 10 according to the first embodiment. The information processing apparatus 10 can communicate with a data server 20 and a terminal device 30 used by a predetermined client via a predetermined network such as the Internet.

  The information processing apparatus 10 is an apparatus that executes learning processing and generation processing described later. The information processing device 10 is realized by a server device, a cloud system, or the like.

  The data server 20 is an information processing apparatus that manages learning data used when the information processing apparatus 10 executes learning processing described later, and distribution content output when the information processing apparatus 10 executes generation processing described later. . The data server 20 is realized by a server device, a cloud system, or the like. For example, the data server 20 executes a distribution service that distributes news and various contents posted by the user to the terminal device 30. Such a distribution service is realized by, for example, various news distribution sites or SNS (Social Networking Service).

  The terminal device 30 is a smart device such as a smartphone or a tablet, and is a mobile terminal device that can communicate with an arbitrary server device via a wireless communication network such as 3G (3rd Generation) or LTE (Long Term Evolution). is there. The terminal device 30 may be not only a smart device but also an information processing device such as a desktop PC (Personal Computer) or a notebook PC.

[1-2. About distribution of summary content)
Here, when there are a plurality of distribution contents to be distributed, the data server 20 does not distribute all of the contents, but sends the summary contents including the text that is a summary of each content to the terminal device 30. May be delivered. The distribution content is, for example, a news article acquired from an information medium or an information distribution entity. As an example, the summary content is a top page of a portal site with a link to distribution content.

  The information medium is information indicating an information distribution medium or a posting medium, for example, information indicating “newspaper”, “magazine”, or the like. The distribution subject is an organization or an individual that distributes information such as news articles. For example, the distribution subject is a mass media such as a newspaper company, a publisher, a broadcasting station (a television station, a radio station), or the like. Of course, the distribution subject is not limited to mass media. The distribution subject may be a portal site operator or a mobile phone company. The distribution subject may be the operator of the information processing apparatus 10 itself.

  FIG. 2 is a diagram illustrating the terminal device 30 on which content is displayed. A state J1 in FIG. 2 is a diagram illustrating a state in which a top page of the portal site is displayed, and a state J2 in FIG. 2 is a diagram illustrating a state in which a page including the distribution content C11 is displayed. The distribution content C11 is, for example, a news article acquired by an operator of the information processing apparatus 10 (for example, an administrator of a portal site) from a predetermined distribution subject. The terminal device 30 shown in the state J1 displays a page on which a plurality of tiles with links to articles are arranged. In each tile, text indicating the content of the distribution content (hereinafter referred to as a headline) is displayed. For example, the heading U11 is displayed on the top tile of the plurality of tiles arranged in a vertical row. The heading U11 is text corresponding to the content of the distribution content C11. When the user taps the tile on which the heading U11 is displayed, a screen including the distribution content C11 is displayed on the terminal device 30. The distribution content C11 includes an article body K11 and an article title T11.

  Text associated with the distribution content (for example, a headline posted on the top page of the portal site) is created by a person based on the content of the distribution content. For example, the headline U11 is created by reading and summarizing the title T11 included in the distribution content C11 by an employee of the portal site operator. However, it is troublesome to manually generate a headline for each distribution content.

  Thus, it is conceivable to automatically generate a headline from the distribution content using a learning model (hereinafter simply referred to as a model) in which features of various information are learned. For example, it is conceivable to automatically generate a title as a headline from text (for example, a title) included in the distribution content.

  In order to create a headline with high accuracy (for example, a headline with high accuracy), it is necessary to generate a model with high accuracy. In order to generate a model with high accuracy, it is necessary to prepare a large number of text sets as learning data. However, since text sets are created manually, the number of text sets that can be prepared is limited. When the number of text sets is small, it is difficult to realize a highly accurate model. When the accuracy of the model is low, the text generated by the device is low in accuracy (eg, accuracy).

[1-3. (Recursive learning)
Therefore, the information processing apparatus 10 performs learning for generating the model M1 by executing the following learning process. The model M1 is a model for generating a heading text from text (for example, a title) included in the distribution content. Hereinafter, an example of the learning process executed by the information processing apparatus 10 will be described with reference to FIG.

  First, the information processing apparatus 10 acquires information serving as learning data from the data server 20. For example, the information processing apparatus 10 acquires the distribution content distributed to the user in the past from the data server 20. Then, the information processing apparatus 10 registers the information acquired from the data server 20 in the learning data database.

  FIG. 3 is a diagram illustrating an example of learning data registered in the learning data database. In the learning data database, information associating the text, title, heading, and score is registered as learning data. A learning data ID (Identifier) is assigned to each set of a text, a title, and a heading. “Body” and “Title” are texts included in the distribution content.

  “Heading” is text for the user to easily grasp the contents of the distribution content. The headline is text corresponding to text (for example, a body or a title) included in the distribution content. A headline serving as learning data is created by a predetermined person (for example, an employee of a portal site operator) reading and summarizing text included in the distribution content. The headline is generated so as to satisfy a predetermined condition. For example, the heading is generated so as to be a predetermined number of characters or less. As an example, the heading is generated to be 13.5 characters or less. At this time, 13.5 characters is the number of characters with one double-byte character as one character. One half-width character is 0.5 character. In the following description, the predetermined condition is referred to as “generation condition”. The generation conditions are not limited to 13.5 characters or less. The number of characters as a generation condition may be more or less than 13.5 characters. Of course, the generation condition is not limited to the number of characters.

  “Heading” can be rephrased as a summary, title, abbreviated title, topic, topic heading, or the like. Similarly, the “title” included in the distribution content can also be called a summary or the like. In the present embodiment, it is assumed that the “headline” is a shortened text created based on the “title” included in the distribution content. Of course, the “headline” may be a shortened text created based on the text. Note that even if the sentence is shortened syntactically, the number of characters of the “heading” may be larger than the number of characters of the “title”. For example, when an abbreviation included in the “title” is replaced with a formal name, the number of characters may increase as a result even if the sentence is shortened syntactically. In such a case, “heading” is a shortened text of “title”.

  In the case of the example in FIG. 3, the headline U11 is a headline created by an employee of the site operator (hereinafter simply referred to as the operator) based on the title T11 or the text K11. The headline U12 is a headline created by the operator based on the title T12 or the text K12. The heading U13 is a heading created by the operator based on the title T13 or the text K13. Note that information is not stored in the item “heading” in some data. For example, in the example of FIG. 3, information having a learning data ID of 2001 or later does not store information in the “heading” item. This indicates that the headline has not been created by the operator. In the following description, a data group in which a heading is generated is referred to as a group G1, and a data group in which a heading is not generated is referred to as a group G2. Since the headline is created manually, the number of contents belonging to the group G2 is actually overwhelmingly larger than the number of contents belonging to the group G1. As an example, the number of contents belonging to the group G1 is 100,000, whereas the number of contents belonging to the group G2 is 1 million. If the content belonging to the group G2 can be used effectively for model learning, the information processing apparatus 10 can acquire a highly accurate model.

  In the present embodiment, the information processing apparatus 10 learns a first model (hereinafter referred to as model M1) using content (for example, a title and a headline) belonging to the group G1. For example, the information processing apparatus 10 generates the model M1 by learning based on the relationship between the title and the headline. Then, the information processing apparatus 10 generates a pseudo heading (hereinafter referred to as a pseudo heading) of content belonging to the group G2 using the model M1. Then, the information processing apparatus 10 stores the generated pseudo heading in the “heading” item of the learning data. FIG. 4 is a diagram illustrating an example of learning data in which pseudo headings are stored. In the example of FIG. 4, headings PU11 to PU16 are pseudo headings. Then, the information processing apparatus 10 learns a second model (hereinafter referred to as a model M2) using content (for example, a title) belonging to the group G2 and a pseudo headline thereof. Since the number of contents belonging to the group G2 is overwhelmingly larger than the number of contents belonging to the group G1, the information processing apparatus 10 can acquire a highly accurate model.

  Returning to FIG. 1, the information processing apparatus 10 acquires a plurality of sets of titles and headings from the data registered in the learning data database. For example, the information processing apparatus 10 acquires titles and headings of content belonging to the group G1. Then, the learning unit of the information processing apparatus 10 generates the model M1 by learning based on the relationship between the title and the headline (Step S1). Here, the model M1 is a model that generates text (hereinafter referred to as output text) that satisfies a generation condition from text input to the model (hereinafter referred to as input text). The information processing apparatus 10 generates the model M1 by performing learning (for example, supervised learning) using the title as input data and the headline as correct answer data.

  In the following description, the text that becomes input data (in the case of the present embodiment, the title of the content belonging to the group G1) is the first text, and the text that becomes correct answer data (in the case of the present embodiment). , Headings of contents belonging to the group G1) may be referred to as second text. In the following description, the correct answer data may be referred to as a teacher label.

  The first text is, for example, text from a predetermined distribution subject. As an example, the first text is the title of an article acquired from a predetermined distribution subject. The second text is, for example, a shortened title (that is, a heading) obtained by converting the first text so as to satisfy the generation condition. Further, the first text may be a text from a predetermined information medium, for example. As an example, the first text is a title of an article acquired from a predetermined information medium. The second text is, for example, a shortened title (that is, a heading) obtained by converting the first text so as to satisfy the generation condition.

  The model M1 may be a model including an encoder that generates feature information indicating features of the input text from the input text, and a decoder that generates output text from the feature information. FIG. 5 is a diagram illustrating an example of the model M1. Specifically, FIG. 5 illustrates a model M that is an example of the model M1.

  The model M is a Sequence to Sequence Model (hereinafter also referred to as Seq2Seq) using LSTM (Long Short Term Memory) which is an example of RNN (Recurrent Neural Network). Seq2Seq is a kind of encoder-decoder model, which allows word strings (Sequence) of different lengths to be input and word strings (Sequence) of different lengths to be output. A model M shown in FIG. 5 includes an encoder E1 and a decoder D1.

  The encoder E1 extracts features of the input text. The encoder E1 has an input layer X and an intermediate layer (hidden layer) H1. Words (words W11 to W13 shown in FIG. 5) included in the first text (in the case of the present embodiment) are sequentially input to the encoder E1. <EOS> is an abbreviation for End Of Statement and indicates the end of text. The encoder E1 shown in FIG. 5 is a diagram in which the input layer X and the intermediate layer H1 are developed in the time axis direction. In addition to the output from the input layer X, the previous output of the intermediate layer H1 is input to the intermediate layer H1. In the example shown in FIG. 5, the number of words included in the first text is three from words W11 to W13, but the number of words included in the first text may be more than three. The encoder E1 outputs feature information F which is a multidimensional quantity (for example, a vector) indicating the feature of the input text.

  The encoder E1 is provided with a layer (hereinafter referred to as an input layer N1) for inputting classification information P indicating the classification of text to be input. The classification information P is information indicating the classification of text (for example, title) input to the input layer X, for example. For example, the classification information P is information indicating, for example, whether a text input to the input layer X is a sports-related text or a political-related text. The input layer N1 converts the classification information P into a multidimensional quantity (for example, a vector) having the number of dimensions corresponding to the input of the intermediate layer H1. In the example of FIG. 5, the output of the input layer N1 is input to the intermediate layer H1 before the word W11 that is the first word of the text is input. By inputting the classification information P of the first text to the encoder E1, the encoder E1 can learn the characteristics of the first text in consideration of the classification of the first text. Of course, the input layer N1 may not be prepared for the encoder E1.

  The decoder D1 outputs a text obtained by shortening the input text (that is, a shortened text). The decoder D1 has an intermediate layer (hidden layer) H2 and an output layer Y. The decoder D1 shown in FIG. 5 is a diagram in which the intermediate layer (hidden layer) H2 and the output layer Y are developed in the time axis direction, like the encoder E1. The feature information F output from the encoder E1 is input to the decoder D1, and words (words W21 to W23 shown in FIG. 5) that are shortened texts are sequentially output from the output layer Y. In the example shown in FIG. 5, the number of words included in the abbreviated text is three from word W21 to W23, but the number of words included in the abbreviated text may be more than three.

  The information processing apparatus 10 learns the model M so that when the first text is input to the encoder E1, the second text corresponding to the first text is output from the decoder D1. For example, the information processing apparatus 10 uses a weight (that is, a value considered when a value is transmitted between the nodes by a method such as back propagation so that the shortened text output from the decoder D1 approaches the second text). , Correct the connection coefficient). As a result, the information processing apparatus 10 causes the model M to learn the characteristics of the first text. Note that the information processing apparatus 10 may correct the weight value based on the cosine similarity between a vector that is a distributed representation of the shortened text and a vector that is a distributed representation of the second text.

  Returning to FIG. 1, the information processing apparatus 10 acquires, as input data, a title for which no heading has been generated from data registered in the learning data database. For example, the information processing apparatus 10 acquires the titles of content belonging to the group G2. At this time, if there are a plurality of titles for which headings are not generated, the information processing apparatus 10 acquires a plurality of titles. And the production | generation part of the information processing apparatus 10 produces | generates the pseudo heading (output data) corresponding to the input title by inputting the acquired title into the model M1. If there are a plurality of acquired titles, the information processing apparatus 10 inputs the plurality of titles into the model M1 and generates a plurality of pseudo headings (step S2).

  Then, the learning data generation unit of the information processing apparatus 10 generates data to be registered in the learning data database based on the input data (title) and the output data (generated pseudo heading). Then, the learning data generation unit of the information processing apparatus 10 registers the generated data in the learning data database (step S3).

  In the following description, text serving as input data to the model M1 is referred to as third text, and output text obtained by inputting the third text into the model M1 is referred to as fourth text. is there. The third text is, for example, a title of content belonging to the group G2. The fourth text is, for example, a pseudo headline for content belonging to the group G2.

  Then, returning to step S1, the learning unit of the information processing apparatus 10 learns recursively based on the relationship between the third text and the fourth text. Specifically, the information processing apparatus 10 acquires a plurality of sets of titles and pseudo headings of content belonging to the group G2 from data registered in the learning data database. The pseudo heading acquired here is the pseudo heading generated in step S2. Then, the information processing apparatus 10 updates the model M1 through learning based on the relationship between the title and the pseudo headline (Step S1). Thereafter, the information processing apparatus 10 repeatedly executes the processes of steps S1 to S3.

  The information processing device 10 generates an output text from the input text using the model M1 generated by recursive learning, and outputs the output text to the terminal device 30.

  According to the present embodiment, the information processing apparatus 10 generates a pseudo headline for content belonging to the group G2 using the model M1. Then, the information processing apparatus 10 generates a model M2 using content (for example, a title) belonging to the group G2 and its pseudo heading. As described above, the number of contents belonging to the group G2 is overwhelmingly larger than the number of contents belonging to the group G1. Therefore, the information processing apparatus 10 can acquire a highly accurate model. In addition, since the information processing apparatus 10 performs learning recursively, it is possible to acquire a model with higher accuracy.

[Embodiment 2]
[1-4. Other examples of information processing apparatus]
Next, another example of learning processing and generation processing executed by the information processing apparatus 10 will be described. FIG. 6 is a diagram illustrating an example of processing executed by the information processing apparatus 10 according to the second embodiment. The configurations of the information processing device 10, the data server 20, and the terminal device 30 are the same as those in the first embodiment.

[1-5. About learning process)
The information processing apparatus 10 performs the following learning process to learn a model for generating a text to be a headline from a text (for example, a title) included in the distribution content. Hereinafter, an example of the learning process executed by the information processing apparatus 10 will be described with reference to FIG.

  First, the information processing apparatus 10 acquires information serving as learning data from the data server 20 (step S101). For example, the information processing apparatus 10 acquires the distribution content distributed to the user in the past from the data server 20. Then, the information processing apparatus 10 registers the information acquired from the data server 20 in the learning data database as shown in FIG. 3, for example. As described above, the learning data registered in the learning database shown in FIG. 3 is divided into a data group (group G1) in which headings are generated and a data group (group G2) in which no headings are generated. .

  In the present embodiment, the information processing apparatus 10 learns a first model (hereinafter referred to as model M1) using content (for example, a title and a headline) belonging to the group G1. For example, the information processing apparatus 10 generates the model M1 by learning based on the relationship between the title and the headline. Then, the information processing apparatus 10 generates a pseudo heading (hereinafter referred to as a pseudo heading) of content belonging to the group G2 using the model M1. Then, the information processing apparatus 10 stores the generated pseudo heading in the “heading” item of the learning data as shown in FIG.

  A learning model for natural language processing (for example, RNN (Recurrent Neural Network)) is also a model for predicting a likely next word from one or a plurality of words input from the present to the past. Therefore, the information processing apparatus 10 can calculate a score (hereinafter simply referred to as a score) indicating the likelihood of the text generated by the learning model based on the learning result of the learning model (for example, the generated neural network). is there. The score can be rephrased as likelihood. In order to further improve the accuracy of the model, the information processing apparatus 10 may provide an item “score” in the learning data and store the calculated score in the learning data as shown in FIGS. 3 and 4. For example, the information processing apparatus 10 stores a score V21 that is a probability (for example, P (PU21 | T21)) that the headline PU21 is formed from the title T21 in the item “score” of the learning data ID “2001”. Similarly, the information processing apparatus 10 stores scores V22 to V26, which are probabilities that headings PU22 to PU26 are formed from the titles T22 to T26, in the “score” item of the learning data. The method of using the score will be described later.

  Returning to FIG. 6, the information processing apparatus 10 acquires a plurality of sets of titles and headings from the data registered in the learning data database. For example, the information processing apparatus 10 acquires titles and headings of content belonging to the group G1. Then, the learning unit of the information processing device 10 generates the model M1 by learning based on the relationship between the title and the headline (Step S102). The information processing apparatus 10 learns the model M1 (for example, supervised learning) by learning using the title as input data and the heading as correct data.

  In the following description, as in the first embodiment, the text that becomes input data during learning (in the case of this embodiment, the title of content belonging to the group G1) becomes the first text and correct data. The text (in the case of the present embodiment, the heading of content belonging to the group G1) may be referred to as the second text. In the following description, the correct answer data may be referred to as a teacher label as in the first embodiment.

  Returning to FIG. 6, the information processing apparatus 10 acquires, as input data, a title for which no headline has been generated from data registered in the learning data database. For example, the information processing apparatus 10 acquires the titles of content belonging to the group G2. At this time, if there are a plurality of titles for which headings are not generated, the information processing apparatus 10 acquires a plurality of titles. And the production | generation part of the information processing apparatus 10 produces | generates the pseudo heading (output data) corresponding to the input title by inputting the acquired title into the model M1. If there are a plurality of acquired titles, the information processing apparatus 10 inputs the plurality of titles to the model M1 and generates a plurality of pseudo headings (step S103).

  Then, the learning data generation unit of the information processing apparatus 10 generates data to be registered in the learning data database based on the input data (title) and the output data (generated pseudo heading). Then, the learning data generation unit of the information processing apparatus 10 registers the generated data in the learning data database (step S104).

  In the following description, as in the first embodiment, the text serving as input data to the model M1 is the third text, and the output text obtained by inputting the third text to the model M1. Sometimes called fourth text. The third text is, for example, a title of content belonging to the group G2. The fourth text is, for example, a pseudo headline for content belonging to the group G2.

  Subsequently, the information processing apparatus 10 acquires a plurality of sets of titles and pseudo headings of content belonging to the group G2 from the data registered in the learning data database. The pseudo heading acquired here is the pseudo heading generated in step S3. Then, the information processing apparatus 10 generates the model M2 by learning based on the relationship between the title and the pseudo headline (Step S105). Here, the model M2 is a model that generates output text from input text, like the model M1. Here, the model M2 may have the same configuration as the model M1 or may have a different configuration. In the case of this embodiment, the model M2 is Seq2Seq using LSTM.

  For example, assume that the model M2 has the same configuration as the model M shown in FIG. At this time, when the third text (for example, title) is input to the encoder E1, the information processing apparatus 10 outputs the fourth text (for example, pseudo heading) corresponding to the third text from the decoder D1. Then, the model M2 is learned. For example, the information processing apparatus 10 uses a weight (that is, a value considered when a value is transmitted between nodes by a method such as back propagation so that the shortened text output from the decoder D1 approaches the fourth text). , Correct the connection coefficient). As a result, the information processing apparatus 10 causes the model M to learn the characteristics of the first text. Note that the information processing apparatus 10 may correct the weight value based on the cosine similarity between a vector that is a distributed representation of the shortened text and a vector that is a distributed representation of the second text.

  Note that the information processing apparatus 10 not only includes the title (third text) and pseudo heading (fourth text) of content belonging to the group G2, but also the title (first text) and heading (first text) of the content belonging to the group G1. The second text) may be a text for learning the model M2. Then, the information processing apparatus 10 may perform learning of the second model based on the first text, the second text, the third text, and the fourth text. For example, assume that the model M2 has the same configuration as the model M shown in FIG. At this time, the information processing apparatus 10 learns the model M so that when the first text or the third text is input to the encoder E1, the second text or the fourth text is output from the decoder D1. Do. Thereby, the information processing apparatus 10 can use more text sets as learning data. Moreover, the information processing apparatus 10 can use the second text (heading) that is considered to be more accurate than the fourth text (pseudo heading) as the learning data. Therefore, the information processing apparatus 10 can generate a highly accurate model M2.

  In step S103, the information processing apparatus 10 generates a plurality of pseudo headings. However, it is not always necessary to use all the generated pseudo headings as learning data. The information processing apparatus 10 selects, as learning data, a pseudo heading (fourth text) selected according to a predetermined criterion from the plurality of pseudo headings (a plurality of fourth texts) generated in step S103. Also good. At this time, the information processing apparatus 10 selects a pseudo heading used for learning the model M2 based on a score (likelihood) that is calculated based on the learning result of the model M1 and indicates the likelihood of the pseudo heading. May be. For example, the information processing apparatus 10 may select a pseudo heading whose score exceeds a predetermined value as learning data. Alternatively, the information processing apparatus 10 may select a predetermined number of pseudo headings as learning data in descending order of score. The score may be, for example, a score registered in the learning data database (for example, V21 to V26 shown in FIG. 4). Then, the information processing apparatus 10 learns the model M2 based on the pseudo heading and the title corresponding to the selected pseudo heading. As a result, the information processing apparatus 10 can use only pseudo headings having a high score among a plurality of pseudo headings as learning data. Therefore, the information processing apparatus 10 can generate a highly accurate model M2.

[1-6. About generation processing)
Next, an example of a generation process that generates an output text corresponding to an input text using the model learned by the above-described learning process will be described. At this time, the output text may be text that satisfies the generation condition. For example, the output text is a shortened text of the input text.

  First, the information processing apparatus 10 acquires information input to the model M from the data server 20 (step S105). For example, the information processing apparatus 10 acquires distribution contents (text and title) scheduled to be distributed from the data server 20.

  Then, the information processing apparatus 10 generates an output text (for example, a headline) corresponding to the input text by inputting the input text (for example, a title) to the model M2 (step S106). For example, if the model M2 is the model M shown in FIG. 5, the information processing apparatus 10 sequentially inputs words included in the input text to the encoder E1. Then, the information processing apparatus 10 sequentially obtains output text and words from the decoder D1. At this time, the information processing apparatus 10 may input the classification information of the input text to the encoder E1 in order to increase the accuracy of the output text.

  When the output text is generated, the information processing apparatus 10 generates summary content using the output text. Then, the information processing device 10 distributes the summary content to the terminal device 30 (step S107).

  According to the present embodiment, the information processing apparatus 10 generates a pseudo headline for content belonging to the group G2 using the model M1. Then, the information processing apparatus 10 generates a model M2 using content (for example, a title) belonging to the group G2 and its pseudo heading. As described above, the number of contents belonging to the group G2 is overwhelmingly larger than the number of contents belonging to the group G1. Therefore, the information processing apparatus 10 can acquire a highly accurate model.

[2. Configuration example of information processing apparatus]
The operation of the information processing apparatus 10 according to the present embodiment has been described above. Hereinafter, the configuration of the information processing apparatus 10 will be described.

  The information processing apparatus 10 is a server host computer (hereinafter simply referred to as a “server”) that processes a request from a client computer such as the terminal apparatus 30. The information processing apparatus 10 may be a PC server, a midrange server, or a mainframe server. Further, the information processing apparatus 10 may be configured by a single server, or may be configured by a plurality of servers that cooperate to execute processing. When the information processing apparatus 10 includes a plurality of servers, the installation locations of these servers may be separated. Even if the installation locations are separated, these servers can be regarded as a single information processing device as long as the processing is executed in cooperation. The information processing device 10 functions as a generation device and a learning device.

  As described above, the information processing apparatus 10 is connected to the data server 20 and the terminal apparatus 30 via the network. The network is a communication network such as a LAN (Local Area Network), a WAN (Wide Area Network), a telephone network (a mobile phone network, a fixed telephone network, etc.), a regional IP (Internet Protocol) network, and the Internet. The network may include a wired network or a wireless network.

  FIG. 7 is a diagram illustrating a configuration example of the information processing apparatus 10 according to the embodiment. The information processing apparatus 10 includes a communication unit 11, a storage unit 12, and a control unit 13. Note that the configuration shown in FIG. 7 is a functional configuration, and the hardware configuration may be different.

  The communication unit 11 is a communication interface that communicates with an external device. The communication unit 11 may be a network interface or a device connection interface. For example, the communication unit 11 may be a LAN interface such as a NIC (Network Interface Card) or a USB interface configured by a USB (Universal Serial Bus) host controller, a USB port, or the like. The communication unit 11 may be a wired interface or a wireless interface. The communication unit 11 functions as a communication unit of the information processing apparatus 10. The communication unit 11 communicates with the data server 20 and the terminal device 30 according to the control of the control unit 13.

  The storage unit 12 is a storage device capable of reading and writing data, such as a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, and a hard disk. The storage unit 12 functions as a storage unit of the information processing apparatus 10. The storage unit 12 stores a learning data database 121, a model database 122, and a content information database 123.

  Learning data is registered in the learning data database 121. FIG. 8 is a diagram illustrating an example of information registered in the learning data database 121. Information having items such as “learning data ID (Identifier)”, “text”, “title”, “heading”, and “score” is registered in the learning data database 121.

  Of the information shown in FIG. 8, “text”, “title”, “headline”, and “score” are “text”, “title”, “headline”, and “score” shown in FIG. Corresponding to In addition to the information shown in FIG. 8, various information related to the user who has browsed the learning data and the summary data may be registered in the learning data database 121. In the example shown in FIG. 8, the information registered in the learning data database 121 includes “K11, K12, K21, K22”, “T11, T12, T21, T22”, “U11, U12, PU21”, “V21”. In this example, text data and binary data are registered.

  Here, the “learning data ID” is an identifier for identifying learning data. The “attribute information” is information indicating text attributes. Further, the “body” is a text that becomes a body included in content (for example, an article) distributed to the user. The “title” is a title attached to the content or the text included in the content. “Heading” is a heading attached to content (text or title).

  For example, in the example illustrated in FIG. 8, information such as the learning data ID “1001”, the text “K11”, the title “T11”, and the heading “U11” is registered in the learning data database 121 in association with each other. Such information indicates, for example, that the learning data indicated by the learning data ID “1001” includes the text indicated by “K11”, the title indicated by “T11”, and the heading indicated by “U11”. In the learning data database 121, information such as a learning data ID “2001”, a text “K21”, a title “T21”, a pseudo heading “PU21”, and a score “V21” are registered in association with each other. Such information includes, for example, the learning data indicated by the learning data ID “2001”, the text indicated by “K21”, the title indicated by “T21”, the pseudo heading indicated by “PU21”, and “V21”. Indicates that a score is included.

  Returning to FIG. 7, model data of the information processing apparatus 10 is registered in the model database 122. FIG. 9 is a diagram illustrating an example of information registered in the model database 122. In the example illustrated in FIG. 9, information such as “model ID” and “model data” is registered in the model database 122.

  Here, the “model ID” is information for identifying each model. The “model data” is data of the model indicated by the associated “model ID”. For example, nodes in each layer, functions adopted by each node, connection relationships between nodes, and connections between nodes Information including a connection coefficient set for.

  For example, in the example shown in FIG. 9, information such as model ID “3001” and model data “M1” is registered in association with each other. Such information indicates, for example, that the model data indicated by “3001” is “M1”. In the example illustrated in FIG. 9, conceptual information such as “M1 to M3” is described as information registered in the model database 122, but in reality, a character string or a numerical value indicating a model structure or a connection coefficient is described. Etc. will be registered.

  The model M is, for example, a model M1 (first model) learned based on the first text and the second text corresponding to the first text. The model M may be a model M2 (second model) learned based on the third text and the fourth text generated by inputting the third text into the model M1. The model M may be a model M3 (third model) learned based on the third text and the fifth text generated by inputting the third text into the model M2. Good.

  The model M generates an output text that satisfies a predetermined condition from the input text. Such a model M includes an input layer to which text is input, an output layer that outputs text corresponding to the text input to the input layer and that satisfies a generation condition, and any one from the input layer to the output layer. A first element belonging to a layer other than the output layer, and a second element whose value is calculated based on the first element and the weight of the first element, and is input to the input layer Text input to the input layer by performing an operation based on the first element and the weight of the first element (that is, the connection coefficient), with each element belonging to each layer other than the output layer as the first element. This is a model for causing a computer to function so as to output text that satisfies the generation condition from the output layer.

  Here, it is assumed that the model M is realized by a regression model represented by “y = a1 * x1 + a2 * x2 +... + Ai * xi”. In this case, the first element included in the model M corresponds to input data (xi) such as x1 and x2. The weight of the first element corresponds to the coefficient ai corresponding to xi. Here, the regression model can be regarded as a simple perceptron having an input layer and an output layer. When each model is regarded as a simple perceptron, the first element can correspond to any node of the input layer, and the second element can be regarded as a node of the output layer.

  Further, it is assumed that the model M is realized by a neural network having one or a plurality of intermediate layers such as DNN (Deep Neural Network). In this case, the first element included in the model M corresponds to any node of the input layer or the intermediate layer. The second element corresponds to the next node, which is a node to which a value is transmitted from the node corresponding to the first element. The weight of the first element corresponds to a connection coefficient that is a weight considered for a value transmitted from a node corresponding to the first element to a node corresponding to the second element.

  The information processing apparatus 10 calculates the output text using a model having an arbitrary structure such as the above-described regression model or neural network. Specifically, when a text (for example, a title) is input, the model M has a coefficient so as to output a text (for example, a heading) that corresponds to the input text and satisfies the generation conditions. Is set. In the model M, when a text and attribute information of the text are input, a coefficient may be set so as to output text corresponding to the input text and satisfying a generation condition. The information processing apparatus 10 generates text (for example, a headline) that satisfies the generation condition using such a model M.

  In the above example, when a text (and attribute information of the text) is input, the model M outputs a text (for example, a headline) that corresponds to the input text and satisfies the generation conditions. An example of a model (hereinafter referred to as model V) is shown. However, the model M according to the embodiment may be a model generated based on a result obtained by repeatedly inputting / outputting data to / from the model V. For example, the model V learns to receive “text (and attribute information of the text)” as input and output “text corresponding to the input text and satisfying the generation condition” output from the model V. It may be a model.

  When the information processing apparatus 10 performs a learning process or a generation process using GAN (Generative Adversarial Networks), the model M may be a model that constitutes a part of the GAN.

  Returning to FIG. 7, content information distributed to the user is registered in the content information database 123. For example, articles acquired from content partners are registered in the content information database 123. FIG. 10 is a diagram illustrating an example of information registered in the content information database 123. In the content information database 123, information having items such as “content ID”, “text”, and “title” is registered. The “content ID” is an identifier for identifying content data. “Text” and “Title” are the same as “Text” and “Title” shown in FIG.

  Returning to FIG. 7, the control unit 13 is a controller and is stored in a storage device inside the information processing apparatus 10 by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). Various programs are implemented by executing the RAM or the like as a work area. The control unit 13 is a controller, and may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

  In addition, the control unit 13 performs processing on the input text (for example, a title) input to the input layer of the model M by information processing according to the model M (models M1, M2, M3, etc.) stored in the storage unit 12. The calculation based on the coefficients of the model M (that is, the coefficients corresponding to the various features learned by the model M) is performed, and output text (for example, headings) is output from the output layer of the model M. In addition to the input text, attribute information of the input text may be input to the input layer.

  As illustrated in FIG. 7, the control unit 13 includes a learning data acquisition unit 131, a learning unit 132, a learning data generation unit 133, a generation unit 134, and an output control unit 135. Blocks constituting the control unit 13 (learning data acquisition unit 131 to output control unit 135) are functional blocks that indicate functions of the control unit 13, respectively. These functional blocks may be software blocks or hardware blocks. For example, each functional block described above may be one software module realized by software (including a microprogram), or may be one circuit block on a semiconductor chip (die). Of course, each functional block may be one processor or one integrated circuit. The method of configuring the functional block is arbitrary. In addition, the control part 13 may be comprised by the functional unit different from the above-mentioned functional block.

  The learning data acquisition unit 131 acquires a first text and a second text corresponding to the first text as learning data. For example, the learning data acquisition unit 131 acquires, as learning data, an article (text and / or title) created by a predetermined content partner from the data server 20. Furthermore, the learning data acquisition unit 131 acquires a headline attached to the article as learning data. The headline may be created by an operator of a portal site that has distributed the article to past users. The learning data acquisition unit 131 registers the acquired data in the learning data database 121 in association with each other.

  In addition, the learning data acquisition unit 131 acquires a third text and a fourth text corresponding to the third text as learning data. For example, the learning data acquisition unit 131 acquires, as learning data, an article (text and / or title) created by a predetermined content partner from the data server 20. Furthermore, the learning data acquisition unit 131 uses, as learning data, a text (for example, a headline) generated by the generation unit 134 by inputting an article (body and / or title) to the model M1 (first model). get. The learning data acquisition unit 131 registers the acquired data in the learning data database 121 in association with each other.

  Note that the learning data acquisition unit 131 may acquire the third text and the fifth text corresponding to the third text as the learning data. The fifth text is, for example, text (for example, a headline) generated by the generation unit 134 inputting an article (body and / or title) to the model M2 (second model).

  The learning unit 132 learns the model M and stores the learned model M in the model database 122. Model M may be model M1 or model M2. Of course, the model M may be a model other than the models M1 and M2 (for example, a model M3 described later). When the learning unit 132 inputs the first text (for example, a title) included in the learning data to the model M, the learning unit 132 inputs the second text (for example, the heading corresponding to the input title) included in the learning data. ) Is set so that the connection coefficient of the model M is output. That is, the learning unit 132 learns the model M so that when the input text is input, the model M outputs the output text corresponding to the input text.

  For example, the learning unit 132 is a node in the input layer of the model M, and a predetermined text (for example, a title) ) And the data is propagated to the output layer of the model M through each intermediate layer, thereby outputting a text (for example, a headline) corresponding to the predetermined text. Then, the learning unit 132 corrects the connection coefficient of the model M based on the difference between the text actually output by the model M and the second text (for example, heading) included in the learning data. Alternatively, the learning unit 132 includes the text actually output by the model M and the fourth text (for example, pseudo heading) or the fifth text (for example, second pseudo heading described later) included in the learning data. Based on the difference, the connection coefficient of the model M is corrected. For example, the learning unit 132 may correct the connection coefficient using a technique such as backpropagation. In addition, the learning unit 132 may correct the connection coefficient based on the cosine similarity between the vector that is the distributed representation of the second text and the vector that is the distributed representation of the text that is actually output by the model M. Good. Further, the learning unit 132 determines the connection coefficient based on the cosine similarity between the vector that is the distributed representation of the fourth text or the fifth text and the vector that is the distributed representation of the text actually output by the model M. Corrections may be made.

  Note that the learning unit 132 may learn the model M using any learning algorithm. For example, the learning unit 132 may learn the model M using a learning algorithm such as a neural network, a support vector machine, clustering, or reinforcement learning.

  Here, the learning unit 132 learns a model M2 (second model) that generates output text from input text based on the third text and the fourth text. At this time, the model M2 may be a model that generates text of a predetermined number of characters or less as output text. The third text may be a title that is not yet associated with a headline. The fourth text is the same as the model M1 (first model) learned based on the first text (for example, title) and the second text (for example, headline) corresponding to the first text. 3 (for example, a pseudo heading) may be generated by inputting the third text. For example, the generation unit 134 may generate a fourth text corresponding to the third text by inputting the third text to the model M1.

  At this time, the learning unit 132 may learn the model M2 based on the first text, the second text, the third text, and the fourth text.

  The generation unit 134 may generate a plurality of fourth texts by inputting a plurality of third texts into the model M (first model). In this case, the learning unit 132 selects the fourth text selected from the plurality of fourth texts generated by the generation unit 134 according to a predetermined criterion and the third text corresponding to the selected fourth text. Based on the above, the model M2 may be learned.

  For example, the learning unit 132 calculates a plurality of second values generated by the generation unit 134 based on a first score that is calculated based on the learning result of the first model and indicates the likelihood of the fourth text. The fourth text used for learning the model M2 is selected from the four texts. Then, the learning unit 132 learns the model M2 based on the selected fourth text and the third text corresponding to the selected fourth text.

  The learning unit 132 learns a model M3 (third model) that generates output text from input text based on the third text and the fifth text. At this time, the fifth text may be a text corresponding to the third text. More specifically, the fifth text may be a text generated by the generation unit 134 inputting the third text into the model M2.

  At this time, the learning unit 132 may learn the model M3 based on the first text, the second text, the third text, and the fifth text.

  The generation unit 134 may generate a plurality of fifth texts by inputting a plurality of third texts into the model M2 (second model). In this case, the learning unit 132 uses the model M3 based on the fifth text selected from the plurality of fifth texts according to a predetermined criterion and the third text corresponding to the selected fifth text. (Third model) may be learned.

  For example, the learning unit 132 generates a score based on the second score indicating the likelihood of the fifth text, which is calculated based on the learning result of the first model or the second model. A fifth text to be used for learning the model M3 is selected from the plurality of fourth texts. Then, the learning unit 132 learns the model M3 based on the selected fifth text and the third text corresponding to the selected fifth text.

  The learning unit 132 also learns a model M including an encoder that generates feature information indicating features of the input text from the input text, and a decoder that generates output text from the feature information. The model M may be any of the models M1 to M3 (first to third models).

  Further, the learning data generation unit 133 generates a text to be used for learning based on the text output from the model M.

  The learning unit 132 performs learning based on the relationship between the third text and the fourth text.

  Further, the learning unit 132 performs recursive learning based on the relationship between the third text and the fourth text.

  In addition, the learning unit 132 generates a second model different from the model by learning the relationship between the third text and the fourth text.

  Note that the structure of the second model perceptron generated based on the relationship between the third text and the fourth text is a model generated based on the relationship between the first text and the second text. It may be a structure in which at least one of the number of intermediate layers and the number of neurons is larger than the structure of the perceptron.

  In addition, the structure of the second model perceptron generated based on the relationship between the third text and the fourth text is a model generated based on the relationship between the first text and the second text. It may be a structure in which at least one of the number of intermediate layers and the number of neurons is smaller than the structure of the perceptron.

  In addition, the learning unit 132 generates the second model by learning based on the relationship between the first text and the second text and the relationship between the third text and the fourth text.

  Further, the learning unit 132 performs learning based on the relationship between the fourth text selected from the plurality of fourth texts according to a predetermined criterion and the third text corresponding to the selected fourth text. A second model is generated.

  In addition, the learning unit 132 calculates a score calculated based on the learning result of the model and indicates the likelihood of the fourth text from among the plurality of fourth texts generated by the generation unit. Select the fourth text to be used for learning. Then, the learning unit 132 performs learning based on the selected fourth text and the third text corresponding to the selected fourth text.

  The learning unit 132 also learns a third model that generates output text from input text based on the third text and the fifth text.

  The learning unit 132 generates a third model by learning based on the relationship between the first text and the second text and the relationship between the third text and the fifth text.

  Further, the learning unit 132 performs learning based on the relationship between the fifth text selected from the plurality of fifth texts according to a predetermined criterion and the third text corresponding to the selected fifth text. Generate a third model,

  Note that the model may include an encoder that generates feature information indicating features of the input text from the input text, and a decoder that generates output text from the feature information.

  In addition, the learning unit 132 performs learning based on the relationship between the third text and the pseudo headline corresponding to the third text.

  The generation unit 134 generates an output text corresponding to the input text by inputting the input text to the model M. At this time, the model M may be any of models M1 to M3 (first to third models). Further, the generation unit 134 may generate a text that satisfies the generation conditions as the output text. For example, the generation unit 134 may generate a text having a predetermined number of characters or less as the output text.

  In addition, the generation unit 134 inputs the third text into the model M1 (first model) learned based on the first text and the second text corresponding to the first text, thereby generating the third text. A fourth text corresponding to the text is generated. At this time, the generation unit 134 may acquire a plurality of fourth texts by inputting the plurality of third texts into the model M1.

  Further, the generation unit 134 generates the fifth text corresponding to the third text by inputting the third text to the model M2 (second model). At this time, the generation unit 134 may acquire a plurality of fifth texts by inputting a plurality of third texts into the model M2.

  In addition, the generation unit 134 generates the sixth text (for example, a headline) corresponding to the third text by inputting the third text to the model M3 (third model). At this time, the generation unit 134 may acquire the plurality of sixth texts by inputting the plurality of third texts to the model M3.

  Further, the generation unit 134 corresponds to the third text to the third text using a model generated by learning based on the relationship between the first text and the second text corresponding to the first text. The fourth text to be generated is generated. At this time, the generation unit 134 may generate a plurality of fourth texts by inputting a plurality of third texts to the first model.

  Further, the generation unit 134 generates a pseudo heading corresponding to the third text from the third text using a model generated by learning based on the first text and the heading corresponding to the first text. .

  The generation unit 134 generates the third text from the third text by using a model generated by learning based on the relationship between the first text and the second text corresponding to the first text. An output text corresponding to the input text is generated from the input text using a predetermined model generated by learning based on the relationship between the fourth text and the fourth text.

  For example, the generation unit 134 uses the model generated by learning based on the relationship between the third text, the first text, and the heading corresponding to the first text, and the pseudo generated from the third text. A headline corresponding to the input text is generated from the input text using a predetermined model generated by learning based on the relationship between the headline and the headline.

  The output control unit 135 outputs the text generated by the generation unit 134 to the user. For example, the output control unit 135 distributes content including the text generated by the generation unit 134 in response to a request from the terminal device 30. Note that the output control unit 135 may provide content including the text generated by the generation unit 134 to the data server 20 and distribute the content from the data server 20.

[3. Processing flow of information processing apparatus]
Next, a procedure of processing executed by the information processing apparatus 10 will be described. The information processing apparatus 10 performs learning processing and generation processing. The information processing apparatus 10 includes a multitask OS, for example, and can execute these processes in parallel.

[3-1. Learning process)
First, the learning process will be described. FIG. 11 is a flowchart illustrating an example of the learning process. The learning process is a process of learning a model M that generates output text from input text. When the information processing apparatus 10 receives a process start command from the user, the information processing apparatus 10 executes a learning process.

  First, the information processing apparatus 10 acquires learning data (step S11). For example, the information processing apparatus 10 acquires information on distribution contents (text and title) from the data server 20. In addition, if there is a heading corresponding to the distribution content, the information processing apparatus 10 acquires the heading. Then, the information processing apparatus 10 registers the acquired data in the learning data database 121.

  Subsequently, the information processing apparatus 10 acquires a combination of a title (first text) and a heading (second text) from the learning data acquired in step S11 (step S12). The information processing apparatus 10 generates a first model (model M1) based on learning using the first text as input data and the second text as correct data (teacher label) (step S13).

  Subsequently, the information processing apparatus 10 inputs a title (third text) to the first model and generates a pseudo heading (fourth text) (step S14). At this time, the title in the first model is a title for which a heading has not yet been generated. Then, the information processing apparatus 10 registers the generated pseudo heading in the learning data database 121.

  Subsequently, the information processing apparatus 10 acquires a combination of a title (third text) and a pseudo heading (fourth text) from learning data registered in the learning data database 121 (step S15). Then, the information processing apparatus 10 generates a predetermined model by learning using the third text as input data and the fourth text as correct data (teacher label) (step S16). The predetermined model may be the first model (model M1) itself, or may be a second model (model M2) different from the first model.

  When the execution of learning ends, the information processing apparatus 10 ends the learning process.

[3-2. Generation process)
Next, the generation process will be described. FIG. 12 is a flowchart illustrating an example of the generation process. The generation process is a process for generating output text (short text of input text) from input text. When the information processing apparatus 10 receives a process start command from the user, the information processing apparatus 10 executes a generation process.

  First, the information processing apparatus 10 acquires content information (step S21). For example, the information processing apparatus 10 acquires information on distribution contents (text and title) from the data server 20.

  Subsequently, the information processing apparatus 10 selects an input text to be input to the second model (model M2) from the data acquired in step S21 (step S22). For example, the information processing apparatus 10 selects a title as input text.

  Then, the information processing apparatus 10 generates the output text by inputting the input text to the second model (step S23). When the generation of the output text is completed, the information processing apparatus 10 ends the generation process.

[4. (Modification)
The above-described embodiment shows an example, and various changes and applications are possible.

  For example, in the above-described embodiment, the information processing apparatus 10 learns the model M1 using the first text as the title and the second text as the heading (short text of the title). However, the information processing apparatus 10 may learn the model M1 using the first text as a body and the second text as a heading (short text of the body or title). Further, in the above-described embodiment, the information processing apparatus 10 learns the model M2 using the third text as a title and the fourth text as a heading (short text of the title). However, the information processing apparatus 10 may learn the model M2 using the third text as the body and the fourth text as the heading (short text of the body or title). At this time, the information processing apparatus 10 may generate output text as input text that is input to the model M2.

  In addition, the text used as the first text, the second text, the third text, and the fourth text can be arbitrarily changed. In this case, the fourth text corresponding to the third text can be paraphrased as pseudo text. For example, it is assumed that the predetermined model is a model that generates a title from the text. It is assumed that the third text is a body text. In this case, the fourth text generated by inputting the third text into the predetermined model is a pseudo title.

  Further, in the above-described embodiment, the information processing apparatus 10 inputs the third text (for example, a title) to the model M1 (first model), whereby the fourth text (corresponding to the third text ( For example, a pseudo heading) was generated. Then, the information processing apparatus 10 learns a model M2 (second model) that generates output text from input text based on the third text and the fourth text. However, the information processing apparatus 10 may generate the fifth text corresponding to the third text by inputting the third text into the second model. Then, the information processing apparatus 10 may learn a model M3 (third model) that generates output text from input text based on the third text and the fifth text.

  Hereinafter, learning of the model M3 will be described with reference to FIG. FIG. 13 is a diagram illustrating another example of processing executed by the information processing apparatus 10 according to the embodiment. Note that steps S102 to S104 shown in FIG. 13 are the same as steps S102 to S104 shown in FIG. When the learning of the model M2 is completed (step S104), the information processing apparatus 10 inputs the title (third text) of the content belonging to the group G2 to the model M2, thereby obtaining a pseudo heading (first item) corresponding to the input title. 5 text). Note that if there are a plurality of acquired titles, the information processing apparatus 10 inputs a plurality of pseudo headings to the model M2 and generates a plurality of pseudo headings (step S201).

  Then, the information processing apparatus 10 learns the model M3 based on the title (third text) and pseudo heading (fifth text) of the content belonging to the group G2 (step S202). Here, the model M3 is a model that generates output text from input text, similarly to the models M1 and M2. The model M2 may have the same configuration as the models M1 and M2, or may have a different configuration.

  Note that the information processing apparatus 10 not only includes the title (third text) and pseudo heading (fifth text) of content belonging to the group G2, but also the title (first text) and heading (first text) of the content belonging to the group G1. The second text) may be a text for learning the model M3.

  In step S201, the information processing apparatus 10 generates a plurality of pseudo headings, but it is not always necessary to use all of the generated pseudo headings as learning data. The information processing apparatus 10 selects, as learning data, a pseudo heading (fifth text) selected according to a predetermined criterion from the plurality of pseudo headings (a plurality of fifth texts) generated in step S201. Also good. At this time, the information processing apparatus 10 selects a pseudo heading used for learning of the model M3 based on a score (likelihood) that is calculated based on the learning result of the model M2 and indicates the likelihood of the pseudo heading. May be.

  Thereafter, the information processing apparatus 10 generates an output text (for example, a heading) corresponding to the input text by inputting the input text (for example, a title) to the model M3 (step S203).

  The information processing apparatus 10 may generate the fourth, fifth,..., Nth models by repeatedly executing Step S201 and Step S202. At this time, N is an integer. When step S201 and step S202 are repeated, the model M2 in step S201 is replaced with the model M3 in step S202. Then, the model M3 in step S202 is replaced with a new model.

  For example, it is assumed that the model M2 in step S201 is the second model and the model M3 in step S202 is the third model in this round. At this time, the information processing apparatus 10 replaces the model M2 in step S201 with the third model and the model M3 in step S202 with the fourth model in a new lap. Thereby, the information processing apparatus 10 can increase the accuracy of the model every time the model generation is repeated.

  In the above-described embodiment, the generation unit 134 generates the fourth text corresponding to the third text by inputting the third text to the first model. After that, the learning unit 132 learned the second model that generates the output text from the input text based on the third text and the fourth text. At this time, the “first model” used by the generation unit 134 can be replaced with the (N−1) th model. In addition, the “second model” used by the learning unit 132 can be replaced with the Nth model. At this time, N is an integer. For example, the generation unit 134 generates the fourth text corresponding to the third text by inputting the third text to the third model. Thereafter, the learning unit 132 learns the fourth model that generates the output text from the input text based on the third text and the fourth text.

  It is also possible to regard the (N-1) th model as the first model and the Nth model as the second model. For example, the model M2 can be regarded as a first model, and the model M3 can be regarded as a second model. The models regarded as the first model and the second model can be arbitrarily changed.

  Note that the second model may be a model in which the first model is made as complex as data increases. Specifically, the structure of the second model perceptron may be more complex than the structure of the first model perceptron. For example, the structure of the second model perceptron may be a structure in which at least one of the number of intermediate layers and the number of neurons is larger than that of the first model perceptron.

  The second model may be a smaller model than the first model by compression or distillation of the first model. Specifically, the structure of the second model perceptron may be smaller than the structure of the first model perceptron. For example, the structure of the second model perceptron may be a structure in which at least one of the number of intermediate layers and the number of neurons is smaller than the structure of the first model perceptron.

  Further, in the above-described embodiment, the encoder and the decoder configuring the model M are configured by LSTM, but may be configured by RNN other than LSTM. The encoder and decoder that constitute the model M may be a neural network other than the RNN, such as a CNN (Convolution Neural Network). In addition, the encoder and decoder constituting the model M may be a neural network that simply changes the number of dimensions of the input information amount. At this time, the encoders constituting the model M may be configured to extract features of the input information by compressing the dimensionality of the input information. In addition, the decoder constituting the model M may be configured to increase the dimension amount of the feature extracted by the encoder and output information having a smaller number of dimensions than the information input to the encoder.

  The control device that controls the information processing apparatus 10 according to the present embodiment may be realized by a dedicated computer system or an ordinary computer system. For example, a program or data (for example, model M) for executing the above-described operation is stored and distributed in a computer-readable recording medium such as an optical disk, a semiconductor memory, a magnetic tape, or a flexible disk, and the program is distributed to the computer. The control device may be configured by installing the software and executing the above-described processing. The control device may be an external device (for example, a personal computer) of the information processing device 10 or an internal device (for example, the control unit 13). Further, the program may be stored in a disk device provided in a server device on a network such as the Internet so that it can be downloaded to a computer. Further, the above-described functions may be realized by cooperation between an OS (Operating System) and application software. In this case, a part other than the OS may be stored and distributed in a medium, or a part other than the OS may be stored in a server device and downloaded to a computer.

  In addition, among the processes described in the above embodiments, all or a part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed All or a part of the above can be automatically performed by a known method. In addition, the processing procedures, specific names, and information including various data and parameters shown in the document and drawings can be arbitrarily changed unless otherwise specified. For example, the various types of information illustrated in each drawing is not limited to the illustrated information.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured.

  Moreover, each embodiment mentioned above can be combined suitably in the range which does not contradict a process content.

[5. Hardware configuration)
The information processing apparatus 10 according to the embodiment and the modification may be realized by a computer 1000 having a configuration as illustrated in FIG. FIG. 14 is a hardware configuration diagram illustrating an example of a computer that implements the functions of the information processing apparatus 10. The computer 1000 includes a CPU (Central Processing Unit) 1100, a RAM 1200, a ROM 1300, an HDD (Hard Disk Drive) 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F). ) 1700.

  The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400 and controls each unit. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started up, a program depending on the hardware of the computer 1000, and the like.

  The HDD 1400 stores programs executed by the CPU 1100, data used by the programs, and the like. The communication interface 1500 receives data from other devices via the network N and sends the data to the CPU 1100, and transmits data generated by the CPU 1100 to other devices via the network N.

  The CPU 1100 controls an output device such as a display and a printer and an input device such as a keyboard and a mouse via an input / output interface 1600. The CPU 1100 acquires data from the input device via the input / output interface 1600. In addition, the CPU 1100 outputs the generated data to the output device via the input / output interface 1600.

  The media interface 1700 reads a program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads such a program from the recording medium 1800 onto the RAM 1200 via the media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase change rewritable disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. Etc.

  For example, when the computer 1000 functions as the information processing apparatus 10 according to the embodiment, the CPU 1100 of the computer 1000 implements the control unit 13 by executing a program or data (for example, model M) loaded on the RAM 1200. To do. The CPU 1100 of the computer 1000 reads these programs or data (for example, model M) from the recording medium 1800 and executes them, but as another example, these programs or data (for example, for example, from other devices via the network N). Model M) may be obtained.

  As described above, some of the embodiments of the present application have been described in detail with reference to the drawings. However, these are merely examples, and various modifications based on the knowledge of those skilled in the art, including the aspects described in the disclosure line of the invention. It is possible to implement the present invention in other forms with improvements.

[6. effect〕
According to the present embodiment, the information processing apparatus 10 uses the model generated by learning based on the relationship between the first text and the second text corresponding to the first text, from the third text. A fourth text corresponding to the third text is generated. Then, the information processing apparatus 10 performs learning based on the relationship between the third text and the fourth text. Accordingly, the information processing apparatus 10 can acquire a highly accurate model because the set of the third text (for example, title) and the fourth text (for example, pseudo heading) is also used as learning data. By using the acquired model, the information processing apparatus 10 can generate a highly accurate output text.

  The information processing apparatus 10 learns recursively based on the relationship between the third text and the fourth text. Thereby, since the information processing apparatus 10 can acquire a highly accurate model, it is possible to generate an output text with high accuracy.

  The information processing apparatus 10 generates a second model different from the model by learning the relationship between the third text and the fourth text. Thereby, since the information processing apparatus 10 can acquire a highly accurate model, it is possible to generate an output text with high accuracy.

  The structure of the second model perceptron generated based on the relationship between the third text and the fourth text is the model perceptron generated based on the relationship between the first text and the second text. It is a structure in which at least one of the number of intermediate layers and the number of neurons is larger than the structure of Thereby, since the information processing apparatus 10 can acquire a highly accurate model, it is possible to generate an output text with high accuracy.

  The structure of the second model perceptron generated based on the relationship between the third text and the fourth text is the model perceptron generated based on the relationship between the first text and the second text. It is a structure in which at least one of the number of intermediate layers and the number of neurons is smaller than that of the structure. Thereby, since the information processing apparatus 10 can acquire a highly accurate model, it is possible to generate an output text with high accuracy.

  The information processing apparatus 10 generates the second model by learning based on the relationship between the first text and the second text and the relationship between the third text and the fourth text. Thereby, since the information processing apparatus 10 can acquire a highly accurate model, it is possible to generate an output text with high accuracy.

  The information processing apparatus 10 generates a plurality of fourth texts by inputting a plurality of third texts into the model. The information processing apparatus 10 learns based on the relationship between the fourth text selected from the plurality of fourth texts according to a predetermined criterion and the third text corresponding to the selected fourth text. To generate a second model. Thereby, since the information processing apparatus 10 can acquire a highly accurate model, it is possible to generate an output text with high accuracy.

  The information processing apparatus 10 learns from a plurality of fourth texts generated by the generation unit based on a score calculated based on the learning result of the model and indicating the likelihood of the fourth text. Select the fourth text to use for. Then, learning is performed based on the selected fourth text and the third text corresponding to the selected fourth text. Thereby, since the information processing apparatus 10 can acquire a highly accurate model, it is possible to generate an output text with high accuracy.

  The information processing apparatus 10 generates the fifth text corresponding to the third text by inputting the third text to the second model. Then, the information processing apparatus 10 learns a third model that generates output text from input text based on the third text and the fifth text. Thereby, since the information processing apparatus 10 can acquire a highly accurate model, it is possible to generate an output text with high accuracy.

  The information processing apparatus 10 generates the third model by learning based on the relationship between the first text and the second text and the relationship between the third text and the fifth text. Thereby, since the information processing apparatus 10 can acquire a highly accurate model, it is possible to generate an output text with high accuracy.

  The information processing apparatus 10 generates a plurality of fifth texts by inputting a plurality of third texts to the second model. Then, the information processing apparatus 10 learns based on the relationship between the fifth text selected from the plurality of fifth texts according to a predetermined criterion and the third text corresponding to the selected fifth text. To generate a third model. Thereby, since the information processing apparatus 10 can acquire a highly accurate model, it is possible to generate an output text with high accuracy.

  The model includes an encoder that generates feature information indicating features of the input text from the input text, and a decoder that generates output text from the feature information. Thereby, since the information processing apparatus 10 can acquire a highly accurate model, it is possible to generate an output text with high accuracy.

  The information processing apparatus 10 generates a pseudo heading corresponding to the third text from the third text using a model generated by learning based on the first text and the heading corresponding to the first text. Then, the information processing apparatus 10 performs learning based on the relationship between the third text and the pseudo headline corresponding to the third text. Thereby, since the information processing apparatus 10 can acquire a highly accurate model, it is possible to generate an output text with high accuracy.

  The information processing apparatus 10 acquires an input text. Then, the information processing apparatus 10 uses the model generated by learning based on the relationship between the third text and the first text and the second text corresponding to the first text from the third text. An output text corresponding to the input text is generated from the input text by using a predetermined model generated by learning based on the relationship with the generated fourth text. Thereby, the information processing apparatus 10 can acquire a highly accurate output text.

  The information processing apparatus 10 uses the model generated by learning based on the relationship between the third text and the first text and the heading corresponding to the first text, and the pseudo heading generated from the third text. The headline corresponding to the input text is generated from the input text using a predetermined model generated by learning based on the relationship between Thereby, the information processing apparatus 10 can acquire a headline with high accuracy.

  The information processing apparatus 10 corresponds to the third text by inputting the third text to the first model learned based on the first text and the second text corresponding to the first text. Generate fourth text. Then, the information processing apparatus 10 learns the second model that generates the output text from the input text based on the third text and the fourth text. Accordingly, the information processing apparatus 10 can acquire a highly accurate model because the set of the third text (for example, title) and the fourth text (for example, pseudo heading) is also used as learning data. By using the acquired second model, the information processing apparatus 10 can generate an output text with high accuracy.

  The information processing apparatus 10 learns the second model based on the first text, the second text, the third text, and the fourth text. As a result, the information processing apparatus 10 uses not only the set of the third text and the fourth text but also the set of the first text and the second text as learning data. Can be obtained. As a result, the information processing apparatus 10 can generate highly accurate output text.

  The information processing apparatus 10 generates a plurality of fourth texts by inputting a plurality of third texts to the first model. The information processing apparatus 10 then selects the second text based on the fourth text selected from the plurality of fourth texts according to the predetermined criterion and the third text corresponding to the selected fourth text. The model is learned. As a result, the information processing apparatus 10 does not simply use the plurality of generated fourth texts as learning data, but uses text selected from the plurality of fourth texts as learning data. Therefore, a highly accurate model can be acquired. As a result, the information processing apparatus 10 can generate highly accurate output text.

  The information processing apparatus 10 calculates a score calculated based on the learning result of the first model, and is based on the first score indicating the likelihood of the fourth text. To select a fourth text to be used for learning the second model. Then, the information processing apparatus 10 learns the second model based on the selected fourth text and the third text corresponding to the selected fourth text. Thereby, since the information processing apparatus 10 uses the pseudo text having the highest likelihood among the plurality of pseudo texts as the learning data, the information processing apparatus 10 can acquire a highly accurate model. As a result, the information processing apparatus 10 can generate highly accurate output text.

  The information processing apparatus 10 generates the fifth text corresponding to the third text by inputting the third text to the second model. Then, the information processing apparatus 10 learns a third model that generates output text from input text based on the third text and the fifth text. As a result, the information processing apparatus 10 uses the pseudo-text generated by using the second model whose accuracy is improved from the first model as learning data, so that a model with higher accuracy can be acquired. As a result, the information processing apparatus 10 can generate highly accurate output text.

  The information processing apparatus 10 learns the third model based on the first text, the second text, the third text, and the fifth text. As a result, the information processing apparatus 10 uses not only the set of the third text and the fifth text but also the set of the first text and the second text as learning data. Can be obtained. As a result, the information processing apparatus 10 can generate highly accurate output text.

  The information processing apparatus 10 generates a plurality of fifth texts by inputting a plurality of third texts to the second model. The information processing apparatus 10 uses the third model based on the fifth text selected according to a predetermined criterion from the plurality of fifth texts and the third text corresponding to the selected fifth text. To learn. Thereby, since the information processing apparatus 10 uses the pseudo text having the highest likelihood among the plurality of pseudo texts as the learning data, the information processing apparatus 10 can acquire a highly accurate model. As a result, the information processing apparatus 10 can generate highly accurate output text.

  The information processing apparatus 10 learns a model including an encoder that generates feature information indicating features of the input text from the input text, and a decoder that generates output text from the feature information. Thereby, the information processing apparatus 10 can acquire a highly accurate model. As a result, the information processing apparatus 10 can generate highly accurate output text.

  The information processing apparatus 10 inputs the third text into the first model learned based on the first text and the heading corresponding to the first text, so that a pseudo heading corresponding to the third text is obtained. Generate. Then, the information processing apparatus 10 learns the second model that generates the heading corresponding to the input text from the input text based on the third text and the pseudo heading corresponding to the third text. Thereby, the information processing apparatus 10 can acquire a highly accurate model for generating a headline. By using the acquired model, the information processing apparatus 10 can generate a headline with high accuracy.

  The information processing apparatus 10 is generated by inputting the third text into the first model learned based on the third text and the first text and the second text corresponding to the first text. The input text input to the second model learned based on the fourth text is acquired. Then, the information processing apparatus 10 generates an output text corresponding to the input text by inputting the input text to the second model. Thereby, the information processing apparatus 10 can acquire a highly accurate output text.

  The information processing apparatus 10 generates a pseudo heading generated by inputting the third text into the first model learned based on the third text and the first text and the heading corresponding to the first text. And the input text input into the 2nd model learned based on these is acquired. Then, the information processing apparatus 10 generates a headline corresponding to the input text by inputting the input text to the second model. Thereby, the information processing apparatus 10 can acquire a headline with high accuracy.

  As described above, some of the embodiments of the present application have been described in detail with reference to the drawings. However, these are merely examples, and various modifications, including the aspects described in the disclosure section of the invention, based on the knowledge of those skilled in the art, It is possible to implement the present invention in other forms with improvements.

  In addition, the “section (module, unit)” described above can be read as “means” or “circuit”. For example, the learning unit can be read as learning means or a learning circuit.

DESCRIPTION OF SYMBOLS 10 ... Information processing apparatus 11 ... Communication part 12 ... Storage part 121 ... Learning data database 122 ... Model database 123 ... Content information database 13 ... Control part 131 ... Learning data acquisition part 132 ... Learning part 133 ... Learning data generation part 134 ... Generation Unit 135 ... Output control unit 20 ... Data server 30 ... Terminal device M1, M2, M3 ... Model E1 ... Encoder D1 ... Decoder F ... Feature information P ... Classification information

Claims (20)

The fourth text corresponding to the third text from the third text using the model generated by learning based on the relationship between the first text and the second text corresponding to the first text A generating unit for generating
A learning unit that learns based on the relationship between the third text and the fourth text;
A learning apparatus comprising:   The learning device according to claim 1, wherein the learning unit learns recursively based on a relationship between the third text and the fourth text.   The learning device according to claim 1, wherein the learning unit generates a second model different from the model by learning a relationship between the third text and the fourth text. .   The structure of the second model perceptron generated based on the relationship between the third text and the fourth text is generated based on the relationship between the first text and the second text. The learning device according to claim 3, wherein the learning device has a structure in which at least one of the number of intermediate layers and the number of neurons is larger than the structure of the perceptron of the model to be used.   The structure of the second model perceptron generated based on the relationship between the third text and the fourth text is generated based on the relationship between the first text and the second text. The learning apparatus according to claim 3, wherein at least one of the number of intermediate layers and the number of neurons is smaller than the structure of the perceptron of the model to be used. The learning unit generates a second model by learning based on a relationship between the first text and the second text and a relationship between the third text and the fourth text. ,
The learning apparatus according to claim 1, wherein the learning apparatus is a learning apparatus. The generation unit generates a plurality of the fourth texts by inputting a plurality of the third texts into the model,
The learning unit is based on a relationship between the fourth text selected from a plurality of the fourth texts according to a predetermined criterion and the third text corresponding to the selected fourth texts. Generating a second model by learning;
The learning device according to claim 1, wherein the learning device is a learning device. The learning unit
Based on the score calculated based on the learning result of the model and indicating the likelihood of the fourth text, it is used for learning from among the plurality of fourth texts generated by the generation unit. Select the fourth text;
Learning based on the selected fourth text and the third text corresponding to the selected fourth text;
The learning device according to claim 7. The generation unit generates a fifth text corresponding to the third text by inputting the third text to the second model,
The learning unit learns a third model that generates output text from input text based on the third text and the fifth text.
The learning apparatus according to claim 3, wherein The learning unit generates a third model by learning based on the relationship between the first text and the second text and the relationship between the third text and the fifth text. ,
The learning apparatus according to claim 9. The generation unit generates a plurality of the fifth texts by inputting a plurality of the third texts into the second model,
The learning unit is based on a relationship between the fifth text selected from a plurality of the fifth texts according to a predetermined criterion and the third text corresponding to the selected fifth texts. Generating the third model by learning;
The learning device according to claim 9 or 10, characterized in that: The model includes an encoder that generates feature information indicating features of the input text from the input text, and a decoder that generates output text from the feature information.
The learning apparatus according to claim 1, wherein the learning apparatus is a learning apparatus. The generation unit uses the model generated by learning based on the first text and a heading corresponding to the first text, to generate a pseudo heading corresponding to the third text from the third text. Produces
The learning unit learns based on a relationship between the third text and the pseudo headline corresponding to the third text;
The learning device according to claim 1, wherein the learning device is a learning device. An acquisition unit for acquiring the input text;
A fourth text generated from the third text using a model generated by learning based on the relationship between the third text, the first text, and the second text corresponding to the first text. A generation unit that generates an output text corresponding to the input text from the input text using a predetermined model generated by learning based on the relationship between the text and the text;
A generating apparatus comprising: The generation unit is generated from the third text using the model generated by learning based on the relationship between the third text and the relationship between the first text and the heading corresponding to the first text. Generating a heading corresponding to the input text from the input text using the predetermined model generated by learning based on the relationship between the pseudo heading and
The generating apparatus according to claim 14. A learning method executed by a learning device,
The fourth text corresponding to the third text from the third text using the model generated by learning based on the relationship between the first text and the second text corresponding to the first text A generating step for generating
And a learning step of performing learning based on the relationship between the third text and the fourth text. A generation method executed by a generation device,
An acquisition process for acquiring the input text;
A fourth text generated from the third text using a model generated by learning based on the relationship between the third text, the first text, and the second text corresponding to the first text. A generation step of generating an output text corresponding to the input text from the input text using a predetermined model generated by learning based on the relationship between the text and the text;
A generation method comprising: The fourth text corresponding to the third text from the third text using the model generated by learning based on the relationship between the first text and the second text corresponding to the first text A generation procedure for generating
A learning procedure for performing learning based on the relationship between the third text and the fourth text;
A learning program to make a computer execute. An acquisition procedure for acquiring the input text;
A fourth text generated from the third text using a model generated by learning based on the relationship between the third text, the first text, and the second text corresponding to the first text. A generation procedure for generating an output text corresponding to the input text from the input text using a predetermined model generated by learning based on the relationship between the text and the text;
A generation program that causes a computer to execute. The third text is generated from the third text using a predetermined model generated by learning based on the relationship between the third text, the first text, and the second text corresponding to the first text. A model generated by learning based on 4 texts,
An input layer where the text is entered,
An output layer for outputting text corresponding to the text input to the input layer;
A first element belonging to any layer from the input layer to the output layer other than the output layer;
A second element whose value is calculated based on the first element and a weight of the first element;
According to the text input to the input layer, the text corresponding to the text input to the input layer is output from the output layer.
A model for making a computer function.

Images