JP2019046158A - Text generation device, text generation method and text generation program - Google Patents

Abstract

To provide a text generation device, a text generation method, and a text generation program for generating a text that explains variation of time series numerical data.SOLUTION: A text generation device for generating text data for explaining fluctuation of time series numerical data including a number sequence associated with the passage of time includes: a learning unit that causes a language model to learn a language model so as to output replacement text data when the time-series numerical data is input, using replacement text data and time-series numerical data as learning data; a generation unit that inputs new time series numerical data into the language model learned by the learning unit, and generates new replacement text data that explains the new time series numerical data by an output of the language model; and a replacement unit that replaces a predetermined character string included in the new replacement text data with a numerical value related to the new time series numerical data according to a predetermined rule.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a text generation device, a text generation method, and a text generation program.

  In recent years, in the field of natural language processing, language models using neural networks such as recurrent neural networks have been studied.

  For example, in Patent Document 1 below, a recognition result acquisition unit that acquires, from a first database, a word recognized from an interactive text, time series information of the word, and identification information that identifies the speaker of the word. A dialog text summarizing device is described, comprising a text summarization unit for correcting words based on words and word time-series information, identification information and a summary model, and outputting the correction result to a first database.

JP, 2017-111190, A

  A language model using a neural network may be trained to generate text based on statistical features of words appearing in training data, using a large amount of text data as training data.

  However, in the case of generating a text that explains the fluctuation of time-series numerical data (for example, stock price etc.) including a series of numerical values that change with the passage of time, the learning data describes the quotation of the time-series numerical data If it is a stock price, it may include opening price, closing price, raising range, etc., and the numerical value associated with the explanation will change in various ways, so each numerical value will be a word that appears only statistically infrequently . Therefore, it was difficult to learn the language model so as to correctly reproduce the description about numerical values, and it was difficult to generate texts that explain the fluctuation of time-series numerical data.

  Therefore, the present invention provides a text generation device, a text generation method, and a text generation program for generating text that explains variation of time series numerical data.

  A text generation device according to an aspect of the present invention is a text generation device that generates text data for explaining fluctuation of time series numerical data including a number sequence associated with the passage of time, and is a time series of text data. The substitution text data is output when the time-series numerical data is input, with the substitution text data in which the numerical value related to the numerical data is replaced with the predetermined character string according to the predetermined rule and the time series numerical data as the learning data. Thus, new time-series numerical data is input to the learning unit for learning the language model and the language model learned by the learning unit, and new substituted text data for explaining the new time-series numerical data by the output of the language model The generation unit to generate and a predetermined character string included in the new replacement text data are related to the new time-series numerical data according to a predetermined rule Comprising a replacement unit for replacing the numerical value, the.

  According to this aspect, new substitution text data describing new time series numerical data is generated by the output of the language model, and a predetermined character string included in the new substitution text data is newly added according to a predetermined rule. By replacing the numerical values related to the series numerical data, it is not necessary to directly output the numerical values related to the time series numerical data by the language model, and even if the numerical values change variously, the description regarding the numerical values can be correctly included. Text can be generated to account for variations in time series numerical data.

  In the above aspect, the numerical values related to the time-series numerical data are associated with the numerical value associated with the predetermined point in time series among the numerical series included in the time-series numerical data and different time points in the numerical series included in the time series numerical data Corresponds to the difference between the numerical values obtained by subtracting the numerical value associated with a predetermined point in the time series numerical data included in the time series numerical data by a predetermined digit, and the numerical time series included in the time series numerical data A numerical value obtained by rounding off the difference between the attached numerical values by a predetermined digit, a numerical value obtained by rounding up the numerical value associated with a predetermined point in the time series numerical data included in time series numerical data by a predetermined digit, and time series numerical data The predetermined rule includes at least one of numerical values obtained by rounding up the difference between the numerical values associated with different time points among the contained sequences by a predetermined digit, and the predetermined rule is a numerical value related to time series numerical data It may be a rule for associating class and a predetermined character string.

  According to this aspect, by correlating the kind of numerical value related to the time series numerical data with the predetermined character string, it is possible to include the numerical value obtained as a result of quoting the time series numerical data and calculating the time series numerical data, Text can be generated to account for variations in time series numerical data.

  In the above aspect, the time-series numerical data is associated with the first time-series numerical data including a number sequence associated with the passage of time at the first interval, and with the passage of time at the second interval longer than the first interval. And second time-series numerical data including a number sequence.

  According to this aspect, by using time-series numerical data including a number sequence associated with the passage of time at different time intervals, time-series numerical data can be correctly included so that words depending on the history of time-series numerical data are correctly included. Text can be generated to account for variations in

  In the above aspect, the generation unit inputs, to the language model, one or more numerical data obtained by converting the time-series numerical data by the one or more methods corresponding to the one or more methods in a one-to-one manner. It is also good.

  According to this aspect, by inputting one or more numerical data obtained by converting the time-series numerical data by one or more methods into the language model, the generated text becomes an absolute value of the time-series numerical data. It is prevented from relying on it.

  In the above aspect, the one or more numerical data are obtained by using numerical data obtained by normalizing the numerical sequence included in the time series numerical data into a predetermined numerical range, and using the average value and the standard deviation of the numerical sequence included in the time series numerical data Numerical data obtained by standardizing time-series numerical data and numerical values associated with predetermined time points among numerical sequences contained in time-series numerical data are made relative to reference values with reference to the numerical value associated with time-series numerical data It may include at least one of numerical data.

  According to this aspect, using the normalized numerical data or the standardized numerical data prevents the generated text from being blurred depending on the absolute value of the time series numerical data, and uses the relative numerical data. Thus, it is possible to generate text that describes the variation of time-series numerical data so as to correctly include words that depend on the history of time-series numerical data.

  In the above aspect, the language model is obtained by converting the first time series numerical data including the number sequence associated with the passage of time at the first interval by one or more methods, in one-to-one correspondence to one or more methods. , One or more second time-series numerical data including a first encoder to which first numerical data corresponding to one or more numerical data is input, and a number sequence associated with the passage of time at a second interval longer than the first interval. And combining the output of the first encoder and the output of the second encoder with the second encoder to which the one or more second numerical data corresponding to the one or more methods obtained by conversion by the method of And a decoder for receiving the data synthesized by the synthesizing unit and outputting replacement text data.

  According to this aspect, time series numerical data including time series numerical data including numerical sequences associated with the passage of time at different time intervals are input to different encoders, and an output is synthesized and input to the decoder. Texts can be generated that account for variations in time series numerical data so as to correctly include history dependent words.

  In the above aspect, the combining unit may combine the output of the first encoder, the output of the second encoder, one or more first numerical data, and one or more second numerical data.

  According to this aspect, by inputting not only the output of the encoder but also a plurality of numerical data to the decoder, the fluctuation of the time-series numerical data so that the word depending on the history of the time-series numerical data is correctly included Can be generated to describe the

  In the above aspect, the data combined with the data combined by the combining unit and the time series of the time series numerical data may be input to the decoder.

  According to this aspect, not only the data combined by the combining unit but also data relating to the time series of time series numerical data is correctly input to the decoder, thereby correctly including the word depending on the history of the time series numerical data Thus, text can be generated to account for variations in time series numerical data.

  According to another aspect of the present invention, there is provided a method of generating text by a computer including a hardware processor and a memory, the method generating text data that describes variation in time series numerical data including a number sequence associated with the passage of time. In the method, time series numerical data is input by using, as learning data, replacement text data in which numerical values related to time series numerical data in text data are replaced with predetermined character strings according to predetermined rules, and time series numerical data. Training the language model to output substitution text data, and inputting new time series numerical data into the learned language model, and explaining the new time series numerical data by the output of the language model Generating new replacement text data, and predetermined characters included in the new replacement text data And it executes, and replacing the numerical value related to the new time-series numerical data with a predetermined rule.

  According to this aspect, new substitution text data describing new time series numerical data is generated by the output of the language model, and a predetermined character string included in the new substitution text data is newly added according to a predetermined rule. By replacing the numerical values related to the series numerical data, it is not necessary to directly output the numerical values related to the time series numerical data by the language model, and it is possible to correctly include the description regarding the numerical values even when the numerical values change variously. In addition, text can be generated that describes the variation of time series numerical data.

  According to another aspect of the present invention, there is provided a text generation program comprising: a computer provided in a text generation device for generating text data for explaining fluctuation of time series numerical data including a numerical sequence associated with the passage of time; When time-series numerical data is input using, as learning data, replacement text data in which a numerical value related to time-series numerical data is replaced with a predetermined character string according to a predetermined rule, and time series numerical data as learning data A learning unit for learning a language model so as to output data, a new time-series numerical data input to the language model learned by the learning unit, and a new replacement for explaining a new time-series numerical data by an output of the language model A generation unit for generating text data, and a predetermined character string included in new replacement text data are newly added according to a predetermined rule. Replacement unit for replacing the numerical value relating to the time series numeric data, to function as a.

  According to this aspect, new substitution text data describing new time series numerical data is generated by the output of the language model, and a predetermined character string included in the new substitution text data is newly added according to a predetermined rule. By replacing the numerical values related to the series numerical data, it is not necessary to directly output the numerical values related to the time series numerical data by the language model, and it is possible to correctly include the description regarding the numerical values even when the numerical values change variously. In addition, text can be generated that describes the variation of time series numerical data.

  According to the present invention, it is possible to provide a text generation device, a text generation method, and a text generation program for generating text that explains variation of time series numerical data.

It is a figure showing the network composition of the text generation device concerning the embodiment of the present invention. It is a figure showing physical composition of a text generation device concerning this embodiment. It is a figure which shows the functional block of the text generation apparatus which concerns on this embodiment. It is a figure showing composition of a language model. It is a figure which shows the rule which matches the kind of numerical value related to time series numerical data, and a predetermined | prescribed character string. It is a flowchart of the process performed with the text generation apparatus which concerns on this embodiment. It is a figure which shows the text produced | generated by the text generation apparatus which concerns on this embodiment. It is a figure which shows the index value which evaluated the proximity of the text produced | generated with the text generation apparatus which concerns on this embodiment, and the text used as a reference | standard.

  Embodiments of the present invention will be described with reference to the accompanying drawings. In addition, what attached the same code | symbol in each figure has the same or same structure.

  FIG. 1 is a diagram showing a network configuration of a text generation device 10 according to an embodiment of the present invention. In the present embodiment, the text generation system 100 stores an initial data set including time series numerical data including a number sequence associated with the passage of time, and text data describing variation of the time series numerical data. The language model 20 correctly describes the fluctuation of the time-series numerical data by using the language model 20 which outputs substitution text data according to the input time-series numerical data and the initial data set stored in the database DB. And a text generating device 10 for learning the language model 20 so that text is generated, and generating text data for explaining fluctuations of the time-series numerical data when new time-series numerical data is acquired. In the present embodiment, the time-series numerical data is a stock price. However, the time-series numerical data includes a number sequence associated with the passage of time, and may be any numerical data continuously acquired, for example, electrocardiogram data or It may be vital data such as blood pressure data, weather data such as temperature and humidity, or traffic data such as traffic volume and number of passengers.

  The text generation system 100 is connected to the communication network N, acquires stock prices from the stock price distribution server 40 at predetermined time intervals, stores the stock prices in the database DB, and inputs the stock prices into the text generation apparatus 10. Also, the text generation system 100 provides the generated text data to the user terminal 30 via the communication network N. In addition, the text generation system 100 may add or edit the initial data set stored in the database DB or learn the language model 20 based on an instruction from the user terminal 30. Here, the communication network N is a wired or wireless communication network, and may be, for example, the Internet or a LAN (Local Area Network). In the text generation system 100, all or some of the components may be configured by a remote computer in the form of so-called cloud computing, but all or some of the components may be configured by a local computer.

  In the language model 20, when time series numerical data is input, replacement text data in which numerical values related to time series numerical data among text data describing time series numerical data are replaced with a predetermined character string according to a predetermined rule Is a model that outputs Here, the numerical value related to the time-series numerical data is citation of the numerical value included in the time-series numerical data, or a numerical value obtained by calculating the numerical value included in the time-series numerical data. The language model 20 may be, for example, a model using a neural network, and may be a so-called encoder-decoder model. The language model 20 may include, for example, a Multi-Layer Perceptron (MLP), a Convolutional Neural Network (CNN), or a Recurrent Neural Network (RNN) as an encoder, and may include an RNN as a decoder. The language model 20 may be different depending on the type of time-series numerical data to be input. The language model 20 will be described in detail later using FIG.

  FIG. 2 is a diagram showing the physical configuration of the text generation device 10 according to the present embodiment. The text generation device 10 includes a central processing unit (CPU) 10a corresponding to a hardware processor, a random access memory (RAM) 10b corresponding to a memory, a read only memory (ROM) 10c corresponding to a memory, and a communication unit 10d. , An input unit 10e, and a display unit 10f. These components are mutually connected so as to be able to transmit and receive data via a bus.

  The CPU 10a is a control unit that performs control related to the execution of a program stored in the RAM 10b or the ROM 10c, and performs calculation and processing of data. The CPU 10 a is an arithmetic device that executes a program (text generation program) that generates text data using the language model 20. The CPU 10a receives various input data from the input unit 10e and the communication unit 10d, and displays the calculation result of the input data on the display unit 10f or stores it in the RAM 10b or the ROM 10c.

  The RAM 10 b is a storage unit capable of rewriting data, and is formed of, for example, a semiconductor storage element. The RAM 10 b stores programs and data such as applications executed by the CPU 10 a.

  The ROM 10 c is a storage unit that can only read data, and is configured of, for example, a semiconductor storage element. The ROM 10 c stores, for example, programs and data such as firmware.

  The communication unit 10 d is a communication interface that connects the text generation device 10 to the communication network N.

  The input unit 10 e receives an input of data from the user, and includes, for example, a keyboard, a mouse, and a touch panel.

  The display unit 10 f visually displays the calculation result by the CPU 10 a, and is configured of, for example, an LCD (Liquid Crystal Display).

  The text generation program may be stored in a computer-readable storage medium such as the RAM 10 b or the ROM 10 c and provided, or may be provided via the communication network N connected by the communication unit 10 d. In the text generation device 10, the CPU 10a executes the text generation program to realize various functions described with reference to the following drawings. Note that these physical configurations are exemplifications and may not necessarily be independent configurations. For example, the text generation device 10 may include an LSI (Large-Scale Integration) in which the CPU 10a, the RAM 10b, and the ROM 10c are integrated. The text generation device 10 may also include an arithmetic circuit such as a graphics processing unit (GPU), a field-programmable gate array (FPGA), or an application specific integrated circuit (ASIC).

  FIG. 3 is a diagram showing functional blocks of the text generation device 10 according to the present embodiment. The text generation device 10 includes a learning unit 11, an acquisition unit 12, a generation unit 13, a replacement unit 14, and a rule storage unit 15.

  The learning unit 11 learns replacement text data D1 and time-series numerical data D2 in which numerical values relating to time-series numerical data among text data describing fluctuations in time-series numerical data are replaced with predetermined character strings according to predetermined rules. The language model 20 is trained so as to output replacement text data when time-series numerical data is input as the input data. The replacement text data D1 and the time-series numerical data D2 used for learning the language model 20 by the learning unit 11 may be stored in the database DB as an initial data set.

  Here, the numerical values related to the time series numerical data are related to the numerical value associated with a predetermined point in time among the series included in the time series numerical data and the different point in time among the series included in the time series numerical data The difference between the numerical values and the numerical value obtained by rounding off the numerical value associated with the predetermined time point among the numerical sequences included in the time series numerical data to a predetermined digit, and the different time point among the numerical sequences included in the time series numerical data Included in the time-series numerical data is the numerical value obtained by rounding off the difference between the numerical values to the predetermined digit, the numerical value obtained by rounding up the numerical value associated with the predetermined time in the time series numerical data included in the time series numerical data At least one of numerical values obtained by rounding up the difference between the numerical values associated with different time points in the number series may be included. In addition, rounding off or rounding up in a predetermined digit of a numerical value may be performed for any digit, such as the tens digit, the hundreds digit, the thousands digit and the 10000 digits. Also, the predetermined rule may be a rule that associates the type of numerical value related to time series numerical data with a predetermined character string. Here, the predetermined character string may be any character string as long as it is a character string distinguishable from normal text data, and for example, a predetermined symbol such as <price1> or <price2> (in this example, “<< It may be a character string whose beginning and end are indicated by "" and ">".

Further, the time series numerical data includes first time series numerical data including a number sequence associated with the passage of time at a first interval, and a number sequence associated with the passage of time at a second interval longer than the first interval. And second time-series numerical data may be included. In the case of the present embodiment, the first time-series numerical data X _short is time-series numerical data related to stock prices acquired at intervals of 5 minutes from a close of one business day to a large closing, and the second time-series numerical data X _long is time-series numerical data on the closing price of stock prices acquired at intervals of business days for seven business days. That is, the first time series numerical data X _short is time series numerical data including a number sequence associated with the passage of time at five-minute intervals, and the second time series numerical data X _long is time at one business day interval Time series numerical data including a number sequence associated with the passage of In the case of the Tokyo Stock Exchange of Japan, the trading attendance time in one business day is 5 hours (300 minutes), and the first time-series numerical data acquired at 5-minute intervals includes 62 pieces of data. This is represented as X _{short, i} (i = 1 to 62). Also, the second time series numerical data X _long includes seven data, which are expressed as X _{long, j} (j = 1 to 7).

  The acquisition unit 12 acquires new time-series numerical data from the stock price distribution server 40. The acquiring unit 12 may acquire new time-series numerical data on the stock price from the stock price distribution server 40, for example, every five minutes.

  The generation unit 13 inputs new time series numerical data to the language model 20 learned by the learning unit 11, and generates new replacement text data describing the new time series numerical data by the output of the language model 20. The generation unit 13 may input, to the language model 20, one or a plurality of numerical data corresponding to the one or a plurality of methods, which are obtained by converting the time-series numerical data by the one or a plurality of methods. . Here, when one or more numerical data are numerical data obtained by normalizing a numerical sequence included in time series numerical data into a predetermined numerical range, and an average value and a standard deviation of the numerical sequence included in time series numerical data, A numerical value obtained by standardizing serial numerical data and a numerical value corresponding to a predetermined time point among numerical sequences contained in time series numerical data as a reference value, a numerical value obtained by relativizing a numerical sequence contained in time series numerical data with respect to the reference value And / or data may be included.

More specifically, numerical data X _{norm, i obtained} by normalizing a numerical sequence X _i (i = 1 to N) included in time series numerical data into a predetermined numerical range is X _{norm, i} = (2X _i − ( It may be numerical data defined by X _max + X _min )) / (X _max −X _min ). Here, X _max = max _i (X _i ) and X _min = min _i (X _i ). In this case, the normalized numerical data X _{norm, i} is numerical data normalized to a numerical range of -1 to 1.

Also, numerical data X _{std, i obtained} by standardizing a numerical sequence X _i (i = 1 to N) included in time-series numerical data is numerical data defined by X _{std, i} = (X _i- μ) / σ May be there. Here, μ = E [X _i ], σ = (var [X _i ]) ^1/2 .

Moreover, when the sequence X _i included in the numerical data at (i = 1 to N) numerically and relative with respect to the reference value r _i of the data X _{move, i} is defined by _{X move, i = X i -r} i It may be numerical data. When the time-series numerical data is a stock price, the reference value r _i may be the closing price of the previous day. That is, for the first time-series numerical data X _short containing a series of numbers associated with the passage of time at 5-minute intervals, let r be the final value of the previous day and let (X _{short, i} − r The numerical data relativized by) may be calculated. Also. For the second time series numerical data X _long including a number sequence associated with the passage of time at business day intervals, the numerical data relativized by (X _{long, j} − X _{long, j−1} ) may be calculated .

  By using normalized numerical data or standardized numerical data, the generated text is prevented from being dependent on the absolute value of time series numerical data, and by using relative numerical data, time series numerical values can be obtained. Texts can be generated that account for variations in time series numerical data so as to correctly include words that depend on the history of the data.

The substitution unit 14 replaces a predetermined character string included in the new substitution text data with a numerical value related to the new time series numerical data according to a predetermined rule. The substitution unit 14 refers to the predetermined rule stored in the rule storage unit 15 and replaces the predetermined character string included in the new substitution text data with the numerical value related to the new time series numerical data. Replacement unit 14, for example, a predetermined string <price1>, the last value of X _long (X _{long, 7,} the day before the closing price) last value of the X _short (X _{short, 62,} the day closing price) or replaced with a difference, a predetermined character string of <price2>, the last value of X _long (X _{long, 7,} the day before the closing price) last value of the X _short of (X _{short, 62,} of the day closing price) Replace the difference with a value rounded to the nearest tenth.

FIG. 4 is a diagram showing the configuration of the language model 20. As shown in FIG. The language model 20 is obtained by converting the first time-series numerical data X _short including the number sequence associated with the passage of time at the first interval by the first pre-processing unit 21 a in one or more methods, 1 or A first encoder 22a into which one or more first numerical data items l _s corresponding one-to-one to a plurality of methods is input, and a number sequence including a number sequence associated with the passage of time at a second interval longer than the first interval obtain a 2:00 series numerical data X _long converted in one or more ways by the second pre-processing section 21b, one or more one-to-one correspondence to the method one or more second numerical data l _l input a second encoder 22b which is an output h _s and a second combining unit 23 for combining the outputs h _l of the encoder 22b of the first encoder 22a, the data m, which is combined by the combining unit 23 is input, the replacement text data Output And the coder 24.

In this example, the first time-series numerical data X _short is given as a 62-dimensional vector including numerical values such as “12167.29” and “12275.83”. The second time-series numerical data X _long is given as a seven-dimensional vector including numerical values such as "12116.57" and "12120.94". The first pre-processing unit 21a converts the input first time-series numerical data X _short by three methods, and converts the first numerical data l _s by the direct sum of three vectors obtained by conversion. Output. Here, regarding the three types of methods, calculation of numerical data obtained by normalizing the input first time-series numerical data X _short into a predetermined numerical range, calculation of standardized numerical data, and a reference value It is to calculate relative numerical data. In the case of this example, the first numerical data l _s output from the first preprocessing unit 21 a is a 186-dimensional vector.

  As described above, by using time-series numerical data including a number sequence associated with the passage of time at different time intervals, fluctuation of time-series numerical data is made to correctly include a word depending on the history of time-series numerical data. Can be generated to describe the In addition, when one or more numerical data obtained by converting time series numerical data by one or more methods is input to the language model, the generated text may be blurred depending on the absolute value of the time series numerical data. Is prevented.

Similarly, the second pre-processing unit 21b converts the input second time-series numerical data X _long by three methods, and generates second numerical data by the direct sum of three vectors obtained by conversion. l Output _l Here, regarding the three types of methods, calculation of numerical data obtained by normalizing the input second time-series numerical data X _long into a predetermined numerical range, calculation of standardized numerical data, and a reference value It is to calculate relative numerical data. In this example, the second numerical data l _l output from the second pre-processing section 21b is a 21-dimensional vector.

The first numerical data l _s output from the first pre-processing unit 21 a is input to the first encoder 22 a, and the vector h _s is output. Here, the dimension of the vector h _s is the number of output nodes included in the output layer of the first encoder 22 a. Similarly, the second numerical value data l _l output from the second preprocessing unit 21 b is input to the second encoder 22 b, and a vector h _l is output. Here, the dimension of the vector _hl is the number of output nodes included in the output layer of the second encoder 22b. The first encoder 22a and the second encoder 22b may be any of MLP, CNN and RNN, and may be other models.

The combining unit 23 outputs the output h _s of the first encoder 22a, the output h _l of the second encoder 22b, the first numerical data l _s output from the first preprocessing unit 21a, and the output from the second preprocessing unit 21b. These data are synthesized by the direct sum of the second numerical data l _l .

The data m combined by the combining unit 23 and data T regarding the time series of time series numerical data are input to the decoder 24. The data T related to the time series corresponds to the latest time of the times associated with the series included in the first time series numerical data X _short , or the series included in the second time series numerical data X _long corresponds It may be data on the range of attached business days.

In this example, the decoder 24 outputs replacement text data of "Nikkei", "average", ",", "up width", "<price 1>", "yen", "over", "</ s>" doing. Here, the character string “<price1>” output fifth is a predetermined character string to be replaced by the numerical value related to the time-series numerical data by the substitution unit 14 of the text generation device 10. Also, the last output character string “</ s>” is a predetermined character string indicating the end of the text data. The text generation device 10 generates new replacement text data as “Nikkei average, increase width <price1> yen exceeds” by these character strings output from the decoder 24. Then, the replacing unit 14, the predetermined character string "<price1>", the last value of X _long (X _{long, 7,} the day before the closing price) last value of the X _short (X _{short, 62,} the day Replace with the difference of the closing price) to generate text data that explains the fluctuation of stock prices.

  As in the language model 20 of this example, time series numerical data including a number sequence associated with the passage of time at different time intervals is input to different encoders, and the output is synthesized and input to the decoder. Texts can be generated to account for variations in time series numerical data, so as to correctly include words that are dependent on the series numerical data history. In addition, a text that explains the fluctuation of time-series numerical data so that a word depending on the history of time-series numerical data is correctly included by inputting not only the output of the encoder but also a plurality of numerical data to the decoder Can be generated. Furthermore, by inputting not only the data synthesized by the synthesis unit but also data relating to the time series of time series numerical data to the decoder, it is possible to correctly include words dependent on the history of time series numerical data. Text can be generated to account for variations in series numerical data.

  According to the text generation device 10 according to the present embodiment, for example, “increase width” can be generated by generating a text that describes the variation of time-series numerical data so as to correctly include a word that depends on the history of time-series numerical data. Correctly generate expressions that refer to the history of past stock prices, such as “following,” “rebound,” or words such as “begin,” “near,” “forward,” “afternoon,” “large,” Can correctly generate time zone-dependent expressions.

  FIG. 5 is a diagram showing a rule D3 for associating types of numerical values related to time-series numerical data with a predetermined character string. The rule D3 is an example of a predetermined rule stored in the rule storage unit 15 and referred to by the replacement unit 14.

The rule D3 is a rule to associate 12 types of numerical values related to time series numerical data with 12 types of character strings. Each string corresponds one-to-one to the numerical value related to time series numerical data. In this example, the string <price1> is associated with the difference between the last value (X _{long, 7)} and the last value of X _short of _{X long (X short, 62)} . Also, the string <price2> is associated with a value obtained by rounding off the difference between the last value of X _{long and} the last value of X _{short to} the nearest tenth.

Also, the string <price3> is associated with the value obtained by rounding off the difference between the last value of X _{long and} the last value of X _{short to} the nearest hundred, and the string <price4> is the last of X _long The difference between the value of X and the last value of X _short is rounded up to the nearest tenth, and the string <price5> is the difference between the last value of X _{long and} the last value of X _short by 100. It is associated with the rounded up value.

Furthermore, the string <price6> is associated with the last value of X _short , the string <price7> is associated with the value of the last value of X _short rounded down to the nearest hundred, and <price8 The string> is associated with the last value of X _short truncated to 1000; the string <price 9> maps the last value of X _{short to} the truncated value of 10000 Be Similarly, the string <price10> is associated with the last value of X _short rounded up to the nearest hundred, and the string <price11> rounds the last value of X _{short to} the nearest 1000. The string <price12> is associated with the value obtained by rounding up the last value of X _{short to} the nearest 10000.

  As described above, by associating the type of numeric value related to time-series numeric data with a predetermined character string, the time-series numeric data can be cited so that it includes the numeric value obtained as a result of computing time-series numeric data. Text can be generated to account for variations in numerical data.

  FIG. 6 is a flowchart of processing executed by the text generation device 10 according to the present embodiment. First, the acquisition unit 12 acquires stock prices recorded at 5-minute intervals as first time-series numerical data (S10), and acquires stock prices recorded at one-business-day intervals as second time-series numerical data (S11) ).

  Thereafter, the generation unit 13 inputs the first time series numerical data and the second time series numerical data to the language model 20. The language model 20 converts the first time-series numerical data into normalized numerical data, standardized numerical data, and relativized numerical data by the first preprocessing unit 21a (S12), and the second preprocessing unit 21b. The second time series numerical data is converted into normalized numerical data, standardized numerical data and relative numerical data (S13). Then, a plurality of first numerical data obtained by converting the first time series numerical data is input to the first encoder 22a (S14), and a plurality of second numerical data obtained by converting the second time series numerical data Numerical data is input to the second encoder 22b (S15).

  Furthermore, the combining unit 23 combines the plurality of first numerical data, the plurality of second numerical data, the output of the first encoder 22a, and the output of the second encoder 22b (S16). Thereafter, the synthesized data and data relating to time series are input to the decoder 24 (S17).

  The substitution unit 14 replaces a predetermined character string of the substitution text data output from the decoder 24 with a numerical value according to a predetermined rule (S18), and generates text data for explaining the fluctuation of time series numerical data. Thus, the process ends.

  According to the text generation device 10 according to the present embodiment, new substitution text data describing new time series numerical data is generated by the output of the language model, and a predetermined character string included in the new substitution text data is By replacing the time series numerical data with the numerical values related to the new time series numerical data according to a predetermined rule, it is not necessary to directly output the numerical values related to the time series numerical data by the language model, and even if the numerical values change variously Texts can be generated to account for variations in time series numerical data, so as to correctly include numerical references.

  FIG. 7 is a diagram showing text generated by the text generation device 10 according to the present embodiment. In the figure, a case where a conventional model is used as a language model and a case where a language model 20 according to the present embodiment or a model obtained by changing a part of the language model 20 according to the present embodiment is used are generated. It shows Table 1 R1 in which the text is summarized. The language model 20 according to the present embodiment is the language model 20 described with reference to FIG. 4 and is a model in which the first encoder 22a and the second encoder 22b are MLP. In addition, a first example of a model obtained by modifying a part of the language model 20 according to the present embodiment is a model that does not use standardized data in the language model 20 described with reference to FIG. 4, that is, the first pre-processing unit 21 a The second pre-processing unit 21 b calculates two types of data normalized by the second pre-processing unit 21 b and relativized data. In addition, a second example of a model in which a part of the language model 20 according to the present embodiment is modified is a model that does not use replacement text data in the language model 20 described with reference to FIG. It is a model that directly generates numerical values related to data. Further, in the third example of the model in which a part of the language model 20 according to the present embodiment is modified, the data relating to the time series of time series numerical data is not input to the decoder 24 of the language model 20 described using FIG. It is a model.

  The figure shows an example E of accurate text data in addition to the first table R1. An example E of accurate text data is "Nikkei average closing, next extension closing price is 16,906 yen, which is 32 yen high".

  On the other hand, an example of the text generated when the conventional model is used as the language model is “Nikkei average, ¥ 20,606 at the end of the fall before the end of the fall”, and the text data delivery time zone is incorrect. That the difference between the previous day's stock and the previous day is incorrectly described as "back", the difference between the previous day's close and the previous day's closing price is incorrectly "57 yen", and the current stock is incorrect It is lacking in accuracy in that it is “20,606 yen”.

  On the other hand, an example of the text generated when using the language model 20 according to the present embodiment described second from the top of Table 1 R1 The text data delivery time zone is correctly described as "large closing", the difference between the stock prices from the previous day is correctly expressed as "seizun", and the difference from the previous day's closing price is correctly calculated as "32 yen high". The present stock price is correctly quoted as "16,906 yen", and all expressions are accurate.

  An example of the text generated when using the first example of the model in which a part of the language model 20 according to the present embodiment is changed, which is described in the third table from the top of Table 1 R1, is “Nikkei average, The next year's large closing is ¥ 16,906 at ¥ 32, and the delivery time zone of the text data is correctly described as “large closing”, the difference between the stock prices from the previous day is correctly expressed as “second opening”, and the difference from the previous day's closing price is “ It is correctly calculated as “32 yen high”, the current stock price is correctly quoted as “16,906 yen”, and all expressions are correct. From this, even in a model that calculates two types of data normalized by the first pre-processing unit 21a and the second pre-processing unit 21 b and relative data, the first pre-processing unit 21 a and the second pre-processing It can be understood that the text data describing the time-series numerical data can be generated with the same or higher accuracy as a model that calculates three types of standardized data, normalized data, and relativized data by the part 21b.

  An example of the text generated in the case of using the second example of the model in which a part of the language model 20 according to the embodiment described in the fourth table from the top of Table 1 R1 is changed is “Nikkei average, continued to grow close of a can of 28 yen <unk> circle ", the delivery time zone of the text data is correctly described as" the close ", the difference between the share price of the previous day correctly described as" firmer ", the difference between the previous day closing price the mistake has been calculated as "28 yen", it has become a can not be the current stock price quotation "<unk> circle". Here, <unk> is a string representing the unknown unknown, it indicates that the appropriate word could not be produced. From this, it is difficult to correctly generate numerical values related to time-series numerical data by directly generating numerical values related to time-series numerical data using a language model, and not only expressions involving computation of time-series numerical data. It also proves difficult to include the citation of time series numerical data.

  An example of the text generated when the third example of the model in which a part of the language model 20 according to the present embodiment is changed, which is described in the fifth from the top of Table 1 R1 is “Nikkei average, The second closing advance closing price is ¥ 16,906 at ¥ 32, and the delivery time zone of the text data is mistakenly described as “front closing”, and the difference between the stock prices from the previous day is correctly described as “second extension” and the previous day closing price The difference is correctly calculated as "32 yen high", and the current stock price is correctly quoted as "16,906 yen". From this, it can be understood that it is difficult to correctly refer to the time zone in which the time-series numerical data is acquired, in a model in which no data relating to the time-series of time-series numerical data is input to the decoder 24.

  FIG. 8 is a diagram showing index values obtained by evaluating the closeness between the text generated by the text generation device 10 according to the present embodiment and the text serving as a reference. In the figure, a case where a conventional model is used as a language model and a case where a language model 20 according to the present embodiment or a model obtained by changing a part of the language model 20 according to the present embodiment is used are generated. It shows Table 2 R2 in which index values that evaluate the closeness of the text and the reference text are summarized. The index value is a value called BLEU (BiLingual Evaluation Understudy), takes a value from 0 to 1, and indicates that the closer to 1 the closer to the reference text (the correct text). This index value is an example of one used to evaluate text, and other index values can be used to evaluate text.

  The language model 20 according to the present embodiment is the language model 20 described with reference to FIG. 4 and is a model in which the first encoder 22a and the second encoder 22b are MLP, and the first encoder 22a and the second encoder 22b. A model with CNN and a model with RNN as the first encoder 22a and the second encoder 22b.

  Further, a first example of a model in which a part of the language model 20 according to the present embodiment is modified is a model not using the first time-series numerical data in the language model 20 described with reference to FIG. It is a model using only series numerical data. The second example of the model in which a part of the language model 20 according to the present embodiment is modified is a model that does not use the second time-series numerical data in the language model 20 described with reference to FIG. It is a model using only data.

  In addition, a third example of a model in which a part of the language model 20 according to the present embodiment is modified is a model that does not use normalized data in the language model 20 described with reference to FIG. It is a model which calculates two types of data standardized by 21a and the 2nd pretreatment part 21b, and relativized data. A fourth example of a model in which a part of the language model 20 according to the present embodiment is modified is a model that does not use standardized data in the language model 20 described with reference to FIG. 4, that is, the first preprocessing unit 21a and the first preprocessing unit 21a. 2 A model for calculating two types of data normalized by the pre-processing unit 21 b and relativized data. The fifth example of the model in which part of the language model 20 according to the present embodiment is modified is a model that does not use relativized data in the language model 20 described with reference to FIG. 4, that is, the first preprocessing unit 21 a and This is a model for calculating two types of data standardized by the second pre-processing unit 21 b and normalized data.

  Further, the sixth example of a model in which a part of the language model 20 according to the present embodiment has been modified converts time series numerical data to the decoder 24 of the language model 20 described using FIG. 4 by one or more methods. , Or a model in which only the output of the first encoder 22a and the output of the second encoder 22b are input to the decoder 24. A seventh example of a model in which a part of the language model 20 according to the present embodiment has been modified is a model not using replacement text data in the language model 20 described with reference to FIG. It is a model that directly generates related numerical values. Further, in the eighth example of a model in which a part of the language model 20 according to the present embodiment is modified, the data relating to the time series of time series numerical data is not input to the decoder 24 of the language model 20 described using FIG. It is a model.

  While the evaluation value of the text generated when the conventional model is used as the language model is “0.244”, the text generation device 10 generates the evaluation value using the language model 20 according to the present embodiment. The evaluation value of the text is “0.415” when using MLP for the encoder, “0.414” when using CNN for the encoder, and “0.415” when using RNN for the encoder. In either case, the evaluation value is significantly improved over the conventional case, and it can be seen that accurate text data can be generated.

  The evaluation value of the text generated using the first example of the model in which a part of the language model 20 according to the present embodiment described in the fifth from the top of Table 2 R2 is "0.356", The evaluation value of the text generated using the second example of the model in which a part of the language model 20 according to the present embodiment described in the sixth from the top of Table 2 R2 is changed to "0.397" It can be seen that the evaluation value is improved by using two types of time-series numerical data acquired at different time intervals. It is considered that this is because, according to the text generation device 10 according to the present embodiment, a word depending on the history of time series numerical data can be correctly generated.

  In addition, the evaluation value of the text generated using the third example of the model in which a part of the language model 20 according to the present embodiment has been described seventh from the top in Table 2 R2 is “0.424 The evaluation value of the text generated using the fourth example of the model in which a part of the language model 20 according to the present embodiment is modified described in the eighth from the top of Table 2 R2 is “0.424”. The evaluation value of the text generated using the fifth example of the model in which a part of the language model 20 according to the present embodiment has been described ninth from the top in Table 2 R2 is “0.408 ". From these facts, using either normalized data or normalized data and relativized data, all normalized data, normalized data and relativized data are used. It turns out that text data more appropriate than in the case can be generated. In addition, it is understood that the evaluation value is deteriorated if the relativized data is not used.

  In addition, the evaluation value of the text generated using the sixth example of the model in which a part of the language model 20 according to the present embodiment has been described tenth from the top of Table 2 R2 is “0.397 The evaluation value of the text generated using the seventh example of the model in which a part of the language model 20 according to the present embodiment has been described from the top of the second table R2 in the second table R2 is “0.313 The evaluation value of the text generated using the eighth example of the model in which a part of the language model 20 according to the present embodiment is modified, described in the twelfth table from the top of Table 2 R2, is “0.358 ". From these facts, a model which inputs only the output of the first encoder 22a and the output of the second encoder 22b to the decoder 24, a model which directly generates numerical values related to time series numerical data by the language model, time series numerical data It can be seen that, when using a model that does not input data regarding time series, the index value is worse than when using the language model 20 according to the present embodiment.

  The embodiments described above are for the purpose of facilitating the understanding of the present invention, and are not for the purpose of limiting the present invention. The elements included in the embodiment and the arrangement, the material, the conditions, the shape, the size, and the like of the elements are not limited to those illustrated, and can be changed as appropriate. In addition, configurations shown in different embodiments can be partially substituted or combined with each other.

  DESCRIPTION OF SYMBOLS 10 ... Text generation apparatus, 10a ... CPU, 10b ... RAM, 10c ... ROM, 10d ... Communications part, 10e ... Input part, 10f ... Display part, 11 ... Learning part, 12 ... Acquisition part, 13 ... Generation part, 14 ... Replacement unit 15 Rule storage unit 20 Language model 21a First pre-processing unit 21b Second pre-processing unit 22a First encoder 22b Second encoder 23 Combining unit 24 Decoder , 30: user terminal, 40: stock price distribution server, 100: text generation system, D1: substitution text data, D2: time series numerical data, D3: rule, E: example of accurate text data, N: communication network, R1 ... Table 1, R2 ... Table 2

Claims (10)

A text generation apparatus for generating text data for explaining fluctuation of time series numerical data including a number sequence associated with the passage of time, wherein
The time-series numerical data is input using, as learning data, replacement text data in which a numerical value related to the time-series numerical data in the text data is replaced with a predetermined character string according to a predetermined rule, and the time-series numerical data A learning unit for learning a language model so as to output the substitution text data
A generation unit which inputs new time-series numerical data to the language model learned by the learning unit, and generates new replacement text data by an output of the language model;
A replacement unit that replaces the predetermined character string included in the new replacement text data with a numerical value related to the new time-series numerical data according to the predetermined rule;
A text generator comprising: The numerical values related to the time series numerical data are
Among the numerical sequences included in the time series numerical data, a numerical value associated with a predetermined time point, and
Among the numerical sequences included in the time-series numerical data, the difference between the numerical values associated with different points in time, and
A numerical value obtained by rounding off a numerical value associated with a predetermined time point in the numerical sequence included in the time series numerical data to a predetermined digit;
A numerical value obtained by rounding off the difference between the numerical values associated with different time points among the numerical sequences included in the time-series numerical data by a predetermined digit,
A numerical value obtained by rounding up a numerical value associated with a predetermined time point in the numerical sequence included in the time series numerical data by a predetermined digit,
At least one of numerical values obtained by rounding up the difference between the numerical values associated with different time points among the numerical sequences included in the time-series numerical data by a predetermined digit,
The predetermined rule is a rule that associates the type of numerical value related to the time-series numerical data with the predetermined character string.
The text generator according to claim 1. The time-series numerical data includes first time-series numerical data including a number sequence associated with the passage of time at a first interval, and a number sequence associated with the passage of time at a second interval longer than the first interval. Including second time series numerical data including
The text generator according to claim 1 or 2. The generation unit inputs, to the language model, one or more numerical data corresponding to the one or more methods, which is obtained by converting the time series numerical data by one or more methods.
The text generator according to any one of claims 1 to 3. The one or more numerical data are
Numerical data obtained by normalizing a number sequence included in the time series numerical data into a predetermined numerical range;
Numerical data in which the time series numerical data is standardized using the average value and the standard deviation of the numerical sequence included in the time series numerical data;
Numerical data obtained by relativizing a numerical sequence included in the time-series numerical data with respect to the reference value using a numerical value associated with a predetermined point in time among the numerical sequences included in the time-series numerical data as the reference value;
At least one of
The text generator according to claim 4. The language model is
One or more one-to-one corresponding to the one or more methods obtained by converting the first time-series numerical data including the number sequence associated with the passage of time at the first interval by one or more methods A first encoder to which first numerical data is input;
One or more of the one or more methods are obtained by converting second time-series numerical data including a number sequence associated with the passage of time at a second interval longer than the first interval. A second encoder to which one or more second numerical data corresponding to
A combining unit that combines the output of the first encoder and the output of the second encoder;
A decoder for receiving the data synthesized by the synthesis unit and outputting the replacement text data;
The text generator according to any one of claims 1 to 5. The combining unit combines the output of the first encoder, the output of the second encoder, the one or more first numerical data, and the one or more second numerical data.
The text generator according to claim 6. The data combined by the combining unit and data related to a time series of the time-series numerical data are input to the decoder.
The text generator according to claim 6 or 7. A text generation method for generating text data that describes, by a computer comprising a hardware processor and a memory, fluctuations in time series numerical data including a number sequence associated with the passage of time, comprising:
The time-series numerical data is input using, as learning data, replacement text data in which a numerical value related to the time-series numerical data in the text data is replaced with a predetermined character string according to a predetermined rule, and the time-series numerical data Training the language model so as to output the substitution text data,
Inputting new time series numerical data into the learned language model, and generating new substitution text data describing the new time series numerical data by the output of the language model;
Replacing the predetermined character string included in the new replacement text data with a numerical value related to the new time-series numerical data according to the predetermined rule;
How to generate text to execute A computer provided in a text generation device for generating text data that describes variation in time series numerical data including a number sequence associated with the passage of time,
The time-series numerical data is input using, as learning data, replacement text data in which a numerical value related to the time-series numerical data in the text data is replaced with a predetermined character string according to a predetermined rule, and the time-series numerical data A learning unit that learns a language model to output the substitution text data
A generation unit for inputting new time series numerical data to the language model learned by the learning unit, and generating new replacement text data describing the new time series numerical data by an output of the language model; A substitution unit that replaces the predetermined character string included in new substitution text data with a numerical value related to the new time series numerical data according to the predetermined rule;
Text generator to act as.

Images