JP6910061B2 - Text generator, text generator and text generator - Google Patents

Description

The present invention relates to a text generator, a text generator and a text generator.

In recent years, in the field of natural language processing, language models using neural networks such as recurrent neural networks have been studied.

For example, in Patent Document 1 below, a recognition result acquisition unit that acquires words recognized from an interactive text, time-series information of words, and identification information for identifying the speaker of a word from a first database, Described is a dialogue text summarizing device having a text summarizing unit that corrects words based on word-to-word time-series information, identification information, and a summarizing model, and outputs the corrected results to a first database.

JP-A-2017-111190

A language model using a neural network may be trained to use a large amount of text data as training data and generate text based on the statistical characteristics of words appearing in the training data.

However, when generating a text explaining the fluctuation of time series numerical data (for example, stock price) including a sequence of numerical values that changes with the passage of time, the training data is a citation of the time series numerical data or an explanation about the amount of change (for example). , If it is a stock price, it may include the opening price, closing price, increase range, etc.), and the numerical values associated with the explanation change variously, so each numerical value becomes a word that rarely appears statistically. .. Therefore, it is difficult to train the language model so as to correctly reproduce the description related to the numerical value, and it is difficult to generate a text explaining the fluctuation of the time-series numerical data.

Therefore, the present invention provides a text generator, a text generation method, and a text generation program that generate text explaining fluctuations in time-series numerical data.

The text generator according to one aspect of the present invention is a text generator that generates text data for explaining fluctuations in time-series numerical data including a number sequence associated with the passage of time, and is a time-series of text data. When the replacement text data and the time-series numerical data in which the numerical values related to the numerical data are replaced with the predetermined character strings according to a predetermined rule are used as training data, the replacement text data is output when the time-series numerical data is input. New time-series numerical data is input to the learning unit that trains the language model and the language model learned by the learning unit, and new replacement text data that explains the new time-series numerical data is input by the output of the language model. It includes a generation unit to be generated and a replacement unit that replaces a predetermined character string included in the new replacement text data with a numerical value related to the new time-series numerical data according to a predetermined rule.

According to this aspect, the output of the language model generates new replacement text data explaining the new time-series numerical data, and the predetermined character string contained in the new replacement text data is changed to a new time according to a predetermined rule. By replacing with the numerical value related to the series numerical data, it is not necessary to directly output the numerical value related to the time series numerical data by the language model, and even if the numerical value changes variously, the description about the numerical value is correctly included. It is possible to generate text explaining the fluctuation of time series numerical data.

In the above aspect, the numerical value related to the time-series numerical data is associated with a numerical value associated with a predetermined time point among the numerical values included in the time-series numerical data and a different time point among the numerical values included in the time-series numerical data. Corresponds to the difference between the numerical values and the numerical value obtained by rounding down the numerical value associated with a predetermined time point in the number sequence included in the time-series numerical data by a predetermined digit and the different time point in the number sequence included in the time-series numerical data. The difference between the attached numerical values is rounded down to a predetermined digit, the numerical value associated with a predetermined time point in the numerical sequence included in the time series numerical data is rounded up to a predetermined digit, and the time series numerical data. Includes at least one of the included number columns, which is the difference between the numbers associated with different time points, rounded up to a given digit, and the given rule is the type of number and the given number related to time-series numerical data. It may be a rule that associates with the character string of.

According to this aspect, by associating the type of numerical value related to the time-series numerical data with a predetermined character string, the numerical value obtained as a result of quoting the time-series numerical data or calculating the time-series numerical data is included. It is possible to generate text explaining the fluctuation of time series numerical data.

In the above aspect, the time-series numerical data is associated with the first time-series numerical data including the sequence associated with the passage of time in the first interval and the passage of time in the second interval longer than the first interval. It may include a second time series numerical data including a sequence of numbers.

According to this aspect, by using time-series numerical data including a sequence of numbers associated with the passage of time at different time intervals, the time-series numerical data correctly includes words that depend on the history of the time-series numerical data. Can generate text that describes the fluctuations in.

In the above aspect, the generation unit inputs one or more numerical data obtained by converting the time series numerical data by one or more methods corresponding to one or more methods one-to-one into the language model. May be good.

According to this aspect, by inputting one or more numerical data obtained by converting time-series numerical data by one or more methods into a language model, the generated text becomes an absolute value of the time-series numerical data. Dependence on blurring is prevented.

In the above embodiment, the one or more numerical data uses the numerical data obtained by normalizing the numerical strings included in the time-series numerical data to a predetermined numerical range, and the average value and standard deviation of the numerical strings included in the time-series numerical data. The numerical data that standardizes the time-series numerical data and the numerical value associated with a predetermined time point among the numerical columns included in the time-series numerical data are used as reference values, and the numerical strings included in the time-series numerical data are relativized with respect to the reference value. Numerical data and at least one of them may be included.

According to this aspect, by using the normalized numerical data or the standardized numerical data, it is prevented that the generated text is blurred depending on the absolute value of the time-series numerical data, and the relativized numerical data is used. By doing so, it is possible to generate a text explaining the fluctuation of the time-series numerical data so as to correctly include the words that depend on the history of the time-series numerical data.

In the above embodiment, the language model is one-to-one with one or more methods obtained by converting first time-series numerical data including a number sequence associated with the passage of time in the first interval by one or more methods. One or more first encoders into which one or more first numerical data corresponding to the above are input, and one or more second time series numerical data including a number sequence associated with the passage of time in a second interval longer than the first interval. The output of the first encoder and the output of the second encoder are combined with the second encoder in which one or more second numerical data corresponding to one or more methods corresponding to one or more methods are input, which is obtained by conversion by the method of. It may include a compositing unit and a decoder in which the data synthesized by the compositing unit is input and the replacement text data is output.

According to this aspect, time-series numerical data including a number sequence associated with the passage of time at different time intervals is input to different encoders, and the outputs are combined and input to the decoder to obtain time-series numerical data. Text can be generated to explain the variation of time series numerical data so that it correctly contains history-dependent words.

In the above aspect, the compositing unit may synthesize the output of the first encoder, the output of the second encoder, one or more first numerical data, and one or more second numerical data.

According to this aspect, by inputting not only the output of the encoder but also a plurality of numerical data to the decoder, the variation of the time-series numerical data so as to correctly include the words depending on the history of the time-series numerical data. Can generate text that describes.

In the above aspect, the data synthesized by the compositing unit and the time-series numerical data of the time-series numerical data may be input to the decoder.

According to this aspect, by inputting not only the data synthesized by the synthesizer but also the data related to the time series of the time series numerical data into the decoder, the words depending on the history of the time series numerical data are correctly included. As such, it is possible to generate text that explains the fluctuations in time series numerical data.

The text generation method according to another aspect of the present invention is a text generation method in which a computer including a hardware processor and a memory generates text data explaining fluctuations in time-series numerical data including a sequence of numbers associated with the passage of time. This is a method in which time-series numerical data is input using replacement text data in which numerical values related to time-series numerical data are replaced with predetermined character strings in a predetermined rule and time-series numerical data as learning data. When this is done, the language model is trained to output the replacement text data, new time-series numerical data is input to the trained language model, and the new time-series numerical data is explained by the output of the language model. Generate new replacement text data and replace the predetermined character string contained in the new replacement text data with a numerical value related to the new time-series numerical data according to a predetermined rule.

According to this aspect, the output of the language model generates new replacement text data explaining the new time-series numerical data, and the predetermined character string contained in the new replacement text data is changed to a new time according to a predetermined rule. By replacing with the numerical value related to the series numerical data, it is not necessary to directly output the numerical value related to the time series numerical data by the language model, and even if the numerical value changes variously, the description about the numerical value is correctly included. In addition, it is possible to generate text explaining the fluctuation of time series numerical data.

The text generator according to another aspect of the present invention uses a computer provided with a text generator for generating text data to explain fluctuations in time-series numerical data including a sequence of numbers associated with the passage of time. Of these, when the replacement text data in which the numerical values related to the time-series numerical data are replaced with the predetermined character strings according to a predetermined rule and the time-series numerical data as training data, the replacement text is input when the time-series numerical data is input. A learning unit that trains a language model to output data, a new replacement that inputs new time-series numerical data into the language model learned by the learning unit and explains the new time-series numerical data by the output of the language model. It functions as a generation unit that generates text data and a replacement unit that replaces a predetermined character string included in the new replacement text data with a numerical value related to the new time-series numerical data according to a predetermined rule.

According to this aspect, the output of the language model generates new replacement text data explaining the new time-series numerical data, and the predetermined character string contained in the new replacement text data is changed to a new time according to a predetermined rule. By replacing with the numerical value related to the series numerical data, it is not necessary to directly output the numerical value related to the time series numerical data by the language model, and even if the numerical value changes variously, the description about the numerical value is correctly included. In addition, it is possible to generate text explaining the fluctuation of time series numerical data.

According to the present invention, it is possible to provide a text generator, a text generation method, and a text generation program that generate text explaining fluctuations in time-series numerical data.

It is a figure which shows the network configuration of the text generator which concerns on embodiment of this invention. It is a figure which shows the physical structure of the text generator which concerns on this embodiment. It is a figure which shows the functional block of the text generator which concerns on this embodiment. It is a figure which shows the structure of a language model. It is a figure which shows the rule which associates a predetermined character string with the type of the numerical value related to time-series numerical data. It is a flowchart of the process executed by the text generator which concerns on this embodiment. It is a figure which shows the text generated by the text generator which concerns on this embodiment. It is a figure which shows the index value which evaluated the closeness of the text generated by the text generator which concerns on this embodiment, and a reference text.

Embodiments of the present invention will be described with reference to the accompanying drawings. In each figure, those having the same reference numerals have the same or similar configurations.

FIG. 1 is a diagram showing a network configuration of the text generator 10 according to the embodiment of the present invention. In the present embodiment, the text generation system 100 stores a database DB that stores an initial data set including time-series numerical data including a number sequence associated with the passage of time and text data explaining fluctuations in the time-series numerical data. Using the language model 20 that outputs the replacement text data according to the input time-series numerical data and the initial data set stored in the database DB, the language model 20 correctly explains the fluctuation of the time-series numerical data. It includes a text generation device 10 that trains a language model 20 so that text is generated, and generates text data that explains fluctuations in the time-series numerical data when new time-series numerical data is acquired. In the present embodiment, the time series numerical data is a stock price. However, the time-series numerical data includes a number sequence associated with the passage of time, and may be any numerical data that is continuously acquired, for example, electrocardiographic data or electrocardiographic data. It may be vital data such as blood pressure data, weather data such as temperature and humidity, and traffic data such as traffic volume and number of passengers.

The text generation system 100 is connected to the communication network N, acquires stock prices from the stock price distribution server 40 at predetermined time intervals, stores them in the database DB, and inputs them to the text generation device 10. Further, the text generation system 100 provides the generated text data to the user terminal 30 via the communication network N. Further, the text generation system 100 may add or edit the initial data set stored in the database DB or learn the language model 20 based on the instruction from the user terminal 30. Here, the communication network N is a wired or wireless communication network, and may be, for example, the Internet or a LAN (Local Area Network). In the text generation system 100, all or part of the components may be configured by a remote computer in the form of so-called cloud computing, but all or some of the components may be configured by a local computer.

The language model 20 is a replacement text data in which, when time-series numerical data is input, the numerical values related to the time-series numerical data among the text data explaining the time-series numerical data are replaced with predetermined character strings according to a predetermined rule. It is a model that outputs. Here, the numerical value related to the time-series numerical data may be a citation of a numerical value included in the time-series numerical data, or a numerical value obtained by calculating a numerical value included in the time-series numerical data. The language model 20 may be, for example, a model using a neural network, and may be a so-called encoder-decoder model. The language model 20 may include, for example, an MLP (Multi-Layer Perceptron), a CNN (Convolutional Neural Network) or an RNN (Recurrent Neural Network) as an encoder, and may include an RNN as a decoder. The language model 20 may be a different model depending on the type of time-series numerical data to be input. The language model 20 will be described in detail later with reference to FIG.

FIG. 2 is a diagram showing a physical configuration of the text generator 10 according to the present embodiment. The text generator 10 includes a CPU (Central Processing Unit) 10a corresponding to a hardware processor, a RAM (Random Access Memory) 10b corresponding to a memory, a ROM (Read Only Memory) 10c corresponding to a memory, and a communication unit 10d. And an input unit 10e and a display unit 10f. Each of these configurations is connected to each other via a bus so that data can be transmitted and received.

The CPU 10a is a control unit that controls execution of a program stored in the RAM 10b or ROM 10c, calculates data, and processes data. The CPU 10a is an arithmetic unit that executes a program (text generation program) that generates text data using the language model 20. The CPU 10a receives various input data from the input unit 10e and the communication unit 10d, displays the calculation result of the input data on the display unit 10f, and stores it in the RAM 10b or ROM 10c.

The RAM 10b is a storage unit capable of rewriting data, and is composed of, for example, a semiconductor storage element. The RAM 10b stores programs and data such as applications executed by the CPU 10a.

The ROM 10c is a storage unit capable of only reading data, and is composed of, for example, a semiconductor storage element. The ROM 10c stores programs and data such as firmware.

The communication unit 10d is a communication interface that connects the text generator 10 to the communication network N.

The input unit 10e receives data input from the user, and is composed of, for example, a keyboard, a mouse, and a touch panel.

The display unit 10f visually displays the calculation result by the CPU 10a, and is configured by, for example, an LCD (Liquid Crystal Display).

The text generation program may be stored in a storage medium readable by a computer such as a RAM 10b or a ROM 10c and provided, or may be provided via a communication network N connected by a communication unit 10d. In the text generation device 10, the CPU 10a executes the text generation program to realize various functions described with reference to the following figures. It should be noted that these physical configurations are examples and do not necessarily have to be independent configurations. For example, the text generator 10 may include an LSI (Large-Scale Integration) in which the CPU 10a and the RAM 10b or ROM 10c are integrated. Further, the text generator 10 may include arithmetic circuits such as a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit).

FIG. 3 is a diagram showing a functional block of the text generator 10 according to the present embodiment. The text generation device 10 includes a learning unit 11, an acquisition unit 12, a generation unit 13, a replacement unit 14, and a regular storage unit 15.

The learning unit 11 learns the replacement text data D1 and the time-series numerical data D2 in which the numerical values related to the time-series numerical data are replaced with predetermined character strings by a predetermined rule among the text data explaining the fluctuation of the time-series numerical data. When time-series numerical data is input as the data for use, the language model 20 is trained so as to output the replacement text data. The replacement text data D1 and the time-series numerical data D2 used for learning the language model 20 by the learning unit 11 may be stored in the database DB as initial data sets.

Here, the numerical values related to the time-series numerical data are associated with the numerical values associated with a predetermined time point among the numerical values included in the time-series numerical data and the different time points among the numerical values included in the time-series numerical data. Correspondence between the difference between the numerical values and the numerical value associated with the specified time point in the number sequence included in the time-series numerical data, rounded down to a predetermined digit, and the different time point in the number sequence included in the time-series numerical data. Included in the time-series numerical data: the numerical value obtained by rounding down the difference between the given numerical values by a predetermined digit, the numerical value in which the numerical value associated with a predetermined time point in the numerical sequence included in the time-series numerical data is rounded up by a predetermined digit, and the numerical value. It may include at least one of a numerical value obtained by rounding up the difference between the numerical values associated with different time points in the numerical sequence to be obtained by a predetermined digit. Further, the numerical value may be rounded down or rounded up to a predetermined digit at any place such as the tens place, the 100s place, the 1000s place, and the 10000s place. Further, the predetermined rule may be a rule for associating a predetermined character string with a type of numerical value related to time-series numerical data. Here, the predetermined character string may be any character string as long as it is a character string that can be distinguished from ordinary text data, and for example, a predetermined symbol such as <price1> or <price2> (in this example, "<" "And"> ") may be a character string whose beginning and end are indicated.

Further, the time-series numerical data includes a first time-series numerical data including a sequence of numbers associated with the passage of time in the first interval and a sequence of numbers associated with the passage of time in a second interval longer than the first interval. It may include the second time series numerical data including. In the case of the present embodiment, the first time-series numerical data X _short is time-series numerical data related to the stock price acquired at intervals of 5 minutes from the approach to the close of one business day, and the second time-series numerical data X. _long is time-series numerical data relating to the closing price of the stock price acquired at business day intervals for 7 business days. That is, the first time-series numerical data X _short is time-series numerical data including a number sequence associated with the passage of time at 5-minute intervals, and the second time-series numerical data X _long is time at 1 business day intervals. It is time series numerical data including a number sequence associated with the progress of. In the case of the Tokyo Stock Exchange in Japan, the trading witness time in one business day is 5 hours (300 minutes), and the first time-series numerical data acquired at 5-minute intervals includes 62 pieces of data. This is expressed as X _{short, i} (i = 1 to 62). Further, the second time series numerical data X _long includes seven data, which are _{represented as X long, j} (j = 1 to 7).

The acquisition unit 12 acquires new time-series numerical data from the stock price distribution server 40. The acquisition unit 12 may acquire new time-series numerical data regarding the stock price from the stock price distribution server 40, for example, at intervals of 5 minutes.

The generation unit 13 inputs new time-series numerical data into the language model 20 learned by the learning unit 11, and generates new replacement text data for explaining the new time-series numerical data by the output of the language model 20. The generation unit 13 may input one or a plurality of numerical data having a one-to-one correspondence to the one or a plurality of methods obtained by converting the time series numerical data by one or a plurality of methods into the language model 20. .. Here, for one or more numerical data, the time is obtained by using the numerical data obtained by normalizing the numerical strings included in the time-series numerical data to a predetermined numerical range and the average value and standard deviation of the numerical strings included in the time-series numerical data. A numerical value obtained by standardizing the series numerical data and a numerical value associated with a predetermined time point among the numerical values included in the time-series numerical data as a reference value, and a numerical value obtained by relativizing the numerical string included in the time-series numerical data with respect to the reference value. It may include at least one of the data.

_{More specifically, the numerical data X norm, i obtained by normalizing the sequence X i} (i = 1 to N) included in the time-series numerical data to a predetermined numerical range is X _{norm, i} = (2X _i − (2X i − (. It may be numerical data defined by X _max + X _min )) / (X _max −X _min). Here, X _max = max _i (X _i ) and X _min = min _i (X _i ). In this case, the normalized numerical data X _{norm, i} is the numerical data normalized in the numerical range of -1 to 1.

_{Further, the numerical data X std, i obtained by standardizing the sequence X i} (i = 1 to N) included in the time-series numerical data is the numerical data defined by X _{std, i} = (X _{i −μ) / σ.} It may be there. Here, μ = E [X _i ] and σ = (var [X _i ]) ^1/2 .

Moreover, when the sequence X _i included in the numerical data at (i = 1 to N) numerically and relative with respect to the reference value r _i of the data X _{move, i} is defined by _{X move, i = X i -r} i It may be numerical data. When the time-series numerical data is a stock price, the reference value r _i may be the closing price of the previous day. _{That is, for the first time-series numerical data X short} including a sequence of numbers associated with the passage of time at 5-minute intervals, when the closing price of the previous day is r, for all i (X _{short, i} −r). ) May calculate the relativized numerical data. also. _{For the second time-series numerical data X long} including a sequence of numbers associated with the passage of time at business day intervals, numerical data relativized by (X _{long, j} −X _{long, j-1} ) may be calculated. ..

By using normalized numerical data or standardized numerical data, it is possible to prevent the generated text from blurring depending on the absolute value of the time-series numerical data, and by using the relativized numerical data, the time-series numerical value is used. Text can be generated to explain the variation in time-series numerical data so that it correctly contains words that depend on the history of the data.

The replacement unit 14 replaces a predetermined character string included in the new replacement text data with a numerical value related to the new time-series numerical data according to a predetermined rule. The replacement unit 14 refers to a predetermined rule stored in the rule storage unit 15 and replaces a predetermined character string included in the new replacement text data with a numerical value related to the new time-series numerical data. Replacement unit 14, for example, a predetermined string <price1>, the last value of X _long (X _{long, 7,} the day before the closing price) last value of the X _short (X _{short, 62,} the day closing price) or replaced with a difference, a predetermined character string of <price2>, the last value of X _long (X _{long, 7,} the day before the closing price) last value of the X _short of (X _{short, 62,} of the day closing price) Replace the difference with a value truncated to the tens digit.

FIG. 4 is a diagram showing the configuration of the language model 20. _{The language model 20 is obtained by converting the first time-series numerical data X short} including the number sequence associated with the passage of time at the first interval by the first preprocessing unit 21a by one or a plurality of methods. _{A first encoder 22a in which one or a plurality of first numerical data l s} corresponding to a plurality of methods one-to-one is input, and a number sequence associated with the passage of time in a second interval longer than the first interval. Two time-series numerical data X _long is converted by the second preprocessing unit 21b by one or more methods, and one or more second numerical data l _l corresponding to one or more methods is input. The second encoder 22b, the _{compositing unit 23 that synthesizes the output h s of the first encoder 22a and the output h l} of the second encoder 22b, and the data m synthesized by the compositing unit 23 are input, and the replacement text data is input. Includes a decoder 24 for output.

In this example, the first time series numerical data X _short is given as a 62-dimensional vector including numerical values such as "12167.29" and "12278.83". Further, the second time series numerical data X _long is given as a 7-dimensional vector including numerical values such as "1211.6.57" and "12120.94". The first preprocessing unit 21a converts the input first time series numerical data X _{short by three kinds of methods, and converts the first numerical data l s} by the direct sum of the three kinds of vectors obtained by the conversion. Output. Here, the three types of methods are to calculate the numerical data obtained by normalizing the input first time-series numerical data X _short into a predetermined numerical range, to calculate the standardized numerical data, and to obtain a reference value. It is to calculate relativized numerical data. In the case of this example, the first numerical data l _s output from the first preprocessing unit 21a is a 186-dimensional vector.

In this way, by using time-series numerical data that includes a number sequence associated with the passage of time at different time intervals, fluctuations in the time-series numerical data so as to correctly include words that depend on the history of the time-series numerical data. Can generate text that describes. In addition, by inputting one or more numerical data obtained by converting time series numerical data by one or more methods into the language model, the generated text is blurred depending on the absolute value of the time series numerical data. Is prevented.

Similarly, the second preprocessing unit 21b converts the input second time series numerical data X _long by three kinds of methods, and the second numerical data by the direct sum of the three kinds of vectors obtained by the conversion. Output l _l. Here, the three types of methods are to calculate the numerical data obtained by normalizing the input second time-series numerical data X _long into a predetermined numerical range, to calculate the standardized numerical data, and to obtain a reference value. It is to calculate relativized numerical data. In the case of this example, the second numerical data l _l output from the second preprocessing unit 21b is a 21-dimensional vector.

_{The first numerical data l s} output from the first preprocessing unit 21a is input to the first encoder 22a, and the vector h _s is output. Here, the dimension of the vector h _s is the number of output nodes included in the output layer of the first encoder 22a. Similarly, the second numerical data l _l output from the second preprocessing unit 21b is input to the second encoder 22b, and the vector h _l is output. Here, the dimension of the vector h _l is the number of output nodes included in the output layer of the second encoder 22b. The first encoder 22a and the second encoder 22b may be any of MLP, CNN and RNN, and may be other models.

The compositing unit 23 is output from the output h _{s of the first encoder 22a, the output h l} of the second encoder 22b, _{the first numerical data l s} output from the first preprocessing unit 21a, and the second preprocessing unit 21b. These data are combined by the direct sum of the second numerical data l _l.

The data m synthesized by the synthesizing unit 23 and the time-series data T of the time-series numerical data are input to the decoder 24. The data T related to the time series corresponds to the data related to the latest time among the times associated with the _{number strings} included in the first time series numerical data X short, or the numerical strings included _{in the second time series numerical data X long.} It may be data about the range of business days attached.

In this example, the decoder 24 outputs replacement text data such as "Nikkei", "average", ",", "raise width", "<price1>", "circle", "exceeds", and "</ s>". is doing. Here, the fifth output character string "<price1>" is a predetermined character string to be replaced with a numerical value related to the time-series numerical data by the replacement unit 14 of the text generation device 10. The last output character string "</ s>" is a predetermined character string indicating the end of the text data. The text generator 10 generates new replacement text data by these character strings output from the decoder 24 as "Nikkei average, increase width <price1> circle exceeds". Then, the replacing unit 14, the predetermined character string "<price1>", the last value of X _long (X _{long, 7,} the day before the closing price) last value of the X _short (X _{short, 62,} the day Replace with the difference of the closing price) to generate text data explaining the fluctuation of the stock price.

As in the language model 20 of this example, time series numerical data including a number sequence associated with the passage of time at different time intervals is input to different encoders, and the outputs are combined and input to the decoder. Text can be generated to explain the variation of the time series numerical data so that it correctly contains the words that depend on the history of the series numerical data. In addition, by inputting not only the output of the encoder but also multiple numerical data to the decoder, a text explaining the fluctuation of the time series numerical data so as to correctly include words that depend on the history of the time series numerical data. Can be generated. Furthermore, by inputting not only the data synthesized by the synthesizer but also the time-series data of the time-series numerical data to the decoder, the time so as to correctly include the words that depend on the history of the time-series numerical data. It is possible to generate text explaining the fluctuation of series numerical data.

According to the text generation device 10 according to the present embodiment, for example, by generating a text explaining the fluctuation of the time-series numerical data so as to correctly include words that depend on the history of the time-series numerical data, for example, "increase". Properly generate expressions that refer to past stock price history, such as words such as "continuation" and "repulsion", and words such as "start", "close", "close ahead", "afternoon", and "close close". It is possible to correctly generate expressions that depend on the time zone, such as.

FIG. 5 is a diagram showing rule D3 for associating a predetermined character string with a type of numerical value related to time-series numerical data. Rule D3 is an example of a predetermined rule stored in the rule storage unit 15 and referenced by the replacement unit 14.

Rule D3 is a rule for associating 12 types of numerical values related to time-series numerical data with respect to 12 types of character strings. Each character string has a one-to-one correspondence with a numerical value related to time-series numerical data. In this example, the string <price1> is associated with the difference between the last value (X _{long, 7)} and the last value of X _short of _{X long (X short, 62)} . Further, the character string <price2> is associated with a value obtained by rounding down the difference between the last _{value of X long and} the last value of X _{short at the tens digit.}

Further, the character string <price3> is associated with the value obtained by rounding down the difference between the last _{value of X long and} the last value of X _{short by} the 100s, and the character string <price4> is _{the last value of X long} . The difference between the last value of X _{long and} the last value of X short is associated with the value rounded up to the tens place, and the character string <price5> is the difference between the last _{value of X long and} the last value of X _{short is 100.} It is associated with the value rounded up to the nearest whole number.

Further, the character string <price6> is _{associated with the last value of X short} , and the character string <price7> is associated with the value obtained by truncating the last value of _{X short} by the 100s digit, and <price8>. The character string> is _{associated with the value obtained by truncating the last value of X short} at the 1000s place, and the character string <price9> is associated with the value obtained by truncating the last value of _{X short at the 10000s place.} Be done. Similarly, the string <price10> is _{associated with the last value of X short} rounded up to the 100s, and the string <price11> rounds up the last value of _{X short to the 1000s.} The character string <price12> is associated with _{the last value of X short} rounded up to the nearest 10,000.

In this way, by associating the types of numerical values related to the time-series numerical data with a predetermined character string, the time-series includes the numerical values obtained as a result of quoting the time-series numerical data and calculating the time-series numerical data. It is possible to generate text that explains the fluctuations in numerical data.

FIG. 6 is a flowchart of processing executed by the text generator 10 according to the present embodiment. First, the acquisition unit 12 acquires the stock price recorded at 5-minute intervals as the first time-series numerical data (S10), and acquires the stock price recorded at one business day interval as the second time-series numerical data (S11). ).

After that, the generation unit 13 inputs the first time-series numerical data and the second time-series numerical data into the language model 20. The language model 20 uses the first preprocessing unit 21a to convert the first time-series numerical data into normalized numerical data, standardized numerical data, and relativized numerical data (S12), and the second preprocessing unit 21b. Converts the second time-series numerical data into normalized numerical data, standardized numerical data, and relativized numerical data (S13). Then, a plurality of first numerical data obtained by converting the first time-series numerical data are input to the first encoder 22a (S14), and a plurality of second numerical data obtained by converting the second time-series numerical data are converted. Numerical data is input to the second encoder 22b (S15).

Further, the compositing unit 23 synthesizes a plurality of first numerical data, a plurality of second numerical data, the output of the first encoder 22a, and the output of the second encoder 22b (S16). After that, the synthesized data and the data related to the time series are input to the decoder 24 (S17).

Of the replacement text data output from the decoder 24, the replacement unit 14 replaces a predetermined character string with a numerical value according to a predetermined rule (S18), and generates text data explaining the variation of the time-series numerical data. This completes the process.

According to the text generation device 10 according to the present embodiment, new replacement text data for explaining the new time-series numerical data is generated by the output of the language model, and a predetermined character string included in the new replacement text data is generated. By replacing the numerical values related to the new time-series numerical data with the numerical values related to the new time-series numerical data according to a predetermined rule, it is not necessary to directly output the numerical values related to the time-series numerical data by the language model, and even if the numerical values change variously, the numerical values can be changed. Text can be generated to explain the fluctuations in the time series numerical data so that the description of the numerical values is correctly included.

FIG. 7 is a diagram showing text generated by the text generator 10 according to the present embodiment. In the figure, it is generated when a conventional model is used as a language model and when a language model 20 according to the present embodiment or a model in which a part of the language model 20 according to the present embodiment is modified is used. Table 1 R1 summarizing the text is shown. The language model 20 according to the present embodiment is the language model 20 described with reference to FIG. 4, and is a model in which the first encoder 22a and the second encoder 22b are MLPs. Further, the first example of the model in which a part of the language model 20 according to the present embodiment is modified is a model of the language model 20 described with reference to FIG. 4, which does not use standardized data, that is, the first preprocessing unit 21a. This is a model for calculating two types of data, normalized data and relativized data by the second preprocessing unit 21b. Further, the second example of the model in which a part of the language model 20 according to the present embodiment is modified is a model of the language model 20 described with reference to FIG. 4 that does not use replacement text data, that is, a time-series numerical value depending on the language model. It is a model that directly generates numerical values related to data. Further, in the third example of the model in which a part of the language model 20 according to the present embodiment is modified, the data related to the time series of the time series numerical data is not input to the decoder 24 of the language model 20 described with reference to FIG. It is a model.

In the figure, in addition to Table 1 R1, an example E of accurate text data is shown. An example E of accurate text data is "Nikkei average close, continuous growth closing price is 16906 yen, which is 32 yen higher."

On the other hand, an example of the text generated when the conventional model is used as the language model is "Nikkei average, the closing price before the fall is 20606 yen, which is 57 yen lower", and the delivery time zone of the text data is incorrect. The point that the stock price is "closed ahead", the difference between the stock price and the previous day is mistakenly expressed as "rebound", the difference from the closing price on the previous day is mistakenly set as "57 yen depreciation", and the current stock price is wrong. It lacks accuracy in that it is set at "20606 yen".

On the other hand, an example of the text generated when the language model 20 according to the present embodiment, which is described second from the top of Table 1 R1, is "Nikkei average, continuous growth close is 16906 yen, which is 32 yen higher. The text data delivery time zone is correctly expressed as "closed", the difference in stock price from the previous day is correctly expressed as "continuation growth", and the difference from the previous day's closing price is correctly calculated as "32 yen higher". The current stock price is correctly quoted as "16906 yen", and all expressions are accurate.

An example of the text generated when the first example of the model in which a part of the language model 20 according to the present embodiment is modified, which is described third from the top of Table 1 R1, is "Nikkei 225, The continuous growth close is 16906 yen, which is 32 yen higher, and the text data delivery time zone is correctly expressed as "close", the difference in stock price from the previous day is correctly expressed as "sequential growth", and the difference from the previous day's closing price is " It is calculated correctly as "32 yen appreciation", and the current stock price is correctly quoted as "16906 yen", and all expressions are accurate. From this, even in the model that calculates two types of data, the data normalized by the first preprocessing unit 21a and the second preprocessing unit 21b and the relativized data, the first preprocessing unit 21a and the second preprocessing unit 21a and the second preprocessing It can be seen that part 21b can generate text data that explains time-series numerical data with an accuracy equal to or higher than that of a model that calculates three types of standardized data, normalized data, and relativized data.

An example of the text generated when the second example of the model in which a part of the language model 20 according to the present embodiment is modified, which is described fourth from the top of Table 1 R1, is "Nikkei 225, The continuous growth close is the yen, which is 28 yen higher, and the text data delivery time zone is correctly expressed as "close", the difference in stock price from the previous day is correctly expressed as "sequential growth", and the difference from the previous day's closing price.を誤って「２８円高」と算出しており、現在の株価が引用できず「＜ｕｎｋ＞円」となっている。 Here, is a character string representing unknown, and indicates that an appropriate word could not be generated. For this reason, if the numerical values related to the time-series numerical data are directly generated by the language model, it is difficult to correctly generate the numerical values related to the time-series numerical data, and not only the expressions involving the calculation of the time-series numerical data but also the expressions. , It turns out that it is also difficult to include citations of time series numerical data.

An example of the text generated when the third example of the model in which a part of the language model 20 according to the present embodiment is modified, which is described in the fifth from the top of Table 1 R1, is "Nikkei 225, The closing price of the previous day is 16906 yen, which is 32 yen higher. The difference is correctly calculated as "32 yen higher", and the current stock price is correctly quoted as "16906 yen". From this, it can be seen that it is difficult to correctly refer to the time zone in which the time-series numerical data was acquired in the model in which the time-series-related data of the time-series numerical data is not input to the decoder 24.

FIG. 8 is a diagram showing index values for evaluating the closeness between the text generated by the text generator 10 according to the present embodiment and the reference text. In the figure, it is generated when a conventional model is used as a language model and when a language model 20 according to the present embodiment or a model in which a part of the language model 20 according to the present embodiment is modified is used. Table 2 R2 summarizes the index values for evaluating the closeness between the text and the reference text. The index value is a value called BLEU (BiLingual Evaluation Understudy), which takes a value from 0 to 1, and the closer it is to 1, the closer it is to the reference text (accurate text). This index value is an example of what is used to evaluate the text, and the text can be evaluated using other index values.

The language model 20 according to the present embodiment is the language model 20 described with reference to FIG. 4, and includes a model in which the first encoder 22a and the second encoder 22b are MLPs, and the first encoder 22a and the second encoder 22b. A model in which CNN is used and a model in which the first encoder 22a and the second encoder 22b are RNNs.

Further, the first example of the model in which a part of the language model 20 according to the present embodiment is modified is the model of the language model 20 described with reference to FIG. 4, which does not use the first time series numerical data, that is, the second time. This model uses only series numerical data. The second example of the model in which a part of the language model 20 according to the present embodiment is modified is a model of the language model 20 described with reference to FIG. 4, which does not use the second time series numerical data, that is, the first time series numerical value. This is a model that uses only data.

Further, the third example of the model in which a part of the language model 20 according to the present embodiment is modified is a model of the language model 20 described with reference to FIG. 4, which does not use normalized data, that is, the first preprocessing unit. This is a model for calculating two types of data, standardized data and relativized data by 21a and the second preprocessing unit 21b. The fourth example of the model in which a part of the language model 20 according to the present embodiment is modified is a model of the language model 20 described with reference to FIG. 4, which does not use standardized data, that is, the first preprocessing unit 21a and the first. 2 This is a model for calculating two types of data, normalized data and relativized data by the preprocessing unit 21b. The fifth example of the model in which a part of the language model 20 according to the present embodiment is modified is a model of the language model 20 described with reference to FIG. 4, which does not use relativized data, that is, the first preprocessing unit 21a and This is a model for calculating two types of data, standardized data and normalized data, by the second preprocessing unit 21b.

Further, in the sixth example of the model in which a part of the language model 20 according to the present embodiment is modified, the time-series numerical data is converted to the decoder 24 in the language model 20 described with reference to FIG. 4 by one or a plurality of methods. This is a model in which one or a plurality of numerical data obtained as described above is not input, that is, a model in which only the output of the first encoder 22a and the output of the second encoder 22b are input to the decoder 24. The seventh example of the model in which a part of the language model 20 according to the present embodiment is modified is a model that does not use replacement text data among the language models 20 described with reference to FIG. 4, that is, time-series numerical data is obtained by the language model. It is a model that directly generates related numerical values. Further, in the eighth example of the model in which a part of the language model 20 according to the present embodiment is modified, the data related to the time series of the time series numerical data is not input to the decoder 24 of the language model 20 described with reference to FIG. It is a model.

The evaluation value of the text generated when the conventional model is used as the language model is "0.244", whereas the evaluation value of the text is generated by the text generator 10 using the language model 20 according to the present embodiment. The evaluation value of the text is "0.415" when MLP is used for the encoder, "0.414" when CNN is used for the encoder, and "0.415" when RNN is used for the encoder. In any case, the evaluation value is significantly improved as compared with the conventional case, and it can be seen that accurate text data can be generated.

The evaluation value of the text generated by using the first example of the model in which the language model 20 according to the present embodiment is partially modified, which is described in the fifth from the top of Table 2, R2, is "0.356". The evaluation value of the text generated by using the second example of the model in which the language model 20 according to the present embodiment is partially modified, which is described in the sixth from the top of Table 2 R2, is "0.397". It can be seen that the evaluation value is improved by using two types of time-series numerical data acquired at different time intervals. It is considered that this is because the text generator 10 according to the present embodiment can correctly generate words that depend on the history of time-series numerical data.

Further, the evaluation value of the text generated by using the third example of the model in which the language model 20 according to the present embodiment is partially modified, which is described in the seventh from the top of Table 2, R2, is "0.424". The evaluation value of the text generated by using the fourth example of the model in which the language model 20 according to the present embodiment is partially modified, which is described in the eighth from the top of Table 2, R2, is "0.424". The evaluation value of the text generated by using the fifth example of the model in which the language model 20 according to the present embodiment is partially modified, which is described in the ninth from the top of Table 2, R2, is "0.408". ". From these facts, when either one of the normalized data and the standardized data and the relativized data are used, the normalized data, the standardized data and the relativized data are all used. It can be seen that more appropriate text data can be generated than in the case. Moreover, it can be seen that the evaluation value deteriorates unless the relativized data is used.

Further, the evaluation value of the text generated by using the sixth example of the model in which a part of the language model 20 according to the present embodiment is modified, which is described in the tenth position from the top of Table 2 R2, is "0.397". The evaluation value of the text generated by using the seventh example of the model in which a part of the language model 20 according to the present embodiment is modified, which is described in the eleventh from the top of Table 2 R2, is "0.313". The evaluation value of the text generated by using the eighth example of the model in which the language model 20 according to the present embodiment is partially modified, which is described in the twelfth from the top of Table 2, R2, is "0.358". ". Based on these facts, a model that inputs only the output of the first encoder 22a and the output of the second encoder 22b to the decoder 24, a model that directly generates numerical values related to time-series numerical data by a language model, and time-series numerical data. It can be seen that when a model in which data related to the time series is not input is used, the index value is worse than when the language model 20 according to the present embodiment is used.

The embodiments described above are for facilitating the understanding of the present invention, and are not for limiting and interpreting the present invention. Each element included in the embodiment and its arrangement, material, condition, shape, size, and the like are not limited to those exemplified, and can be changed as appropriate. In addition, the configurations shown in different embodiments can be partially replaced or combined.

10 ... Text generator, 10a ... CPU, 10b ... RAM, 10c ... ROM, 10d ... Communication unit, 10e ... Input unit, 10f ... Display unit, 11 ... Learning unit, 12 ... Acquisition unit, 13 ... Generation unit, 14 ... Replacement unit, 15 ... Regular storage unit, 20 ... Language model, 21a ... First preprocessing unit, 21b ... Second preprocessing unit, 22a ... First encoder, 22b ... Second encoder, 23 ... Synthesis unit, 24 ... Decoder , 30 ... user terminal, 40 ... stock distribution server, 100 ... text generation system, D1 ... replacement text data, D2 ... time series numerical data, D3 ... rule, E ... accurate text data example, N ... communication network, R1 … Table 1, R2… Table 2

Claims (10)

A text generator that generates text data that explains changes in time-series numerical data, including a sequence of numbers associated with the passage of time.
The time-series numerical data is input using the replacement text data in which the numerical values related to the time-series numerical data among the text data are replaced with predetermined character strings according to a predetermined rule and the time-series numerical data as learning data. In this case, the learning unit that trains the language model so as to output the replacement text data,
A generation unit that inputs new time-series numerical data to the language model learned by the learning unit and generates new replacement text data by the output of the language model.
A replacement unit that replaces the predetermined character string included in the new replacement text data with a numerical value related to the new time-series numerical data according to the predetermined rule.
A text generator equipped with. Numerical values related to the time series numerical data are
Of the sequence of numbers included in the time-series numerical data, the numerical value associated with a predetermined time point and
The difference between the numerical values associated with different time points in the sequence of numbers included in the time series numerical data, and
Of the sequence of numbers included in the time-series numerical data, the numerical value associated with a predetermined time point is rounded down to a predetermined digit, and
Of the sequence of numbers included in the time-series numerical data, the difference between the numerical values associated with different time points is rounded down to a predetermined digit, and
Of the sequence of numbers included in the time-series numerical data, the numerical value associated with a predetermined time point is rounded up to a predetermined digit, and
Includes at least one of the numerical values obtained by rounding up the difference between the numerical values associated with different time points in the sequence of numerical values included in the time-series numerical data by a predetermined digit.
The predetermined rule is a rule for associating the type of numerical value related to the time series numerical data with the predetermined character string.
The text generator according to claim 1. The time-series numerical data includes a first time-series numerical data including a sequence of numbers associated with the passage of time in the first interval, and a sequence of numbers associated with the passage of time in a second interval longer than the first interval. Including the second time series numerical data including,
The text generator according to claim 1 or 2. The generation unit inputs one or a plurality of numerical data having a one-to-one correspondence to the one or a plurality of methods, which is obtained by converting the time series numerical data by one or a plurality of methods, into the language model.
The text generator according to any one of claims 1 to 3. The one or more numerical data is
Numerical data obtained by normalizing the sequence of numbers included in the time-series numerical data to a predetermined numerical range, and
Numerical data obtained by standardizing the time-series numerical data using the average value and standard deviation of the sequence included in the time-series numerical data, and
Numerical data obtained by relativizing the sequence included in the time-series numerical data with respect to the reference value, using the numerical value associated with a predetermined time point as the reference value among the sequence included in the time-series numerical data.
Including at least one of
The text generator according to claim 4. The language model is
One or more corresponding to the one or more methods, which is obtained by converting the first time series numerical data including a sequence of numbers associated with the passage of time in the first interval by one or a plurality of methods. The first encoder to which the first numerical data is input and
One-to-one with the one or more methods obtained by converting the second time series numerical data including the sequence associated with the passage of time in the second interval longer than the first interval by the one or more methods. A second encoder into which one or more second numerical data corresponding to
A compositing unit that synthesizes the output of the first encoder and the output of the second encoder,
A decoder in which the data synthesized by the synthesis unit is input and the replacement text data is output is included.
The text generator according to any one of claims 1 to 5. The synthesizing unit synthesizes the output of the first encoder, the output of the second encoder, the one or more first numerical data, and the one or more second numerical data.
The text generator according to claim 6. Data synthesized by the synthesis unit and data related to the time series of the time series numerical data are input to the decoder.
The text generator according to claim 6 or 7. A text generation method in which a computer equipped with a hardware processor and memory generates text data that explains fluctuations in time-series numerical data including a sequence of numbers associated with the passage of time.
The time-series numerical data is input using the replacement text data in which the numerical values related to the time-series numerical data among the text data are replaced with predetermined character strings according to a predetermined rule and the time-series numerical data as learning data. In that case, the language model is trained to output the replacement text data, and
Input new time-series numerical data into the trained language model, and generate new replacement text data explaining the new time-series numerical data by the output of the language model.
Replacing the predetermined character string included in the new replacement text data with a numerical value related to the new time-series numerical data according to the predetermined rule,
How to generate text to execute. A computer equipped with a text generator that generates text data that explains the fluctuations in time-series numerical data, including a sequence of numbers associated with the passage of time.
The time-series numerical data is input using the replacement text data in which the numerical values related to the time-series numerical data among the text data are replaced with predetermined character strings according to a predetermined rule and the time-series numerical data as learning data. In this case, the learning unit that trains the language model so as to output the replacement text data.
A generation unit that inputs new time-series numerical data to the language model learned by the learning unit and generates new replacement text data for explaining the new time-series numerical data by the output of the language model, and the generation unit. A replacement unit that replaces the predetermined character string included in the new replacement text data with a numerical value related to the new time-series numerical data according to the predetermined rule.
A text generator that functions as.

Images