JP7245314B2 - Information processing device and program - Google Patents

Description

The present invention relates to an information processing device and a program using a machine learning model.

A technique is known in which different random seeds are set for each of a plurality of machine learning models made up of a plurality of neural networks, etc. to improve the accuracy of estimation results obtained by ensemble processing of the plurality of machine learning models. (For example, Patent Document 1).

WO2019/171416

On the other hand, it is known that every time the seed set in the machine learning model is changed, the output result of the machine learning model may change significantly. In such cases, the developer of the machine learning model must repeatedly adjust the machine learning model itself and verify the adjusted machine learning model in order to ensure the stability of the output of the machine learning model. must.

This is the same as in the case of performing ensemble processing of a plurality of machine learning models in which random seeds different from each other are set as in the conventional example described above. In other words, when there is a large variation in the output estimation results among multiple machine learning models, there is a possibility that the desired estimation accuracy cannot be achieved, and the machine learning models need to be readjusted and revalidated. I had a problem with the load.

The present invention has been made in view of the above circumstances, and is an information processing device that can improve the accuracy of estimation results obtained by ensemble processing of a plurality of machine learning models while reducing the need for adjustment and verification of machine learning models. and to provide programs.

One aspect of the present invention for solving the problems of the above conventional example is an information processing device, which is a machine learning model that receives input data and outputs predetermined estimated information, wherein the machine learning parameters are holding means for holding information related to N machine learning models, which are integers of two or more set differently from each other; Learning processing means for performing machine learning on each of the models; Evaluation acquisition means for acquiring evaluation information obtained by evaluating each of the machine learning models; Based on the obtained evaluation information, among the N machine learning models, and a selection means for selecting an integer number n of machine learning models equal to or more than 2 and less than the N, and executing a predetermined estimation process using the selected plurality of machine learning models.

According to the present invention, it is possible to improve the accuracy of estimation results obtained by ensemble processing of a plurality of machine learning models while reducing the need for adjustment and verification of machine learning models.

1 is a block diagram showing a configuration example of an information processing device according to an embodiment of the present invention; FIG. 1 is a functional block diagram showing an example of an information processing device according to an embodiment of the present invention; FIG. FIG. 4 is a flow chart showing an example of machine learning processing by the information processing apparatus according to the embodiment of the present invention; FIG. 4 is a flow chart showing an example of setting processing of a machine learning model by the information processing apparatus according to the embodiment of the present invention; FIG. 4 is a flow chart showing an example of ensemble processing by the information processing device according to the embodiment of the present invention; FIG. 4 is an explanatory diagram showing an example of information generated by the information processing device according to the embodiment of the present invention; FIG. 10 is a flow chart showing another example of setting processing of a machine learning model by the information processing apparatus according to the embodiment of the present invention;

An embodiment of the present invention will be described with reference to the drawings. An information processing apparatus 1 according to an embodiment of the present invention includes a general computer device including a control unit 11, a storage unit 12, an operation unit 13, a display unit 14, and a communication unit 15, as illustrated in FIG. It can be realized by using

In the following description, an example in which stock trends are estimated by the information processing device 1 will be described, but this example is only an example, and the information processing device 1 can also be applied to estimation of other information.

The control unit 11 is a program control device such as a CPU, and executes programs stored in the storage unit 12 . In this embodiment, the control unit 11 stores in the storage unit 12 N machine learning models that receive input data and output predetermined estimation information. Here, N is an integer of 3 or more. These multiple machine learning models are set with different machine learning parameters, as will be described later.

The control unit 11 also receives input data to be input to the plurality of machine learning models, and causes each of the machine learning models to perform machine learning using the received input data. The control unit 11 acquires evaluation information obtained by evaluating each of the plurality of machine learning models, and selects n machine learning models out of the N machine learning models based on the acquired evaluation information. Here, n is an integer of 2 or more and less than N.

Then, the control unit 11 executes predetermined estimation processing using the selected plurality of machine learning models. A detailed operation of the control unit 11 will be described later.

The storage unit 12 is a memory device, a disk device, or the like, and holds programs executed by the control unit 11 . In this embodiment, the program may be provided by being stored in a computer-readable and non-temporary recording medium such as a DVD, and stored in the storage section 12 . The storage section 12 also operates as a work memory for the control section 11 .

The operation unit 13 is a mouse, a keyboard, or the like, receives an instruction operation from the user, and outputs information representing the content of the instruction operation to the control unit 11 . The display unit 14 is a display or the like, and displays information according to instructions output by the control unit 11 .

The communication unit 15 is a network interface or the like, and transmits information to a server or the like connected via a network according to instructions output by the control unit 11 . The communication unit 15 also receives information from a server or the like connected via a network and outputs the information to the control unit 11 .

Next, operation of the control unit 11 will be described. In the present embodiment, the control unit 11 executes a program stored in the storage unit 12 to functionally function as a machine learning model setting unit 21 and a learning processing unit 22 as illustrated in FIG. , an evaluation acquisition unit 23, a selection unit 24, and an ensemble processing unit 25 are implemented.

The machine learning model setting unit 21 sets machine learning parameters for N machine learning models that output predetermined estimation information, and stores information related to the machine learning models in the storage unit 12 . Here, N is an integer of 3 or more. In one example of the present embodiment, the machine learning model setting unit 21 sets different seeds of information related to the machine learning model as machine learning parameters.

Here, the seed of information related to the machine learning model is a quantity that characterizes the machine learning model. is. That is, in one example of the present embodiment, the machine learning model setting unit 21 sets a model composed of a multi-layered neural network as each of the N machine learning models, and sets initial values such as weights and bias values between the respective layers. are set randomly, for example, to be different seeds from each other.

In one example of the present embodiment, input data input to the N machine learning models of the information processing device 1 includes, for example, a plurality of feature amounts. In this example, it is assumed that a plurality of feature quantities are calculated based on indices (stock index information) of a plurality of countries. Specifically, the input data is a vector quantity of a predetermined dimension, and various widely known methods in the field of machine learning technology can be used to generate a vector of a predetermined dimension reflecting a plurality of feature values as the input data. Therefore, a detailed description is omitted here.

In addition to or instead of the index of the market to be estimated, indexes of other countries are also used here. This is because it is thought that prediction accuracy can be improved by including in the input data features based on index data from other countries, which change even while the target market for estimation is closed.

The learning processing unit 22 executes machine learning processing using input data and correct data corresponding to the input data for each of the N machine learning models set by the machine learning model setting unit 21 .

That is, the learning processing unit 22 inputs input data to each of N machine learning models set with different seeds, and based on the result of comparison between the output and the correct data corresponding to the input data, Update the weights and bias information between nodes of the corresponding machine learning model by backpropagation. Input data to be input to the N machine learning models may be common.

In this example, the input data is input to the machine learning model using feature values based on indices of markets in a plurality of countries before the opening of the market in Japan on a certain date D. In this example, the correct data corresponding to the output of the learning processing unit 22 is the actual value of a predetermined index, such as the ratio of the closing price to the opening price of the Nikkei Stock Average (hereinafter referred to as "N225") on the date D. The data based on is regarded as the correct data. Hereinafter, the ratio of the closing price to the opening price of the index will be referred to as the rate of change of the index information. The rate of change estimated here or used as correct data may be the ratio of the high or low price to the opening price.

The evaluation acquisition unit 23 acquires evaluation information V obtained by evaluating each of the N machine learning models machine-learned by the learning processing unit 22 when a predetermined evaluation timing arrives. Specifically, the evaluation acquisition unit 23 acquires evaluation information V obtained by evaluating each of the N machine learning models subjected to the machine learning process at the timing when it is determined that the machine learning process in the learning processing unit 22 has ended. Generate.

This evaluation information V may be, for example, an evaluation value such as widely known estimation accuracy, namely Accuracy, or precision, or the like. As an example, as in the above example of the present embodiment, when estimating an index such as the rate of change of N225, how accurately can the rate of change, which is the ratio of the closing price to the opening price of the index, be correctly estimated? A value that expresses the accuracy, that is, the correct answer rate or the winning rate is used as the estimation accuracy.

The selection unit 24 selects n machine learning models from among the N machine learning models using a predetermined condition based on the evaluation information V acquired by the evaluation acquisition unit 23 . Here, n is an integer of 2 or more and less than N. For example, the selection unit 24 may select n machine learning models whose evaluation information V does not vary from each other and whose evaluation of the evaluation information V is judged to be sufficiently high.

In one example of the present embodiment, the selection unit 24 arranges the evaluation information V representing the estimation accuracy of each of the N machine learning models in descending order, that is, in descending order of the highest evaluation. Then, the selection unit 24 selects the top n machine learning models whose evaluation information V representing estimation accuracy is highly evaluated.

The ensemble processing unit 25 executes predetermined estimation processing using the selected n machine learning models. In the example of the present embodiment, the ensemble processing unit 25 accepts a plurality of types of index information obtained on a date D prior to the date to be estimated, and selects Input data to be input to each of the n machine learning models is generated.

The ensemble processing unit 25 inputs the generated input data to each of the selected n machine learning models, and obtains a result of estimation by each machine learning model. Here, the result of estimation is, for example, information related to country index information input as correct data during machine learning, as described above. In one example of the present embodiment, the ensemble processing unit 25 estimates the rate of change of N225 on a certain date D based on the index information of each country until just before the opening of the market in Japan on the date D. Then, the ensemble processing unit 25 outputs a transaction sign of "buy", "sell", or "N/A (no action, do nothing)" obtained based on the estimated rate of change of N225. .

Specifically, the ensemble processing unit 25 uses the results of individual estimation by the selected n machine learning models, and if the estimated change rate of N225 is equal to or greater than a predetermined first threshold, the “buy ” output the transaction sign. Also, the ensemble processing unit 25 outputs a “sell” transaction sign if the estimated rate of change of N225 is equal to or less than a predetermined second threshold value. Furthermore, the ensemble processing unit 25 outputs a transaction sign of "N/A" when the estimated change rate of N225 is less than the first threshold and greater than the second threshold. Here, the first threshold is a numerical value representing a percentage greater than 0, such as +1% or +0.1%, and the second threshold is a numerical value representing a percentage less than 0, such as -1%. Alternatively, it is -0.1%. These thresholds are determined empirically or experimentally.

[motion]
Next, an operation example of the information processing apparatus 1 according to the example of the present embodiment will be described. In the example below, in order to decide whether to purchase an index fund that tracks N225, one of Japan's economic indicators, the Japanese market opens today before 9:00 am. Estimate the value related to N225 later, for example, its rate of change, and use "Buy", "Sell", and "N/A" to represent the trades that should be made during the opening of the Japanese market from 9:00 am to 3:00 pm today. An example of outputting one of the transaction signs will be described.

In one example of the present embodiment, the information processing apparatus 1 first performs a process of setting a plurality of machine learning models at the stage of the machine learning process, as illustrated in FIG. 3 (S11).

The details of this processing are specifically illustrated in FIG. In one example of the present embodiment, the information processing apparatus 1 determines a plurality of pieces of index information that serve as sources of input data for machine learning processing as processing for setting a plurality of machine learning models (S111). In this step S111, the information processing apparatus 1 determines the index information of the markets of a plurality of countries with many transactions with Japan as the information to be the source of the input data. Note that in this step, the input of information specifying the index information that is the source of the input data is received from the user, and the input information is used to create a plurality of index information that is the source of the input data for the machine learning process. Index information may be determined.

The information processing apparatus 1 acquires a plurality of pieces of index information IA, IB, IC, . Here, i is a natural number i=1, 2, . . . The index information IA, IB, IC, . stock price), etc. These information are generally obtained through a network.

Further, in step S112, the information processing apparatus 1 acquires, as new correct data, the rate of change of N225 to be estimated in a predetermined market that opens on or after the plurality of dates.

The information processing device 1 obtains a feature amount for each of the obtained pieces of index information I (S113). In the present embodiment, the information processing apparatus 1 provides (a) the ratio (%) of the closing price to the opening price of the index information I on the date Di′ before the opening of the Japanese market on the date Di;
(b) ratio of closing price to high price (%);
(c) the ratio of the closing price to the low price (%), and
(d) Ratio (%) of the opening price of the date Di to the closing price of the previous day of the relevant date Di';
(e) the ratio (%) of the closing price on date Di' to the closing price on the previous day on date Di';
(f) volatility on date Di';
A feature amount representing each of the is calculated.

The information processing apparatus 1 next sets a plurality of machine learning models set to different seeds (S114). Specifically, here, the information processing apparatus 1 initializes 32 machine learning models with mutually different seeds and stores them in the storage unit 12 .

As described above, the machine learning model here is, for example, a model made up of a multi-layered neural network, and the information processing apparatus 1 uses a predetermined method to determine the weights and bias values between nodes in each layer of each machine learning model. etc. are set to be different from each other, for example, randomly.

The information processing apparatus 1 returns to step S12 in FIG. 3 and executes machine learning processing for each of the 32 machine learning models set here (S12).

In this step S12, the information processing apparatus 1 generates input data using the feature amount on the date Di obtained in step S113 for each of the 32 machine learning models set in step S114. Then, the information processing apparatus 1 executes machine learning processing using the generated input data and correct data corresponding to the input data.

Here, the information processing apparatus 1 inputs the generated input data to each machine learning model. The information processing apparatus 1 obtains the difference between the output of the machine learning model that receives the input data and the correct data corresponding to the input data. Then, the information processing apparatus 1 updates weights and bias information between nodes of the machine learning model by back propagation based on the difference. Since this backpropagation process is widely known, detailed description thereof is omitted here.

The information processing apparatus 1 generates evaluation information V representing estimation accuracy for each of the 32 machine learning models machine-learned in step S12 (S13). Then, the information processing apparatus 1 arranges the evaluation information V representing the estimation accuracy of each of the 32 machine learning models in descending order, that is, in descending order of evaluation, and ranks the top n machine learning models, for example, 4 machine learning models. is selected (S14).

The information processing device 1 outputs information specifying each of the n machine learning models selected in step S14 (S15).

Thereafter, the information processing device 1 executes an estimation process of performing estimation using a plurality of selected machine learning models. In this process, as exemplified in FIG. 5, the information processing device 1 receives index information IA, IB, . Generate input data to feed into a machine learning model.

The information processing apparatus 1 inputs the obtained input data to each of the selected n machine learning models, and obtains the estimation result of each machine learning model (S21).

In this example of the present embodiment, each machine learning model is in a state of being machine-learned to estimate the change rate of N225, so the estimation result of the machine learning model obtained by the information processing apparatus 1 is the target It is an estimate of the change rate of N225 after the market opens.

Then, the information processing device 1 executes ensemble processing of each machine learning model. Specifically, based on the rate of change of N225 estimated by each machine learning model, the information processing apparatus 1, for example, when the estimated rate of change of N225 is greater than or equal to a predetermined first threshold value, " Obtain a trade signal of "buy" and "sell" if below a predetermined second threshold. If the estimated N225 change rate is less than the first threshold and greater than the second threshold, the information processing device 1 obtains a transaction sign of "N/A".

The information processing apparatus 1 thus obtains the transaction signs of "buy", "sell", and "N/A" corresponding to individual estimation results obtained from each machine learning model. The information processing apparatus 1 accumulates the numbers of "buy", "sell", and "N/A" transaction signs, respectively, and determines the transaction sign with the largest accumulated value as the transaction sign to be output (S22). . This means that the trading signature to be output is determined by a majority vote. When the number of "buy" transaction signs and the number of "sell" transaction signs are the same and the number is greater than the number of "N/A" transaction signs, the information processing device 1 may determine a trade sign of "N/A" as the trade sign to output. Then, the information processing apparatus 1 outputs the transaction signature determined in step S22 and presents it to the user (S23).

The user can refer to this transaction signature to set the transaction. Alternatively, the information processing device 1 may perform a transaction at a predetermined time, such as 15:00, without any manual operation, based on the transaction signature obtained in step S22. Specifically, when the "buy" transaction sign is obtained in step S22, the information processing apparatus 1 places an order to purchase a predetermined unit of funds linked to N225 at 15:00. Further, when the "sell" transaction sign is obtained in step S22, the information processing apparatus 1 places an order to sell a predetermined unit of the fund linked to N225 at 15:00. Furthermore, the information processing apparatus 1 does not conduct transactions without human intervention throughout the day when the transaction signature of "N/A" is obtained in step S22.

In this manner, the information processing apparatus 1 of the present embodiment uses N machine learning models that are machine-learned using different machine-learning parameters. Then, the information processing apparatus 1 selects a predetermined number of machine learning models with relatively close machine learning evaluation results and high evaluations. The information processing device 1 performs ensemble processing of the selected machine learning model. As a result, the information processing apparatus 1 according to the present embodiment can suppress variations in the estimation results and reduce the need for adjustment and verification of the machine learning model. In addition, the information processing apparatus 1 of the present embodiment can improve the accuracy of estimation results obtained by ensemble processing of a plurality of machine learning models.

[Modified example of seed determination processing]
In the description so far, the seeds of the N machine learning models set by the machine learning model setting unit 21 are different from each other, so that they are set randomly. However, in the present embodiment, the seeds of the N machine learning models may be different from each other as follows instead of being simply randomized.

In one example of this embodiment, the machine learning model is a model consisting of a multi-layered neural network. In this case, the seed set by the machine learning model setting unit 21 initializes initial values of weights and biases between layers of the neural network. Hereinafter, the weight vector between the j-th layer and the j+1-th layer in the i-th machine learning model is written as Wi,j, and the bias is written as bi,j.

The machine learning model setting unit 21 makes the seeds of the N machine learning models different from each other. However, the machine learning model setting unit 21 makes a part of the seed of each machine learning model common to a part of the seed of other machine learning models. That is, the machine learning model setting unit 21 sets the values of some of Wi,j and bi,j to be equal to the values of Wi',j and bi',j for any i. where i'≠i.

In order to set such seeds, the machine learning model setting unit 21 performs, for example, the following processing. The machine learning model setting unit 21 randomly sets the weight W1,j and the bias b1,j between the layers of the first machine learning model among the N machine learning models.

The machine learning model setting unit 21 sets a part of the weight W2,j and the bias b2,j between each layer of the second machine learning model, for example, the weight W2,1 and the bias between the first layer and the second layer. b2,1 is set by copying the weight W1,1 and the bias b1,1 between the corresponding layers of the first machine learning model. In addition, the machine learning model setting unit 21 randomly sets the weight W2,k and the bias b2,k (k=2, 3, . . . ) between the layers other than the copied weight and bias.

Hereafter, the machine learning model setting unit 21 sets the weight Wi,j and the bias bi,j between each layer of the i-th machine learning model, which is part of the The weight Wi,m and the bias bi,m are set by copying the weight Wi-1,m and the bias bi-1,m between the corresponding layers of the (i-1)th machine learning model. Here, i is an integer of 2 or more, and m=(i+1) mod (M−1)+1. Note that M is the number of layers of the machine learning model, and a mod b is the remainder obtained by dividing a by b. That is, m is a value that repeats m=1, 2, . . . M−1, 1, 2, . The machine learning model setting unit 21 randomly determines weights and biases between layers of the i-th machine learning model other than the weights and biases copied above.

In this way, the weights and biases between layers of the i-th machine learning model are the same as the weights and biases between the corresponding layers of the (i−1)-th machine learning model, and the weights and biases between other layers will be different.

According to this example, a plurality of machine learning models are set in which only some of the seeds overlap, and it is expected to obtain machine learning models that produce similar outputs while the results of learning processing differ from each other.

Note that the seed setting example described here is just an example. The operation of the machine learning model setting unit 21 is
- the seeds of the N machine learning models are different from each other;
- At least part of the seed of each machine learning model is set so as to be common to the seeds of other machine learning models;
Anything can be used as long as it satisfies the following conditions.

[Other examples of machine learning parameters]
Moreover, the machine learning parameters in the embodiment of the present invention are not limited to the seeds of the N machine learning models set by the machine learning model setting unit 21 . A machine learning parameter in the embodiment of the present invention may be the type of input data to be input to the machine learning model.

In this example, the machine learning model setting unit 21 uses a plurality of types of information as information that is the source of input data. As an example, the machine learning model setting unit 21 selects one or more different index information from a plurality of pieces of index information related to markets in different countries, and combines the selected index information in a predetermined number X ways. only create.

The machine learning model setting unit 21 also calculates a plurality of feature amounts based on each of the selected index information. Then, the machine learning model setting unit 21 selects at least some of the calculated plurality of feature amounts, and creates a predetermined number Y of combinations of the selected feature amounts. Here, as an example, it is assumed that the product of X and Y, X×Y, is equal to the number N of machine learning models.

The machine learning model setting unit 21 uses the X×Y=N combination of feature amounts as input data for each of the N machine learning models for machine learning processing of the N machine learning models. Here, the N combinations of feature amounts may be created by the machine learning model setting unit 21 based on predetermined conditions without being subjected to human operation, or combinations explicitly specified by the user may be used. may be created by the machine learning model setting unit 21.

The operation of this machine learning model setting unit 21 will be described using a more specific example. In one example of the present embodiment, the information processing apparatus 1 uses index information IA(D1), IA(D2), . D1), IB (D2), . . . , Ii (Dk), . Here, for each index information Ii (Dk),
(a) ratio of closing price to opening price (%),
(b) ratio of closing price to high price (%);
(c) the ratio of the closing price to the low price (%), and
(d) Ratio (%) of the opening price of the date Dk to the closing price of the previous day of the relevant date Dk,
(e) the ratio (%) of the closing price of date Dk to the closing price of the previous day of date Dk;
(f) volatility on date Dk;
A plurality of types of feature amounts such as are calculated.

That is, here, the control unit 11 receives a plurality of pieces of index information IA, IB, IC, . In the above example, the control unit 11 calculates the number of index information items I×6 types of feature amounts for each of a plurality of past dates D1, D2, . In (d) and (e) above, the ratio between the previous day's closing price and the next day's market information is calculated because the index It takes into consideration that information may change.

In addition, the machine learning model setting unit 21 selects dates after the date Dk as correct data corresponding to input data including feature amounts calculated based on one or more pieces of first index information Ii (Dk) on the date Dk. obtain information related to the second index information in . Here, the second index information shall be index information of a country to be estimated by the machine learning model, and the first index information shall include index information of a country different from the country to be estimated.

Then, as illustrated in FIG. 6, the machine learning model setting unit 21 selects part of the index information IA, IB, IC, ID, . A combination x of different index information is created. Further, the machine learning model setting unit 21 enumerates a plurality of combinations y of types (a) to (f) of feature amounts calculated based on each combination of index information. In the example of Figure 6,
Combination 1: (a, b, c)
Combination 2: (a, b, c, f)
Combination 3: (a, b, c, d, f)
Combination 4: (a, b, c, e, f)
are listed respectively. Note that in FIG. 6, a plurality of combinations of feature amounts are set by multiplying a combination of each index information and a combination of types of feature amounts, which will be described later. The evaluation information V of the machine learning model machine-learned by inputting the input data based on each is also shown. This evaluation information V will be described later.

The machine learning model setting unit 21 corresponds to each of X×Y=N combinations of feature amounts, which are X combinations of index information prepared here and Y combinations of types of feature amounts. N machine learning models are initialized and stored in the storage unit 12 .

In this example as well, the machine learning model may be, for example, a model consisting of a multilayer neural network. Corresponds to model initialization. A widely known method can be adopted for this initialization method.

The learning processing unit 22 executes machine learning processing using input data corresponding to each of the N machine learning models set by the machine learning model setting unit 21 and correct data corresponding to the input data.

That is, the learning processing unit 22 performs the above combination 1 for each of the index information IA, IB, IC, IH of each date D1, D2, . Let the input data containing the feature amounts (a), (b), and (c) of the type concerned be the corresponding input data.

Further, the learning processing unit 22 obtains the output of the first machine learning model when the input data is input, and the output obtained by inputting the input data related to the date Di to the first machine learning model and the correct data corresponding to the input data on the date Di, and based on the result of the comparison, weights and bias information between nodes of the first machine learning model are updated by back propagation.

The learning processing unit 22 similarly inputs the input data corresponding to the j-th machine learning model for the j-th machine learning model, and compares the output with the correct data corresponding to the input data. Based on the result of , the weight and bias information between the nodes of the j-th machine learning model are updated by back propagation. where j=2,3...N.

The evaluation acquisition unit 23 acquires evaluation information V obtained by evaluating each of the N machine learning models machine-learned by the learning processing unit 22 when a predetermined evaluation timing arrives. Specifically, the evaluation acquisition unit 23 acquires evaluation information V obtained by evaluating each of the N machine learning models subjected to the machine learning process at the timing when it is determined that the machine learning process in the learning processing unit 22 has ended. Generate.

This evaluation information V may be, for example, an evaluation value such as widely known estimation accuracy, namely Accuracy, or precision, or the like. As an example, when estimating a predetermined index as in the above example of the present embodiment, a value representing how accurately the change rate of the closing price with respect to the opening price of the predetermined index was estimated, that is, the accuracy rate, Alternatively, the winning percentage is used as the estimation accuracy.

By the operation of the evaluation acquisition unit 23, the information processing apparatus 1 of the present embodiment arranges X combinations x of index information in the row direction and indexes related to the combinations in the column direction, as illustrated in FIG. X×Y=N pieces of evaluation information V are obtained by arranging Y combinations y of types of feature amounts calculated based on each piece of information. Each piece of this evaluation information V is evaluation information for each of the N machine learning models.

The selection unit 24 selects n machine learning models from among the N machine learning models using a predetermined condition based on the evaluation information V acquired by the evaluation acquisition unit 23 . Here, n is an integer of 2 or more and less than N.

In one example of the present embodiment, the selection unit 24 arranges the evaluation information V representing the estimation accuracy of each of the N machine learning models in descending order, that is, in descending order of the highest evaluation. Then, the selection unit 24 selects the top n machine learning models whose evaluation information V representing estimation accuracy is highly evaluated.

The ensemble processing unit 25 executes predetermined estimation processing using the selected n machine learning models. In the example of the present embodiment, the ensemble processing unit 25 receives the index information IA(D), IB(D), . Similarly, generate corresponding input data for each of the n selected machine learning models.

For example, the feature values (a), (b), and (c) included in each index information IA, IB, IC, and IH of the day before and two days before the date to be estimated are extracted, and the extracted feature values Machine learning model L1 machine-learned using input data based on a combination of and from index information IA, IB, IC, and IH, feature amounts (a), (b), (c), and (f) included in each is extracted, and a machine learning model L2 that has undergone machine learning using input data based on the combination of the extracted feature amounts is selected. In this case, for the machine learning model L1, the ensemble processing unit 25 performs index information IA(D), IB(D), IC(D), and IH(D) on dates D before the date to be estimated. and extracts feature amounts (a), (b), and (c) from each, and generates input data based on the extracted feature amounts.

Also, for the machine learning model L2, the feature values (a), (b), (c), and (f) are extracted, and input data is generated based on the extracted feature amounts.

The ensemble processing unit 25 similarly extracts the same type of information as that used when performing machine learning on the selected machine learning model. That is, in this example, the ensemble processing unit 25 extracts the same type of feature amount from the index information of the same country as the index information of the country used in the machine learning. Then, the ensemble processing unit 25 generates input data based on the extracted feature amount.

The ensemble processing unit 25 inputs the input data generated corresponding to the selected machine learning model, and obtains the result of estimation by each machine learning model. Here, the result of estimation is, for example, the country index information input as correct data during machine learning, as described above, and specifically, the change rate of N225 on the day after the date D described above. Then, the ensemble processing unit 25 selects one of the transaction signs of "buy", "sell", and "N/A (no action, do nothing)" based on the rate of change that is the result of the estimation. Output.

Specifically, the ensemble processing unit 25 uses the results of individual estimation by the selected n machine learning models, and if the estimated change rate of N225 is equal to or greater than a predetermined first threshold, the “buy ” output the transaction sign. Also, the ensemble processing unit 25 outputs a “sell” transaction sign if the estimated rate of change of N225 is equal to or less than a predetermined second threshold value. Furthermore, the ensemble processing unit 25 outputs a transaction sign of "N/A" when the estimated change rate of N225 is less than the first threshold and greater than the second threshold. Here, the first threshold is a numerical value representing a percentage greater than 0, such as +1% or +0.1%, and the second threshold is a numerical value representing a percentage less than 0, such as -1%. Alternatively, -0.1% or the like is used. These thresholds are determined empirically or experimentally.

The ensemble processing unit 25 obtains the "buy", "sell", and "N/A" transaction signs corresponding to the individual estimation results obtained from each machine learning model. Then, the ensemble processing unit 25 outputs the transaction signature obtained in the largest number among them.

However, the processing of the ensemble processing unit 25 described here is an example, and another method, for example, averaging individual estimation results by each machine learning model to obtain an estimated value of the rate of change of N225, and the estimated value If the N225 at the time of opening today is equal to or greater than a predetermined first threshold value, a transaction signal of "buy" is output, and if the value is equal to or less than a predetermined second threshold value, a transaction signal of "sell" is output. . Also in this example, if the estimated value is less than a predetermined first threshold value and greater than a second threshold value, a transaction sign of "N/A" is output.

[Selection processing example]
In the examples so far, as a method of selecting a machine learning model to be used for estimation processing during machine learning processing, evaluation information V of N machine learning models machine-learned with mutually different machine learning parameters is used. are obtained, arranged in the order of the evaluation information V, and the top n machine learning models are selected. However, the selection method in this embodiment is not limited to this method.

For example, regardless of the index information, the information processing apparatus 1 can determine the average value, median value, maximum value, or the like of the evaluation values as the evaluation information V of the machine learning model to which the input data having the same combination of feature amounts is input. Find statistics. The information processing apparatus 1 finds a combination of feature amounts that indicates that the statistical amount obtained here has the highest evaluation.

Specifically, if the evaluation information V illustrated in FIG. 3 is
For the combination of feature types (a, b, c), the average of the evaluation values is 52.05 and the standard deviation is 1.23.
For the combination (a, b, c, f), the average of the evaluation values 53.46, the standard deviation 1.72
For the combination (a, b, c, d, f), the average of the evaluation values 52.36, the standard deviation 0.70
For the combination (a, b, c, e, f), the average of the evaluation values 51.77, the standard deviation 0.69
statistic is obtained. The information processing device 1 uses this statistic to determine the type of one feature amount under conditions such as the standard deviation being below a predetermined value (for example, the average value of standard deviations) and the average evaluation value being the highest. choose a combination of In the example here, the combination (a, b, c, d, f) is selected from the above conditions.

Then, the information processing apparatus 1 may select a machine learning model that has undergone machine learning processing using input data based on combinations of the types of feature amounts (a, b, c, d, f) regardless of the index information. .

In another example, the information processing apparatus 1 receives the input data having the same combination of index information regardless of the combination of the types of feature amounts, the average value of the evaluation values as the evaluation information V of the machine learning model, Find a statistic such as the median or maximum value. The information processing apparatus 1 finds a combination of index information indicating that the statistic obtained here has the highest evaluation.

If the evaluation information V illustrated in FIG. 3,
(1) Combination of index information IA, IB, IC and IH (2) Combination of index information IA, IB, IC, IF and IH...
(8) Evaluation values for each combination of index information IA, IB, IC, ID, IE, IF, IG, IH, and II:
(1) average 51.79, standard deviation 0.90
(2) Average 51.98, standard deviation 0.58
(3) average 52.62, standard deviation 0.89
(4) mean 51.81, standard deviation 1.20
(5) average 53.43, standard deviation 0.39
(6) average 51.69, standard deviation 0.31
(7) average 53.23, standard deviation 2.91
(8) average 52.72, standard deviation 0.80
statistic is obtained. The information processing apparatus 1 uses this statistic to select one combination of feature amounts under conditions such as the standard deviation being below the average and the average evaluation value being the highest. In this example, the combination (V) of index information, that is, the combination of index information IA, IB, IC, ID, IE, IH, and II is selected from the above conditions.

Then, the information processing apparatus 1 selects a machine learning model subjected to machine learning processing using input data whose combination of index information is index information IA, IB, IC, ID, IE, IH, and II, regardless of the feature amount. good too.

According to such a selection example, it is possible to obtain a plurality of combinations of types of index information and/or feature amounts with relatively small variations and relatively high evaluation values. Also, in this example, ensemble processing of a plurality of machine learning models machine-learned using combinations of obtained feature amounts is performed. As a result, it is possible to improve the accuracy of ensemble processing results of a plurality of machine learning models while suppressing variations in estimation results and reducing the need for machine learning model adjustment and verification.

However, this example is only an example, and for example, when the evaluation values, which are the evaluation information, are arranged in descending order, a combination of the top n feature amounts may be selected.

[Two step selection]
In addition, in the description so far, for example, a machine learning model that has undergone machine learning is used as it is for estimation processing in order to select a combination of feature amounts used to generate input data, but the present embodiment is not limited to this. do not have.

In one example of the present embodiment, the information processing device 1 performs the second stage selection process illustrated in FIG. It may be started. In this second-stage selection process, the information processing apparatus 1 prepares N′ machine learning models and sets different seeds, which are examples of machine learning parameters (S31). Here, as a method for making the seeds different from each other, the method already described, such as the method of setting the seeds at random, can be adopted.

Also, here, N' is an integer of 3 or more. Then, the information processing apparatus 1 executes the machine learning process of the N' machine learning models using the input data based on the combination of feature amounts selected in the selection process of the first stage (S32).

The information processing device 1 generates evaluation information for each of the N' machine learning models obtained as a result of the machine learning process of step S32 (S33). The evaluation information here may be information on the estimation accuracy already described. The information processing apparatus 1 arranges the N' machine learning models in descending order of the evaluation information generated in step S33, and selects some of them, the top n' machine learning models (S34). Here, n' is an integer of 2 or more and less than N'.

Then, the information processing apparatus 1 uses the n′ machine learning models selected here for estimation processing by ensemble processing.

Note that, in this example, it is not always necessary to have a plurality of combinations of feature amounts to be initially selected. Therefore, in this example, the information processing apparatus 1 selects one combination of feature amounts with which the evaluation information V has the highest value in the process of step S14 in FIG. Depending on the combination, machine learning processing in step S32 may be performed.

The information processing apparatus 1 may also select one combination of feature amounts as follows. That is, the information processing apparatus 1 provides statistics such as the average value, the median value, or the maximum value of the evaluation values as the evaluation information V of the machine learning model to which the input data having the same combination of feature amounts is input, regardless of the index information. ask for quantity. Then, a combination of feature amounts is found in which the statistic obtained here satisfies a predetermined condition. Here, the predetermined condition may be, for example, a condition that the standard deviation is below the average value and the average evaluation value is the highest. As already mentioned, under this condition, the combination (a, b, c, d, f) is selected in the example of FIG.

The information processing apparatus 1 also provides statistics such as the average value, the median value, or the maximum value of the evaluation values as the evaluation information V of the machine learning model to which the input data having the same combination of index information is input, regardless of the feature amount. ask for quantity. The information processing apparatus 1 finds a combination of index information whose statistic obtained here satisfies a predetermined condition.

The predetermined condition here may be a condition such that the standard deviation is below the average value and the average evaluation value is the highest. As already described, in the example of FIG. 6, the combination (V) of index information, that is, the combination of index information IA, IB, IC, ID, IE, IH, and II, is selected from the above conditions.

By these selections, the information processing apparatus 1 sets the combination of index information as index information IA, IB, IC, ID, IE, IH, and II, and the combination of feature amounts as (a, b, c, d, As f), input data is generated and used based on the combination of the feature amounts.

According to an example of such selection, it is possible to obtain a combination of index information and/or feature amounts with relatively small variations and relatively high evaluation values even when index information or feature amounts are missing. can be done.

[When the market is closed]
Further, when the information processing apparatus 1 of the present embodiment operates as the ensemble processing unit 25 and satisfies a predetermined condition, the information processing apparatus 1 may output information without using the machine learning model.

Specifically, the information processing apparatus 1 may output a transaction sign of "N/A" when there is index information that cannot be obtained among the plurality of types of index information that are the sources of the input data. As such a case, it is conceivable that the market related to the index information is closed due to a holiday or the like.

In addition, the information processing device 1, when the market to be estimated, for example, the Japanese market related to N225 in the example so far, is closed due to a holiday or other reasons, the transaction sign of "N/A" to output

[Other Modifications]
Also, in the above example, we described an example in which multiple countries with large import and export transactions with Japan are selected and the index information of the selected countries is used. A plurality of countries may be selected, or a plurality of countries with large export volumes to Japan may be selected. Further, the information processing device 1 may select a plurality of countries with large export or import volumes over a predetermined period (for example, several years or longer).

Also, here, the case where the target of machine learning estimation is the Nikkei Stock Average (N225) is taken as an example, but this is, of course, only an example, and for example, the target of estimation may be TOPIX.

Also, the amount of change in index information may be subject to machine learning estimation. For example, the amount of change in N225 from the previous day may be subject to machine learning estimation.

1 information processing device, 11 control unit, 12 storage unit, 13 operation unit, 14 display unit, 15 communication unit, 21 machine learning model setting unit, 22 learning processing unit, 23 evaluation acquisition unit, 24 selection unit, 25 ensemble processing unit .

Claims (8)

a machine learning model that receives an input of input data and outputs predetermined estimated information, the holding means holding information related to N machine learning models, an integer equal to or greater than 2;
input data of different types, the type of the input data is based on the type of information that is the source of the input data, and the type of feature amount that is calculated based on the information that is the source of the input data learning processing means for receiving determined input data and using the received different types of input data to cause each of the machine learning models to perform machine learning;
an evaluation acquisition means for acquiring evaluation information obtained by evaluating each of the machine learning models;
selection means for selecting n machine learning models, an integer equal to or greater than 1 and less than N, from among the N machine learning models based on the acquired evaluation information;
including
The learning processing means comprises X combinations of one or more different types of information that are sources of the input data, selected from a plurality of types of information that are sources of the input data, and the selected input. Using X×Y=N types of input data obtained by combining Y combinations of a plurality of types of feature amounts calculated based on the information that is the source of the data, N pieces of input data held by the holding means Machine learning each of the machine learning models of
An information processing apparatus that executes a predetermined estimation process using the machine learning model selected by the selection means. The information processing device according to claim 1,
The evaluation acquisition means acquires, as the evaluation information, an evaluation value representing the estimation accuracy of each of the N machine learning models held by the holding means,
The information processing device, wherein the selecting means selects the top n machine learning models in descending order of the evaluation value from the N machine learning models. The information processing device according to claim 1 or 2,
The predetermined estimation process inputs input data to each of the selected n machine learning models, which is an integer of 2 or more, and performs statistical processing on results output from each of the n machine learning models. and an information processing device that acquires an estimation result. The information processing device according to any one of claims 1 to 3,
The learning processing means uses the received input data to cause each of the machine learning models to perform machine learning so as to output information related to market estimation of the country to be estimated,
The input data includes a feature value obtained based on market index information in a country different from the country to be estimated,
The information processing device in which the evaluation information is an accuracy rate of the information related to the market estimation. The information processing device according to claim 4,
The information processing device, wherein the input data includes a feature amount calculated based on at least one of an opening price, a closing price, a high price, and a low price obtained based on the information of the market index. The information processing device according to claim 4 or 5,
An information processing device that outputs information representing recommended trading behavior in the market of the country targeted for the estimation based on the information related to the market estimation. The information processing device according to any one of claims 4 to 6,
The information processing apparatus in which the information related to the market estimation is information related to changes in the market index of the country targeted for the estimation. the computer,
a machine learning model that receives an input of input data and outputs predetermined estimated information, the holding means holding information related to N machine learning models, an integer equal to or greater than 2;
input data of different types, the type of the input data is based on the type of information that is the source of the input data, and the type of feature amount that is calculated based on the information that is the source of the input data learning processing means for receiving determined input data and using the received different types of input data to cause each of the machine learning models to perform machine learning;
an evaluation acquisition means for acquiring evaluation information obtained by evaluating each of the machine learning models;
selection means for selecting n machine learning models, an integer equal to or greater than 1 and less than N, from among the N machine learning models based on the acquired evaluation information;
functioning as a means for executing a predetermined estimation process using the selected plurality of machine learning models,
When functioning as the learning processing means, X combinations of one or more different types of information that are sources of the input data, which are selected from a plurality of types of information that are sources of the input data, and the selection The holding means holds X×Y=N types of input data obtained by combining Y combinations of a plurality of types of feature amounts calculated based on the information that is the source of the input data. A program that machine-learns each of N machine-learning models.

Images