JP6993863B2 - Information processing system and learning method of information processing system - Google Patents

Description

The present invention relates to an information processing system for supporting prediction and judgment using data of companies, humans, and social activities.

Artificial intelligence technology is attracting attention in order to utilize the data that is collected and accumulated from moment to moment in companies and society.

In particular, with regard to image recognition that identifies faces and objects from images by capturing the characteristics of data and voice recognition that identifies languages from the characteristics of voice, a technology called deep learning has realized a significant improvement in accuracy in recent years. ..

With the development of machine learning and artificial intelligence technologies including deep learning, it is expected that it will be possible to predict business and society from data. Forecasting technology using such data and machine learning is expected to be widely used for forecasting corporate performance, demand, accidents and failures. Such prior art includes, for example, Patent Document 1.

JP-A-2017-201526

In machine learning, a model formula for prediction is generated from the data by extracting the characteristics of the events hidden in the past data. This is called "learning" in artificial intelligence (AI) terminology.

However, learning for rare events that occur infrequently is more difficult due to the lack of past performance data.

In conventional machine learning including deep learning, the prediction parameters included in the prediction formula are adjusted so that the prediction error becomes small by using the past actual data. However, in low-frequency events, the phenomenon of "overfitting" is that the prediction parameters are adjusted according to the events that happen to occur in a specific situation, resulting in overfitting, and in new situations the prediction accuracy is rather lowered. Was a big problem.

In a preferred embodiment of the present invention, in an information processing system that inputs original data and produces a prediction result, at least first data and second data are generated from the original data. The first prediction formula that makes a prediction using the first data has at least one parameter, and has a first learner that adjusts the parameter using the first prediction result by the first prediction formula. .. The second prediction formula that makes a prediction using the second data has at least one parameter, and has a second learner that adjusts the parameter using the second prediction result by the second prediction formula. .. Then, there is at least one common parameter in the parameter adjusted by the first learner and the parameter adjusted by the second learner.

In another preferred embodiment of the present invention, a plurality of teacher data consisting of a set of explanatory variables and a first result data are prepared, a plurality of first learning data consisting of a set of explanatory variables are prepared, and a plurality of parameters are used. The first prediction data is obtained from the first training data by using the prediction formula using the prediction parameters, and the prediction parameters are changed so that the error between the first result data and the first prediction data becomes small. To obtain the first prediction parameter. In addition, a plurality of modified data consisting of a set of explanatory variables and a second result data are prepared, a plurality of second learning data consisting of a set of explanatory variables are prepared, and a prediction formula using prediction parameters is used. The second prediction data is obtained from the training data of 2, and the prediction parameters are changed so that the error between the second result data and the second prediction data becomes small, and the second prediction parameter is obtained. Then, at least one of the change in the error with respect to the change in the second prediction parameter and the change in the correlation coefficient between the second result data and the second prediction data with respect to the change in the second prediction parameter is evaluated. , A predetermined parameter is extracted from the prediction parameters, and the first prediction parameter is corrected for the parameter corresponding to the extracted predetermined parameter among the first prediction parameters.

It is possible to avoid the problem of low prediction accuracy for events with a small amount of data, which was inherent in conventional machine learning (including deep learning).

It is a conceptual diagram which shows the information processing system of an Example. It is a block diagram which shows the predictor which constitutes an Example. It is a block diagram which shows the structure of the information processing system of an Example. It is a block diagram which shows the learning apparatus 2 which constitutes the information processing system of an Example. It is a flow chart which shows the processing flow of the learning apparatus 2 of an Example.

The embodiments will be described in detail with reference to the drawings. However, the present invention is not limited to the description of the embodiments shown below. It is easily understood by those skilled in the art that a specific configuration thereof can be changed without departing from the idea or purpose of the present invention.

In the configuration of the invention described below, the same reference numerals may be used in common among different drawings for the same parts or parts having similar functions, and duplicate description may be omitted.

Notations such as "first", "second", and "third" in the present specification and the like are attached to identify components, and do not necessarily limit the number, order, or contents thereof. is not it. Further, the numbers for identifying the components are used for each context, and the numbers used in one context do not always indicate the same composition in the other contexts. Further, it does not prevent the component identified by a certain number from functioning as the component identified by another number.

The position, size, shape, range, etc. of each configuration shown in the drawings and the like may not represent the actual position, size, shape, range, etc. in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the position, size, shape, range and the like disclosed in the drawings and the like.

The publications, patents and patent applications cited herein form part of the description herein.

The components represented in the singular form herein are intended to include the plural, unless explicitly stated in the context.

In the specific examples described below, incorrect data is entered by intentionally inputting incorrect data into AI in addition to the conventional first learning cycle that uses past data to reduce prediction errors. Provide a second learning cycle to learn not to be affected by. This not only learns the characteristics of the "signal" to react from past data, but also learns that it is not affected by meaningless "noise".

In a more preferred form, in order to be able to explain the basis of the results obtained from artificial intelligence, instead of the "majority vote" used in conventional deep learning, "sum, product, and denial" are used as basic elements in multiple layers. The prediction formula is constructed by the network structure.

This avoids the problem of low prediction accuracy for events with little data, which was inherent in conventional machine learning (including deep learning), and has a high level of prediction ability even with a small amount of data. , Can be exclusively disassembled and explained.

FIG. 1 is a conceptual diagram showing a specific example of the information processing system of the present invention. In this specific example, the original data (101) is input, and an accurate prediction model for predicting the teacher data (correct answer data) included in the original data is output. Here, the prediction model is specifically a predictor 1 (106) which is an algorithm for prediction and a prediction parameter (112) which is a parameter thereof.

As a specific example, consider the forecast of loan screening. The original data is the information of the lender (for example, the conditional data that defines the conditions such as gender, age, years of service, borrowing amount, annual income, etc. included in the loan application data such as a mortgage), and the teacher data is It is the data of the past performance (result) of whether the loan project has become bad debt, that is, the result data. Conditional data corresponds to explanatory variables, and result data corresponds to objective variables. With regard to various past lenders, M data of lenders (explanatory variable) and 1 teacher data (objective variable) indicating whether or not the loan has been lost are combined, and N data regarding the past performance of various lenders. Prepare the set. A loan is represented by a bundle (ie, a vector) of data consisting of M + 1 data. When this M + 1 dimensional vector data is collected for N cases, the original data becomes table data of N rows and M + 1 columns, or database or text data. This information processing system outputs a model (prediction formula and prediction parameter) that predicts whether or not the lender will be in bad debt as a result of the loan.

This information processing system will be described with an example of loan forecasting. First, the original data is preprocessed into a form that can be easily processed by a computer (102). For example, it is assumed that the data is classified into categories such as financial industry, manufacturing industry, and civil servant as the classification considering that the work classification is included. This is replaced with the numbers 1 and 0, which are 1 when the applicant is in the financial industry and 0 when the applicant is not. This is a numerical value that indicates that the place of employment is in the financial industry. The data classified by category can be converted into numerical information of 1 and 0 (multiple data columns are formed for each category) in this way.

The case where the original data is numerical data will be described. For example, if a numerical value of annual income is input, the value of annual income is classified into five stages. For example, if the category with the highest annual income is 100 million yen or more, it is set to 1 if the applicant's annual income is 100 million yen or more, and 0 if it is not. As a result, numerical information such as annual income can be converted into normalized information from 0 to 1. However, if this is done for all five classifications and converted to 1 and 0, the differences within the classification will be rounded. For example, focusing on the classification of 5 million yen to 10 million yen, applicants of 5.01 million yen and 9.99 million yen will be treated the same in the same category. To avoid this, do the following: For example, if the applicant's annual income is 5 million yen or less, it is set to 0, if it is 10 million yen or more, it is set to 1, and if it is 5 to 10 million yen, (annual income-5 million yen) ÷ 500. It is a continuous value (analog value) that changes from 0 to 1 with the formula of 10,000 yen. This allows for a normalized, continuously changing number from 0 to 1 depending on the annual income. This allows the information of the original continuous change to be normalized without rounding.

The learning data 1 (105) is extracted from the processed data (123) by the data extractor 1 (104). If there are N rows of processed data, this is trained in smaller units for training. Therefore, data is randomly extracted from the original data. For this purpose, random number generation 1 (103) is used. Random sampling can be performed by extracting the data rows corresponding to the generated random numbers. Such extraction rules can be set in advance by the user (operator) before learning.

There are two outputs of the data extractor 1. One is learning data 1 (105). This is an extract of the explanatory variables. The other is teacher data (107). This is the past performance (result) data corresponding to the learning data 1 (105), and in the case of a loan, whether or not the loan is bad is quantified by 1 and 0 (for example, the bad debt is "1". , If it is not a bad debt, set it to "0").

This learning data 1 (105) is input to the predictor 1 (106), and the probability of bad debt is predicted. The predictor 1 calculates a predicted value based on a prediction formula incorporating a prediction parameter (112). A specific example of the prediction formula will be described in detail later with reference to FIG. 2, but in any case, it is a formula incorporating prediction parameters. This prediction parameter is initially set to an appropriate initial value (for example, a random number generated by random number generation 3 (110) is used). Therefore, the prediction data 1 (108) of the first prediction result and the past teacher data (107) do not match at all. That is, the error is large. However, this prediction error can be calculated. In the learner 1 (109), this prediction error is calculated as follows.

Prediction error = (Numerical value of teacher data)-(Numerical value of predicted data)

Therefore, if each of the prediction parameters (112) included in the prediction formula is slightly changed (increased or decreased), this prediction error also changes. By gradually changing (increasing or decreasing) the prediction parameters so that the prediction error becomes smaller, the prediction error can be reduced and the accuracy of the prediction formula can be improved.

It is the learner 1 (109) that adjusts this prediction parameter (112). Specifically, by differentiating the prediction error with the prediction parameter and changing the prediction parameter (112) by a magnitude proportional to the differential coefficient, the prediction error can be efficiently lowered and the prediction accuracy can be improved. .. This proportionality coefficient is one of the specific examples of the learning parameter 1 (111). In this way, the learner 1 (109) adjusts the prediction parameter (112), so that the predictor 1 (106) → the prediction data 1 (108) → the learner 1 (109) → the prediction parameter (112) → the prediction. By executing the process on the learning cycle of the device 1 (106), the prediction accuracy can be improved to some extent. Such a learning cycle can be performed by conventional supervised machine learning techniques.

However, when the prediction target is an event that rarely occurs, such as a bad debt in a loan, there is a problem that sufficient prediction accuracy cannot be realized by this learning alone.

In general, in low-frequency events, overfitting occurs by adjusting the prediction parameters according to the events that happen to occur in a specific situation, and in new situations, the prediction accuracy is rather lowered, which is called "overfitting". The phenomenon is more likely to occur.

In this embodiment, a second learning cycle is provided in order to accurately predict such a rare event. This will be described below.

The data extractor 2 (114) extracts the learning data 2 (115) from the processed data (123). If there are N rows of processed data, this is trained in smaller units for training. Therefore, data is randomly extracted from the original data. For this purpose, random number generation 2 (103) is used. The learning data 2 (115) may be the same as the learning data 1 (105). At this time, in parallel, modified data (119) that is intentionally different from the teacher data (107) is automatically generated. As a method of creating modified data, 1 and 0 are intentionally mixed and assigned to the data group that was originally in bad debt (the case where the bad debt was originally 1), or to the data group that is not in bad debt. Similarly, 1 and 0 can be mixed and assigned. Random number generation 4 (122) can also be used to assign (wrong) data that is different from the original data. The rule for extracting the learning data 2 (115) can be set in advance by the user (operator). Further, the teacher data (that is, the modified data (119)) in the learning data 2 (115) does not use the data from the original data (101), and has a different label or numerical value as the objective variable with respect to the explanatory variable of the original data. It can be generated by giving as.

In the learning device 2 (120), supervised learning is performed in the same manner as in the learning device 1 (109), and the prediction parameter (112) is learned. However, the data to be a teacher is modified data (119). Then, after learning, the learner 2 (120) evaluates the magnitude of the reaction given to the prediction parameter by the modified data (119) (reactivity evaluation).

In this embodiment, the algorithm (predictor) does not have to be common between the predictor 1 (106) and the predictor 2 (116), but the features used for the prediction need to be common. .. As a result, the predictor 1 (106) and the predictor 2 (116) can make a correspondence between the feature quantities.

In the reactivity evaluation, for example, the error is calculated by comparing the modified data (119) which is not a teacher and the predicted data 2 (117) predicted by the predictor 2 (116). Then, the learning device 2 (120) calculates and evaluates the amount of change in the error between the modified data (119) and the prediction data 2 (117) with respect to the change in each prediction parameter of the predictor 2 (116). If the change in error is large with respect to the change in a certain prediction parameter, it can be said that the prediction parameter is a parameter that reacts sensitively to the modified data. As a simple method, the magnitude of the change in the error focuses on the magnitude of the proportional coefficient between the change in the error and the change in the parameter.

Further, in another method of reactivity evaluation, the correlation coefficient between the modified data (119) and the prediction data 2 (117) is calculated to quantify the similarity. Then, the change in the feature amount used in the prediction formula by the predictor 2 can be quantified by calculating the correlation coefficient between the prediction data 2 (117) and the modified data (119). If the change in the correlation coefficient between the two is large with respect to the change in a certain feature amount, it can be said that the feature amount is a parameter that reacts sensitively to the modified data. That is, this method focuses on the magnitude of the change in the correlation coefficient.

Therefore, the parameter of the predictor 1 (106) related to this sensitively reacting feature is brought close to 0. This is because this parameter is a parameter that reacts sensitively to erroneous information and noise contained in the data, and data bias that tends to occur due to the small amount of data. As a specific method, a weighting coefficient is assigned to each parameter, and a weighting coefficient smaller than that of other parameters is assigned to a parameter that reacts sensitively to the modified data. As a method of reducing the parameter, the parameter can be substantially reduced by giving a penalty that the error looks large as the parameter becomes large.

Specifically, the following is effective as an example of the data extraction method in the data extractor 2 (114). In the data extractor 2 (114), p cases with 1 teacher data are extracted from the training data 1 (105), and the teacher data is 1 from the processed data (123) that has not been learned yet. Add q cases of. This data set is extracted as learning data 2 (115). The teacher data in the learning data 2 is originally a data set consisting of only p + q 1s. Here, q 1's are inverted to 0. Therefore, modified data (119) consisting of p 1s and q 0s can be created. This is, of course, different from reality, but when this is learned, the changes in the predictive parameters that respond sensitively to it become large. Since such a parameter is a parameter that reacts sensitively to data bias and noise, it is possible to improve the prediction accuracy by making it close to 0. Specifically, each parameter may be weighted, and the weight of the sensitive predictive parameter may be smaller than that of the other parameters. The parameters such as p and q can be set in advance by the user (operator).

The learning cycle consisting of the predictor 2 (116) → the prediction data 2 (117) → the learner 2 (120) → the prediction parameter (112) → the predictor 2 (116) is extracted from the data extractor 2 (114). By learning about various cases, you can learn to be insensitive to data that should not react.

As described above, the learning cycle of predictor 1 (106) → prediction data 1 (108) → learner 1 (109) → prediction parameter (112) → predictor 1 (106) on the left side of FIG. 1 is data. It is a learning cycle that learns to react sensitively to the signs that should be reacted. On the other hand, the learning cycle of predictor 2 (116) → prediction data 2 (117) → learner 2 (120) → prediction parameter (112) → predictor 2 (116) on the right side becomes a sign that it should not react. It becomes a learning cycle to learn the insensitivity of.

By learning the prediction parameters by learning both of these learning cycles, it is possible to greatly improve the prediction accuracy of rare projects that occur infrequently. In this embodiment, both learning cycles are performed in synchronization. The learning cycle including the predictor 1 (106) on the left side of FIG. 1 can follow the learning method of the conventional deep neural network (DNN). On the other hand, the learning cycle including the predictor 2 (116) on the right side of FIG. 1 follows the conventional learning method of DNN, and based on the learning result, the learning including the predictor 1 (106) on the left side. Compensate for changes in predictive parameters based on the cycle.

The learning parameter 1 (111) and the learning parameter 2 (121) are set by the user (operator) for each of the learning device 1 (109) and the learning device 2 (120) before learning. It shall be left. Since the learning result (learning speed and prediction accuracy) changes when the learning parameter is changed, it is better to allow the user to change the learning parameter by referring to the learning result. Alternatively, the learning parameters may be automatically changed according to a predetermined rule, and preferable learning parameters may be automatically set based on the learning result when each learning parameter is used.

Further, the data extraction rule by the data extractor 1 (104) and the data extractor 2 (114) shall be set by the user (operator) before the learning. Since the learning result (learning speed and prediction accuracy) changes when the data extraction rule is changed, it is better to allow the user to change the data extraction rule by referring to the learning result. Alternatively, the data extraction rule may be automatically changed by a predetermined rule, and a preferable data extraction rule may be automatically set based on the learning result when each data extraction rule is used.

For the predictor 1 (106) and the predictor 2 (116) in FIG. 1, the conventionally used DNN can be applied. Each layer of a typical DNN usually performs a non-linear operation. However, when applying artificial intelligence technology (including machine learning) to decisions involving social responsibility in finance and industry, it is required that the basis of prediction be provided in a form that can be understood by humans. Has been done. However, in general deep learning, it is difficult to explain the rationale for why the prediction results are correct, and the black box is a barrier to application.

FIG. 2 shows examples of other configurations of predictor 1 (106) and predictor 2 (116) in FIG. The input layer (201) in FIG. 2 is a layer for inputting m vector data from x1 to xm. This data contains attribute information such as the annual income and gender of the loan applicant.

In this figure, the arrows indicate the flow of data. This input data is processed by the processing layer (211), and the predicted value is output to the output layer (212). The forecast data 1 (108) outputs the predicted value for each of various cases (in the case of a loan, a loan application project) included in the learning data 1 (105).

The processing layer (211) is composed of a single layer or a plurality of layers, and in the processing layer 1 (202), a product is calculated among a plurality of data in the input layer. Let this be p1, p2, ..., pk. In the figure
p1 = x1 x x2, p2 = x1 x x3. Here, x means an arithmetic product or a logical product. By this product processing, a compound index that "x1 is 1 and x2 is 1" is generated in p1, and more detailed conditions can be expressed. The same applies to p2 and later.

In the processing layer 2 (203), an important index is selected from a large number of combination indexes generated in the processing layer 1. In this example, p1, p3, and pk are selected, but p2 is not selected (arrows are not connected). To make this selection concretely, for example, the correlation is calculated among a large number of indexes generated in the processing layer 1, and the similarity between the indexes is quantified by the absolute value of the correlation. As a result, similar indexes are grouped together as a cluster, and the index having the highest correlation with the teacher data is selected for each cluster. As a result, similar indicators can be thinned out, and the indicators used can be made highly independent of each other. The more independent the indicators are, the more stable the forecast formula is.

The index selected by the processing layer 2 (referred to as q1, q2, ... qo) is the input of the processing layer 3 (204). In the processing layer 3, the weighted sum is calculated by combining these indexes. In particular,

Weighted sum = w1 x q1 + w2 x q2 + ...

And here w1, w2, ... are the weights of each index. A large weight value corresponds to an emphasis on that index. In FIG. 1, the arrows corresponding to q1, q2, ... Are broken lines, but this is to distinguish that they are weighted.

The output of the processing layer 3 is further an input of the processing layer 4 (205). In the processing layer 4, the weighted sum is input to the nonlinear function. As the nonlinear function, a sigmoid function or a ramp function that rises linearly above the 0 threshold value below the threshold value is used. This makes it possible to express a non-linear dependency. The weighted sum of the processing layer 3 and the nonlinear function of the processing layer 4 are collectively called a majority logic.

In this way, complex functions (prediction formulas) can be combined by combining the arithmetic processes (product, selection, weighted sum, nonlinear function) represented by ○ in FIG. 2, changing the order, and changing the network connection form. ) Can be expressed. Further, by changing the weight used for the weighted sum (204) and the selection criterion used for the selection layer (203) (for example, a predetermined correlation value when an index having a predetermined correlation or less is made into an independent cluster) as parameters. , It is possible to change the function flexibly. In FIG. 1, what is expressed as a prediction parameter (112) refers to these parameters such as weights and selection criteria.

In this figure, an example including four or more processing layers is shown, but in the simplest case, the index of the input layer can be output as it is. On the contrary, it is also possible to combine such various processing layers in multiple layers to create an extremely complicated prediction formula.

Here, by constructing the processing layer with a combination of only the product, selection, and weighted sum, and using the non-linear layer only for the output layer, the prediction formula can be expressed.

Y ＝ σ [Σw (Πxi)] σ [・] represents a nonlinear function (for example, a sigmoid function).
(For example, y ＝ w1 (x1) (x2) ＋ w2 (x2) (x3) (x8) (x9), in this case σ is an identity function)

Can be made into the form. In the above example, it can be seen that the result (output) is determined by "x1 and x2" and "x2 and x3 and x8 and x9". In this way, it is possible to always decompose the prediction results into the factors and explain the formula in human-understandable words. This is a feature not found in conventional deep learning and neural networks.

FIG. 3 shows a system configuration diagram of this embodiment. The hardware configuration of this embodiment can be configured by a general information processing device, for example, a server. The information processing device includes a processing device (301) and a storage device. The storage device includes, for example, a database (302), a program storage device (303), and an arithmetic storage device (304). Further, although not shown, an input device and an output device, which are general as an information processing device, are provided.

The processing device (301) executes various programs stored in the program storage device (303).

The database (302) is, for example, a magnetic disk device, and is a prediction parameter (112), processed data (123), original data (101), learning data 1 (105), learning data 2 (115), and a teacher. Data (107), modified data (119), learning parameter 1 (111), learning parameter 2 (121), and the like are stored.

The program storage device (303) includes a preprocessing unit (102), a random number generator (103, 113, 110, 122), a data extractor 1 (104), a data extractor 2 (114), a predictor 1 (106), and the like. It stores programs such as a predictor 2 (116), a learner 1 (109), and a learner 2 (120).

The arithmetic storage device (304) temporarily stores data read from a database (302) or a program storage device (303), or stores data when a processing device (301) performs an operation or the like. As the program storage device (303) and the arithmetic storage device (304), various known semiconductor memories can be used.

In this embodiment, the functions such as calculation and control are performed by the processing device (301) executing the program stored in the program storage device (303) in cooperation with other hardware. It will be realized. A program executed by a computer, its function, or a means for realizing the function may be referred to as a "function", a "means", a "part", a "vessel", a "module", or the like. Further, this configuration may be configured by a single computer, or any part of the input device, the output device, the processing device, and the storage device may be configured by another computer connected by a network. .. Further, in this embodiment, the same function as the function configured by using the program can be realized by hardware such as FPGA (Field Programmable Gate Array) and ASIC (Application Specific Integrated Circuit). Such aspects are also included in the scope of this example.

FIG. 4 is a block diagram showing details of the learner 2 (120). The learning device 2 (120) includes a learning unit (1201), a reactivity evaluation unit (1202), and a parameter correction unit (1203).

FIG. 5 is a flow chart of processing performed by the learner 2 (120). In the process S501, the learning unit (1201) performs the conventional supervised learning using the modified data (119) as the teacher data. However, as described above, the modified data (119) is, for example, data in which some of the processed data that was originally "with bad debt (1)" is changed to "no bad debt (0)". Alternatively, some of the processed data that was originally "no bad debt (0)" may be changed to "with bad debt (1)". As a result of learning in the learning unit (1201), the prediction parameters are calculated so that the error with the modified data (119) becomes small.

In the process S502, the reactivity evaluation unit (1202) evaluates the sensitivity of the reaction to the modified data (119) of each parameter (reactivity evaluation). Therefore, as described above, for example, the change in the prediction error with respect to the change in the prediction parameter is evaluated. Then, the prediction parameters sensitive to the modified data are extracted.

In the process S502, the parameter correction unit (1203) corrects sensitive parameters so as to be “insensitive”. One method for that is to give a smaller weight to the value of the parameter learned by the learner 1 (109) than the other parameters for the sensitive parameter extracted by S502. Alternatively, set the parameter to zero. To this end, the learner 2 (120) corrects the prediction parameter (112).

As another method, the sensitive parameters are learned by the learning device 1 (109) so that the prediction error becomes large, contrary to the usual learning. To this end, the learner 2 (120) modifies the learning algorithm for specific parameters of the learner 1 (109). By learning so that the prediction error becomes large, the influence of the modified data can be suppressed more strongly. The above is a specific example of "desensitizing" sensitive parameters, but these multiple methods may be combined.

In another example, in FIG. 1, by changing the original data, this same information processing system can also be used for forecasting for investment decisions. In this case, the original data is a numerical group (M pieces) representing the management information and financial information of the investee company and the situation of the target market. The teacher data is one piece of actual data of the return (for example, the amount of dividend obtained) obtained from the investee as a result of the investment. For N various investment destinations, this investment destination information and result return information are input, and a model of how much return can be obtained when investing in an unknown investment destination is output.

The original data is a data set of M + 1 columns and N rows, and this is input to the original data (101) in the form of a table, text, or database.

In addition to this, it can be used for forecasting inventory and shortages in the supply chain. In this case, information such as inventory and shortage status, delivery date, day of the week, weather, etc. is used as an explanatory variable, and the amount of resulting inventory and shortage (order backlog) is used as teacher data (objective variable) in a tabular format. You can enter the data of.

It can also be used to predict accidents in plants. In this case, sensor values such as temperature and pressure collected from the plant and employee characteristics (experience, etc.) are used as explanatory variables, and whether an accident has occurred as a result is used as teacher data.

Furthermore, it becomes possible to predict defects in the production line. Conditions such as manufacturing equipment operation information and temperature, as well as information such as environmental temperature and material supplier are used as explanatory variables, and the presence or absence of defects is input to the teacher data (objective variable).

It can also be used to predict new product hits. The attributes of the product so far (product category, color, characteristics of the name, price, etc.) and the launch time can be used as explanatory variables, and the sales after the sale can be used as teacher data (objective variable).

The present invention can be applied to a wide range of applications other than those listed here if data consisting of explanatory variables and teacher data can be prepared.

In the above-described embodiment, when a model formula for prediction is generated from data using machine learning, in rare events that occur infrequently, the parameters are adjusted according to the event that happened to occur in a specific situation. We focused on the fact that "over-learning" occurs, which leads to overfitting and lowers prediction accuracy. Then, in addition to the first learning that uses past data to reduce the prediction error, the second learning that learns not to be affected by the wrong data by intentionally inputting the wrong data into the AI. We are proposing a configuration with a cycle.

101 ... Original data 102 ... Preprocessing device 103 ... Random number generation 1
104 ... Data extractor 1
105 ... Learning data 1
106 ... Predictor 1
107 ... Teacher data 108 ... Prediction data 1
109 ・ ・ ・ Learner 1
110 ... Random number generation 3
111 ... Learning parameter 1
112 ... Prediction parameter 113 ... Random number generation 2
114 ... Data extractor 2
115 ... Learning data 2
116 ... Predictor 2
117 ... Forecast data 2
119 ... Modified data different from teacher data 120 ... Learner 2
121 ... Learning parameter 2
122 ... Random number generation 4
123 ・ ・ ・ Processed data

Claims (14)

In an information processing system that inputs original data and outputs prediction results
At least the first data and the second data are generated from the original data,
The first prediction formula for making a prediction using the first data has at least one parameter.
It has a first learner that adjusts the parameters using the first prediction result by the first prediction formula.
The second prediction formula for making a prediction using the second data has at least one parameter.
It has a second learner that adjusts the parameter using the second prediction result by the second prediction formula.
The parameters adjusted by the first learner and the parameters adjusted by the second learner have at least one common parameter.
An information processing system characterized in that the teacher data in the second data is data given a label or a numerical value different from the original data without using the data from the original data. In the information processing system of claim 1,
The first prediction formula is an information processing system characterized by including a weighted sum and a non-linear function. In the information processing system of claim 1,
The first prediction formula is an information processing system characterized by including a product and a weighted sum. In the information processing system of claim 1,
The second learning device includes a learning unit and a reactivity evaluation unit.
The learning unit adjusts a plurality of parameters including the common parameter.
The plurality of parameters are adjusted so that the error between the second data and the second prediction result is small.
The reactivity evaluation unit
An information processing system that extracts, among the plurality of parameters, a parameter in which the amount of change in the error is larger than a predetermined value with respect to the change in the parameter. In the information processing system of claim 1,
The second learning device includes a learning unit and a reactivity evaluation unit.
The learning unit adjusts a plurality of parameters including the common parameter.
The plurality of parameters are adjusted so that the error between the second data and the second prediction result is small.
The reactivity evaluation unit
An information processing system that extracts, among the plurality of parameters, a parameter in which the amount of change in the correlation coefficient between the second data and the second prediction result is larger than a predetermined value with respect to the change in the parameter. In the information processing system of claim 1,
The second learning device includes a learning unit, a reactivity evaluation unit, and a parameter correction unit.
The learning unit adjusts a plurality of parameters including the common parameter.
The plurality of parameters are adjusted so that the error between the second data and the second prediction result is small.
The reactivity evaluation unit
From the plurality of parameters, a parameter in which the error or the amount of change in the correlation coefficient between the second data and the second prediction result is larger than a predetermined value with respect to the change in the parameter is extracted.
The parameter correction unit
An information processing system that corrects the parameters adjusted by the first learner with respect to the extracted parameters. In the information processing system of claim 6 ,
The parameter correction unit
An information processing system that corrects the extracted parameters to reduce the weights of the parameters adjusted by the first learner. In the information processing system of claim 6 ,
The parameter correction unit
An information processing system that corrects the extracted parameters so that the parameters adjusted by the first learner approach zero. In the information processing system of claim 6 ,
The parameter correction unit
An information processing system in which the first learner adjusts the plurality of extracted parameters so that the error between the first data and the first prediction result becomes large. Prepare multiple teacher data consisting of a set of explanatory variables and the first result data,
Prepare a plurality of first training data consisting of a set of explanatory variables,
The first prediction data is obtained from the first learning data by using the prediction formula using the prediction parameters consisting of a plurality of parameters.
The prediction parameters are changed to obtain the first prediction parameters so that the error between the first result data and the first prediction data becomes small.
Prepare multiple modified data consisting of a set of explanatory variables and the second result data,
Prepare a plurality of second training data consisting of a set of explanatory variables,
The second prediction data is obtained from the second learning data by using the prediction formula using the prediction parameters.
The prediction parameter was changed to obtain the second prediction parameter so that the error between the second result data and the second prediction data would be small.
At least one of the change in the error with respect to the change in the second prediction parameter and the change in the correlation coefficient between the second result data and the second prediction data with respect to the change in the second prediction parameter. Evaluate, extract a predetermined parameter from the prediction parameters,
Among the first prediction parameters, the first prediction parameter is adjusted with respect to the extracted parameter corresponding to the predetermined parameter.
The teacher data is a part of the original data, and the modified data is data obtained by modifying the original data and is different from the original data.
Information processing system learning method. In the learning method of the information processing system of claim 10 ,
Of the first prediction parameters, the parameters corresponding to the predetermined parameters are corrected to reduce the weight of the first prediction parameters.
Information processing system learning method. In the learning method of the information processing system of claim 10 ,
By changing the prediction parameter so that the error between the first result data and the first prediction data becomes large for the parameter corresponding to the predetermined parameter among the first prediction parameters. Correct the first prediction parameter,
Information processing system learning method. In the learning method of the information processing system of claim 10 ,
The teacher data is a part of the original data, and the modified data is data obtained by modifying the original data and is different from the original data.
The method of modifying the original data can be changed.
Information processing system learning method. Prepare multiple teacher data consisting of a set of explanatory variables and the first result data,
Prepare a plurality of first training data consisting of a set of explanatory variables,
The first prediction data is obtained from the first learning data by using the prediction formula using the prediction parameters consisting of a plurality of parameters.
The prediction parameters are changed to obtain the first prediction parameters so that the error between the first result data and the first prediction data becomes small.
Prepare multiple modified data consisting of a set of explanatory variables and the second result data,
Prepare a plurality of second training data consisting of a set of explanatory variables,
The second prediction data is obtained from the second learning data by using the prediction formula using the prediction parameters.
The prediction parameter was changed to obtain the second prediction parameter so that the error between the second result data and the second prediction data would be small.
At least one of the change in the error with respect to the change in the second prediction parameter and the change in the correlation coefficient between the second result data and the second prediction data with respect to the change in the second prediction parameter. Evaluate, extract a predetermined parameter from the prediction parameters,
Among the first prediction parameters, the first prediction parameter is adjusted with respect to the extracted parameter corresponding to the predetermined parameter.
The teacher data is a part of the original data, and the modified data is data obtained by modifying the original data and is different from the original data.
The method of modifying the original data can be changed.
Information processing system learning method.

Images