JP2021149590A - Leaning device, learning method, and learning program - Google Patents

Abstract

To allow for appropriately selecting multiple sparse estimation methods suitable for data and setting the degree of application of each of the multiple sparse estimation methods for a model.SOLUTION: An information processing device 10 provided herein comprises an acquisition unit 12a configured to acquire multiple sets of data to which sparse estimation methods can be applied, and a learning unit 122c configured to train a model configured to generate a given output by applying multiple sparse estimation methods to the multiple sets of data used as inputs on the degree of application of each of the multiple sparse estimation methods.SELECTED DRAWING: Figure 1

Description

The present invention relates to a learning device, a learning method and a learning program.

Conventionally, various methods have been proposed to estimate causal relationships between data based on sensor data collected from factories, plants, etc., and to use them to solve problems such as prediction and abnormality / change point detection. ing.

For example, as an orthodox method, the observed multiple sensor data is used to predict how much a specific time delay of a specific sensor affects the behavior of a desired sensor (whether there is a causal relationship). There are methods for building learning models.

Furthermore, a statistical method (for example, LASSO) in which minute causal relationships (or no actual causal relationships) are removed by applying a sparse estimation method (a method that can estimate a non-essential estimated value to 0) is widely known. Has been done. Although noise is not a little included in actual sensor data, improvement of noise robustness can be expected by using such sparse estimation.

In addition, by adjusting the degree of sparsity, it is possible to statistically determine criteria such as the effective / ineffective threshold value previously determined by the analyst.

A.E. Hoerl and R.W. Kennard, “Ridge Regression: Biased Estimation for Nonorthogonal Problems”, Technologies, vol. 12, no. 1, pp. 55-67, 1970. R. Tibshirani, “Regression shrinkage and selection via the lasso”, Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 58, no. 1, pp. 267-288, 1996. H. Zou and T. Hastie, “Regularization and variable selection via the elastic net”, Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 67, no. 2, pp. 301-320, 2005. H. Zou, “The Adaptive Lasso and Its Oracle Properties”, J. Am. Stat. Assoc., Vol. 101, no. 476, pp. 1418-1429, 2006. H. Zou, T. Hastie, and R. Tibshirani, “Sparse Principal Component Analysis”, Journal of Computational and Graphical Statistics, vol. 15, no. 2, pp. 265-286, 2006. R. Tibshirani, M. Saunders, S. Rosset, J. Zhu, and K. Knight, “Sparsity and smoothness via the fused lasso”, Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 67, no . 1, pp. 91-108, 2005.

LASSO, Elastic Net, Adaptive LASSO, Group LASSO, Fused LASSO, etc., which have different characteristics, have been proposed as sparse estimation methods. The analyst needs to select the sparse estimation method to be used individually according to the problem to be solved from these sparse estimation methods, and adjust the degree of sparseness with respect to the selected sparse estimation method.

However, it takes a lot of statistical learning costs for the analyst to properly select the sparse estimation method and master the sparse estimation method. Moreover, even if the analyst gains an understanding of statistics, a great deal of time and effort is required for trial and error such as selection of a sparse estimation method and tuning of the degree of sparseness.

Furthermore, in the past, each sparse estimation method is often used alone, and there are few examples of a sparse estimation method having a hybrid property that integrates a plurality of them.

The present invention has been made in view of the above, and learning that can appropriately select a plurality of sparse estimation methods suitable for data and set the degree of application of each of the plurality of sparse estimation methods for a model. It is an object of the present invention to provide a device, a learning method and a learning program.

In order to solve the above-mentioned problems and achieve the object, the learning device of the present invention uses an acquisition unit that acquires a plurality of data to which the sparse estimation method can be applied, and a plurality of data as inputs, and a plurality of sparse estimations. It is characterized by having a learning unit for learning each degree of application of a plurality of sparse estimation methods to a model that applies a method and performs a predetermined output.

Further, the learning method of the present invention relates to a process of acquiring a plurality of data to which a sparse estimation method can be applied, and a model in which a plurality of data are input and a plurality of sparse estimation methods are applied to perform a predetermined output. It is characterized by including a step of learning each degree of application of a plurality of sparse estimation methods.

Further, the learning program of the present invention relates to a step of acquiring a plurality of data to which a sparse estimation method can be applied, and a model in which a plurality of data are input and a plurality of sparse estimation methods are applied to perform a predetermined output. It is characterized by a step of learning each degree of application of a plurality of sparse estimation methods and a computer executing the sparse estimation method.

According to the present invention, it is possible to appropriately select a plurality of sparse estimation methods suitable for data and set the degree of application of each of the plurality of sparse estimation methods for the model.

FIG. 1 is a block diagram showing a configuration example of the learning device according to the first embodiment. FIG. 2 is a diagram illustrating an outline of prediction processing executed by the information processing apparatus shown in FIG. FIG. 3 is a diagram schematically showing an example of a plurality of sparse estimation methods applied to a model. FIG. 4 is a diagram showing a relational expression when the six sparse estimation methods illustrated in FIG. 3 are integrated and applied to a model. FIG. 5 is a diagram illustrating a method of searching for a set value of the degree of sparseness of the sparse estimation method. FIG. 6 is a diagram showing an example of a sparse estimated solution acquired in the first learning process. FIG. 7 is a flowchart showing a processing procedure of the learning process according to the embodiment. FIG. 8 is a diagram illustrating a flow of construction for a sparse learning model according to a conventional method and an embodiment. FIG. 9 is a diagram showing a computer that executes a program.

Hereinafter, the learning device, the learning method, and the embodiment of the learning program according to the present application will be described in detail with reference to the drawings. The learning device, learning method, and learning program according to the present application are not limited by this embodiment.

[Embodiment]
In the following embodiment, the configuration of the information processing device 10 and the processing flow of the information processing device 10 according to the embodiment will be described in order, and finally, the effect of the embodiment will be described.

[Configuration of information processing device]
First, the configuration of the information processing apparatus 10 will be described with reference to FIG. FIG. 1 is a block diagram showing a configuration example of an information processing device according to an embodiment. The information processing device 10 is a model to which a plurality of sparse estimation methods are applied, and constructs a model for accurately explaining a target data from a plurality of data. The information processing device 10 searches for a plurality of sparse estimation methods suitable for the data and each degree of application (degree of sparseness) of the plurality of sparse estimation methods by training the model of a plurality of data, and the plurality of sparses. Apply the estimates to the model appropriately.

The information processing device 10 collects a plurality of data acquired by sensors installed in monitored equipment such as a factory or a plant, and predicts the behavior of the target data by inputting the collected data. The predicted value of the behavior of the target data is output as an output value using the trained model for. The data used at this time does not necessarily have to be serial data (for example, time series data).

Here, it is a figure explaining the outline of the prediction processing executed by the information processing apparatus 10 with reference to FIG. FIG. 2 is a diagram illustrating an outline of prediction processing executed by the information processing apparatus 10 shown in FIG.

In FIG. 2A, it is shown that a sensor, a device for collecting operation signals, and the like are attached to a reactor, a device, or the like in the plant, and data is collected at regular intervals. Then, in FIG. 2B, the process data is plotted for each item (process), and the information processing apparatus 10 cuts out the data of the analysis window (shaded portion). This assumes that anomaly detection and change point detection are performed using the degree to which the causal relationship based on the data in the window changes when the window is slid over time. Since this is a preprocessing according to the purpose of the analyst, it is not always necessary to use the analysis window.

The information processing apparatus 10 calculates the predicted value of the target data to be predicted by using the model based on the numerical value extracted from the process data (see (C) of FIG. 2). FIG. 2D shows a plot of predicted values of target data calculated by the information processing apparatus 10 at regular intervals. For example, when introducing it to a plant, etc., this plot can be used to predict future behavior, or anomaly detection that generates an alarm when the difference between the predicted value and the measured value deviates from the preset range. It is expected to be used for change point detection.

As shown in FIG. 1, the information processing device 10 includes a communication processing unit 11, a control unit 12, and a storage unit 13. The processing of each part of the information processing apparatus 10 will be described below.

The communication processing unit 11 controls communication regarding various information exchanged with the connected device. Further, the storage unit 13 stores data and programs necessary for various processes by the control unit 12, and has a data storage unit 13a, a setting information storage unit 13b, and a model storage unit 13c. For example, the storage unit 13 is a storage device such as a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory).

The data storage unit 13a stores the data acquired by the acquisition unit 12a. For example, the data storage unit 13a stores at least the latest process data for a frame having a predetermined time width as data.

The setting information storage unit 13b stores the setting information required when searching for a plurality of sparse estimation methods applied to the model. For example, the initial setting information among the setting information is set by the analyst. Specifically, the setting information is information for setting the search range of a plurality of sparse estimation methods to be applied to the model. The setting information includes the range of the sparse estimation method to be searched (a plurality of sparse estimation methods to be searched), the range of the degree of sparseness of each sparse estimation method, the number of searches, the end criterion, and the like. For example, as the setting information, the range of the sparse estimation method and the range of the degree of each sparseness may be set based on the prior knowledge and know-how of the analyst. Further, the setting information may be information indicating a range outside the search target (for example, a sparse estimation method excluded from the search target).

The model storage unit 13c stores the parameters of the model optimized by the learning unit 122c (described later). The parameters of the model include a plurality of sparse estimation methods applied to the model and the degree of sparseness of each sparse estimation method.

The control unit 12 has an internal memory for storing a program that defines various processing procedures and the like and required data, and executes various processing by these. For example, the control unit 12 has an acquisition unit 12a, a preprocessing unit 12b, and an estimation unit 12c. Here, the control unit 12 is, for example, an electronic circuit such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphical Processing Unit), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array). It is an integrated circuit such as.

The acquisition unit 12a acquires a plurality of data related to the processing target, which is a plurality of data to which the sparse estimation method can be applied. For example, the acquisition unit 12a periodically (for example, every minute) receives numerical data of a multivariate time series from a sensor installed in a monitored facility such as a factory or a plant, and stores it in the data storage unit 13a. Here, the data acquired by the sensor is, for example, various process data such as temperature, pressure, sound, and vibration of the equipment to be monitored, the equipment in the plant, and the reactor. The data acquired by the acquisition unit 12a is not limited to the data acquired by the sensor, and may be, for example, numerical data manually input.

The pre-processing unit 12b performs a predetermined pre-processing on the time-series data acquired by the acquisition unit 12a to process the time-series data. For example, the preprocessing unit 12b may cut out the process data having a predetermined width and calculate a representative value (average value) for each sensor for each value included in the process data having the cut out width. The processing of the preprocessing unit 12b may be omitted.

The estimation unit 12c takes the time series data acquired by the acquisition unit 12a as an input, and outputs a predetermined output value using a trained model for predicting the state of the monitored equipment. For example, when the time-series data is processed by the pre-processing unit 12b, the estimation unit 12c inputs the processed time-series data into the trained model, and sets a preset fixed time later for the monitored equipment. Predict the state.

Here, the estimation unit 12c receives, for example, the time series data of each sensor acquired by the acquisition unit 12a as an input, and the plurality of sparse estimation methods and the sparses of each sparse estimation method applied by the learning unit 122c described later. Predict the condition of the monitored equipment using a trained model with adjusted sexuality. The trained model used by the estimation unit 12c may be any model, and the prediction method may be any method.

The configuration of the estimation unit 12c will be described. The estimation unit 12c has a setting information receiving unit 121c, a learning unit 122c, and an output unit 123c.

The setting information receiving unit 121c accepts input of setting information required when searching for a plurality of sparse estimation methods applied to the model.

The learning unit 122c makes a model that takes a plurality of data as an input and applies a plurality of sparse estimation methods to perform a predetermined output to learn each sparseness degree of the plurality of sparse estimation methods. The learning unit 122c has a second learning process for optimizing the degree of sparseness of each of the plurality of sparse estimation methods applied to the model, and each sparseness of the plurality of sparse estimation methods calculated in the second learning process. The first learning process of optimizing the parameters of the model is repeatedly executed with the degree set.

The output unit 123c predicts the state of the monitored equipment by using a model (learned model) in which the learning unit 122c has optimized the parameters. The output unit 123c visualizes the future predicted value of the desired target data, the abnormality detection based on the future predicted value, the change point detection, the factor relationship, and the like, and outputs the result.

[Example of sparse estimation method]
Next, an example of a multiple sparse estimation method applied to the model will be described. FIG. 3 is a diagram schematically showing an example of a plurality of sparse estimation methods applied to a model. “Parameter” shown in FIG. 3 is a parameter of the model and is an estimation target of the model. The sparse estimation method applied to the model in the present embodiment is a method of determining unnecessary parameters of the model as “0”.

“RIDGE” shown in FIG. 3 is a method of reducing and estimating parameters as a whole. Strictly speaking, "RIDGE" is not a sparse estimation method, but in the present embodiment, "RIDGE" is also searched for application to the model.

And "LASSO" is a method of sparsely estimating a non-essential parameter as 0. "Elastic Net = (1-t) x LASSO + t x RIDGE" is a method that mixes "RIDGE" and "LASSO" to mitigate the disadvantages of each and emphasize the advantages.

"Adaptive LASSO" is an extension of "LASSO" and is a method that theoretically guarantees the correctness of the estimated solution under certain statistical assumptions. "Adaptive LASSO" is a method that theoretically guarantees the correct answer if statistical assumptions are met. "Group LASSO" is a method of setting a group structure for parameters and obtaining a sparse estimation solution along the group structure.

"Fused LASSO" is a sparse estimation method in which adjacent parameters along a series become equal when a series structure is inherent in the data. That is, "Fused LASSO" is a method of performing sparse estimation so that the difference between adjacent parameters is "0". The series structure is a structure in which adjacent parameters such as the time direction of time series data and adjacent pixels of image data should be continuously changed. Then, the normal "Fused LASSO" considers up to the first-order difference, but in the present embodiment, this is extended and incorporated up to an arbitrary N (> 1)-order difference.

In the present embodiment, a plurality of applied sparse estimation methods and the degree of sparseness in each sparse estimation method are searched, and an optimum sparse learning model is automatically constructed.

[Processing of learning department]
The learning unit 122c searches for a plurality of sparse estimation methods suitable for the data and each sparseness degree of the plurality of sparse estimation methods by training the model with a plurality of data, and appropriately performs the plurality of sparse estimation methods. Apply to model. That is, a plurality of sparse estimation methods are virtually integrated and applied to the model.

FIG. 4 is a diagram showing a relational expression when the six sparse estimation methods illustrated in FIG. 3 are integrated and applied to a model. In the relational expression of FIG. 4, the first term is an error function, the second term is a term indicating LASSO, the third term is a term indicating RIDGE, and the fourth term is a term indicating Group LASSO. The fifth term is a term indicating Fused LASSO up to the Nth-order difference. _{The terms α L1} , α _L2 , α _g , and α _Dd in the second to fifth terms are hyperparameters indicating the strength of sparsity (degree of sparsity) of LASSO, RIDGE, Group LASSO, and Fused LASSO, respectively. _{Further, γL 1} , γL ₂ , γ _g , and γ _Dd in the second to fifth terms are hyperparameters indicating the strength of Adaptive LASSO. The values of α _L1 , α _L2 , α _g , and α _Dd allow 0, and if there is a sparse estimation method that is not used, the hyperparameter corresponding to this sparse estimation method is fixed to 0.

The learning unit 122c is sparse for the data by optimizing each sparse estimation method applied to the model, αL ₁ , αL ₂ , αL ₃ , αL ₄ , γL ₁ , γL ₂ , γL ₃ , γL _4. It makes it possible to obtain an estimated solution appropriately. The learning unit 122c optimizes the hyperparameters of each sparse estimation method applied to the model by executing the following double iterative processing of the first learning and the second learning. Therefore, the first learning process and the second learning process executed by the learning unit 122c will be described.

First, the first learning process optimizes the parameters of the model in a state where each degree of sparseness of the plurality of sparse estimation methods is set. In the model, each degree of sparseness of the plurality of sparse estimation methods calculated in the second learning process described later is set. Each degree of sparsity of these plurality of sparsity estimation methods is fixed until the first learning process is completed. In the first learning process, the initial setting information set in the setting information storage unit 13b is applied.

The first learning process is a process in which a model learns the relationship between a plurality of input data (for example, time series data) and a predetermined output (observed value to be monitored after a certain period of time) for the plurality of data. .. For example, in the first learning process, sparse estimation is performed using a model, and the error between the actual observed value output from the sensor and the predicted value output by the model for the input of a plurality of data is evaluated. Update the model parameters. The first learning process optimizes the parameters of the model by repeatedly evaluating the error between the observed value and the predicted value and updating the parameters until a predetermined first termination criterion is satisfied. The first termination criterion is, for example, that the number of repetitions of the first learning process reaches a predetermined number of times, the improvement of the error between the observed value and the predicted value is less than the predetermined value, and the like.

Then, in the second learning process, each sparseness degree of the plurality of sparse estimation methods applied to the model is optimized. For example, in the second learning process, the accuracy of each sparseness degree of the plurality of sparse estimation methods is obtained based on a series of learning results by the learning unit 122c, and the plurality of sparse estimation methods applied to the model based on each accuracy. Set each degree of sparsity. The accuracy is an index showing the probability that a good evaluation (for example, a result with high prediction accuracy) can be obtained by applying this sparsity degree to the model. For example, it is conceivable to use the value of the acquisition function. The learning unit 122c executes the next first learning process in a state where each degree of sparseness of the plurality of sparse estimation methods set in the second learning process is fixed.

Further, the learning unit 122c sets the degree of sparseness of the plurality of sparse estimation methods to be applied to the model based on each accuracy, and then performs the first learning process when the predetermined second end criterion is satisfied. The parameter information of the optimized model is input to the output unit 123c without being executed. The parameter information of the model includes each degree of sparseness of the sparse estimation method applied to the model. The second end criterion may be, for example, that the number of repetitions (the number of searches) of the second learning process has reached a predetermined number, and that the error between the observed value and the predicted value is less than the predetermined value.

FIG. 5 is a diagram illustrating a method of searching for a set value of the degree of sparseness of the sparse estimation method. In FIG. 5A-13, the vertical axis indicates the error between the observed value and the predicted value obtained in the first learning process, and the horizontal axis indicates the degree of sparsity. Then, in FIG. 5 (b-13), the vertical axis indicates the accuracy and the horizontal axis indicates the degree of sparsity. (A-13) of FIG. 5 and (b-13) of FIG. 5 correspond to the degree of sparsity of LASSO in the 13th second learning process.

In FIG. 5, each sparse estimation method is represented by one plan view, and the same processing is performed in the dimension space of the entire set value of each sparse estimation method. This is represented by the depth of the plan view in FIG. That is, as shown in the graph groups G13a and G13b of FIG. 5, behind the graph showing the error or accuracy regarding the sparsity degree of LASSO in the 13th first learning process, the 13th first learning process is performed. Graphs showing the error or accuracy of the degree of sparseness of other sparse estimation methods overlap. The first to twelfth graphs are continuous on the left side of the page along the axis of "efficient search".

In the 13th second learning process, the learning unit 122c has an error and a confidence interval between the observed value and the predicted value in the 13th first learning process (see (1) in (a-13) of FIG. 5). The final evaluation result for is used as the result of this search, and the degree of sparseness is statistically possible, including the results of the previous search (results in the first and second learning processes from the first to the twelfth). A function indicating the sex (for example, see (b-13) in FIG. 5) is calculated. As shown in FIG. 5 (b-13), this function is a function showing the relationship between the accuracy and the sparsity degree. The learning unit 122c finds the most probable (maximum function) accuracy from this function. Then, the learning unit 122c calculates the set value in the next search (the set value of the degree of sparseness of the sparse estimation method) based on this accuracy, applies it to the model, and executes the first learning process.

For example, as shown in FIG. 5 (b-13), the accuracy of the LASSO degree of sparsity is maximized at the point P13, so that the learning unit 122c determines the degree of sparsity corresponding to the accuracy shown at the point P13. , Set as the degree of sparsity of LASSO in the next 14th search (see (2) in (b-13) and arrow Y13 in FIG. 5). Similarly, for the other sparsity estimation methods, the learning unit 122c sets the sparsity degree corresponding to the most probable accuracy as the sparsity degree in the next 14th search. Then, the learning unit 122c applies the sparseness degree of each set sparse estimation method to the model, and executes the first learning process.

Similarly, in the 14th second learning process, the learning unit 122c has a final evaluation result in the first learning process obtained in the 14th first learning process ((a-14) in FIG. 5). (See) and the accuracy of each sparseness degree of the sparse estimation method is obtained based on the learning results from the first to the thirteenth times. Then, the learning unit 122c calculates the set value in the next 15th search based on this accuracy, applies it to the model, and executes the 15th first learning process. For example, for LASSO, the degree of sparsity according to the accuracy of point P14 in FIG. 5 (b-14) is set as the degree of sparsity of LASSO in the next 15th first learning process.

By repeatedly executing the first learning process and the second learning process, the learning unit 122c may be considered to be the best combination among the combinations of each sparseness degree of the plurality of sparse estimation methods. To find a combination of high sparseness.

FIG. 6 is a diagram showing an example of a sparse estimated solution acquired in the first learning process. In the first learning process, the parameters of the model are optimized based on the combination of each set sparsity degree. As a result, as shown in FIG. 5, white color (= 0) is localized. It can be seen that in this embodiment, a sparse estimation solution is obtained by removing non-essential parameters.

[Processing procedure of learning process]
Next, the learning process of the model executed by the information processing apparatus 10 will be described. FIG. 7 is a flowchart showing a processing procedure of the learning process according to the embodiment.

As shown in FIG. 7, the information processing apparatus 10 receives the input of the setting information required when the setting information receiving unit 121c searches for a plurality of sparse estimation methods applied to the model (step S1). The learning unit 122c sets the degree of sparseness of each sparse estimation method for the model (step S2). In the case of the first learning process, the learning unit 122c sets the degree of sparseness of each sparse estimation method for the model according to the initial setting information set in the setting information storage unit 13b. Further, in the case of the second and subsequent learning processes, the learning unit 122c sets each sparseness degree of the plurality of sparse estimation methods calculated in the immediately preceding second learning process in the model.

Subsequently, the learning unit 122c executes the first learning process for optimizing the parameters of the model in a state where each sparseness degree of the plurality of sparse estimation methods is set. For example, the learning unit 122c performs sparse estimation using the model, and evaluates the error between the actual observed value output from the sensor and the predicted value output by the model with respect to the input of a plurality of data. The model parameters are updated (step S3). The learning unit 122c determines whether or not the first termination criterion is satisfied (step S4), and if the first termination criterion is not satisfied (step S4: No), observes until the first termination criterion is satisfied. The error between the value and the predicted value is evaluated and the parameter is updated repeatedly.

When the first termination criterion is satisfied (step S4: Yes), the learning unit 122c performs a second learning process for optimizing each sparseness degree of the plurality of sparse estimation methods applied to the model. Based on a series of learning results by the learning unit 122c, the accuracy of each sparseness degree of the plurality of sparse estimation methods is obtained (step S5). Then, the learning unit 122c determines whether or not the second end criterion is satisfied (step S6).

If the second termination criterion is not met (step S6: No), the process proceeds to step S2, and the plurality of sparse estimation methods applied to the model are based on the respective accuracy in the plurality of sparse estimation methods obtained in step S5. Each sparsity degree is set (step S2). Then, the first learning process is executed.

On the other hand, when the second termination criterion is satisfied (step S6: Yes), the process is terminated by applying a parameter including each sparsity degree of the optimized plurality of sparsity estimation methods to the model. The output unit 123c predicts the state of the monitored equipment using the trained model, and outputs the result.

[Effect of Embodiment]
FIG. 8 is a diagram illustrating a flow of construction for a sparse learning model according to a conventional method and an embodiment.

As shown in FIG. 8, in the conventional method, learning is performed by selecting an individual or a small number of combinations from various sparse estimation methods. At this time, in the conventional method, it is necessary for the analyst to decide which sparse estimation method to select according to the data type and the problem setting, and the analyst further individually selects the selected sparse estimation method. It is necessary to set the degree of sparsity.

Here, if the degree of sparsity is increased, a more sparse estimated solution can be obtained, but if it is made extremely large, for example, in LASSO, all parameters are estimated to be 0, which is inconvenient. On the contrary, if the degree of sparseness is made extremely small, a sparse estimation solution cannot be obtained. Therefore, the analyst needs to tune the degree of sparsity to an appropriate value, and the analyst has to make trial and error before tuning (see (1) in FIG. 8 and frame W1). As described above, with the conventional method, the influence of prior knowledge and know-how to be personalized to the analyst is large, and it is difficult to effectively overcome trial and error and obtain a highly accurate model.

On the other hand, in the present embodiment, a model in which a plurality of data are input and a plurality of sparse estimation methods are applied to perform a predetermined output is made to learn each degree of sparseness of the plurality of sparse estimation methods. In the present embodiment, the second learning process for optimizing each sparseness degree of the plurality of sparse estimation methods applied to the model, and each sparse of the plurality of sparse estimation methods calculated in the second learning process. The first learning process of optimizing the parameters of the model is repeatedly executed with the degree of sex set. Thereby, in the present embodiment, it is possible to automatically select a plurality of sparse estimation methods and tune the degree of sparseness of each sparse estimation method (see (2) in FIG. 8 and frame W2).

That is, in the present embodiment, for example, a learning method is realized in which all the sparse estimation methods having different properties shown in FIG. 3 are included, and the degree of sparseness of each sparse estimation method can be automatically adjusted. .. As a result, in the present embodiment, from many sparse estimation methods having individual characteristic properties, a hybrid sparse estimation method appropriately adapted to the data, including the degree of individual sparseness, is multifaceted. It automatically tunes to the optimum state from the viewpoint, and the process that has been acquired by trial and error can be effectively acquired in a lump. Such tuning is generally a combination problem and is often difficult in real time, but in the present embodiment, a statistical method can be used to efficiently perform a search in real time. (See FIG. 5).

Then, in the present embodiment, even if the analyst does not have a background in statistics, a series of processes are automatically performed, so that it is possible to build a model at the same level as an analyst with a background. be. Further, in the present embodiment, it is possible to reflect the knowledge of the analyst in the construction of the model by specifying the degree of sparseness of the specific sparse estimation method in advance.

Further, in the present embodiment, since a plurality of sparse estimation methods are included, the range of applicable data becomes very wide. Further, in the present embodiment, since a plurality of sparse estimation methods have an inclusive relationship, by limiting the sparse estimation methods to be learned in the setting information, only a specific sparse estimation method among the plurality of sparse estimation methods can be used. It is also possible to set the learning of the model only to. For example, in the present embodiment, by setting the degree of sparseness of the sparse estimation method that is not desired to be used to 0, it is possible to construct a model limited to only a part of the sparse estimation methods.

As described above, according to the present embodiment, not only a highly accurate model and an estimated solution can be obtained as compared with the conventional method, but also unnecessary non-essential factor relations are reduced to 0 due to the utility of sparse estimation. You can get a sparse estimated solution that you dropped. Further, according to the present embodiment, even if the analyst has no statistical background, an appropriate model can be automatically constructed based on various sparse estimation methods, and the analyst has prior know-how. Can reflect that know-how in part or in whole, and automatically tune only the remaining unclear parts.

Further, in the present embodiment, it can be applied uniformly regardless of the data, and the process of trial and error, which has been conventionally performed, can be automated. As a result, in the present embodiment, not only can the optimum model be constructed more efficiently than the analyst, but also a model that is difficult for the analyst to come up with may be acquired. Then, according to the embodiment, it is effective to automatically estimate the combinatorial optimization problem of the degree of sparseness of the sparse estimation method within the search range by using the statistical method. Efficient search can be performed.

As described above, according to the present embodiment, the optimum sparse estimation learning model is automatically constructed in the range based on the prior knowledge and know-how of the analyst, or even when there is no prior knowledge or know-how. Therefore, the best model can be constructed more efficiently than the analyst does manually. Therefore, according to the present embodiment, it is possible to appropriately select a plurality of sparse estimation methods suitable for the data and set the degree of application of each of the plurality of sparse estimation methods for the model, and remove non-essential parameters. You can build a more accurate and sparse model that you dropped. As a result, it is possible to clearly know the model that appropriately describes the analysis target, and it is expected that the accuracy of various machine learning methods such as future prediction and abnormality detection will be improved.

Further, in the present embodiment, the model is highly expandable, and it is easy to incorporate a new sparse estimation method other than the sparse estimation method shown in FIG. 3 into the model. Further, once incorporated into the model, by executing the learning method according to the present embodiment, it is possible to automatically construct a sparse model in which the optimum degree of sparseness including the sparse estimation method incorporated later is set. That is, it can be said that the learning method according to the present embodiment is a unified model construction method for various various sparse estimation methods in general.

The learning method of this embodiment automatically constructs an optimum sparse estimation learning model in a range based on the prior knowledge and know-how of the analyst, or even when there is no prior knowledge or know-how. The best model can be constructed more efficiently than the analyst can do manually. Thereby, according to the present embodiment, it is possible to construct a more accurate and sparse model in which non-essential parameters are removed. Then, according to the present embodiment, it is possible to clearly know the learning model that appropriately describes the analysis target, and it is expected that the accuracy of various machine learning methods such as future prediction and abnormality detection will be improved.

[System configuration, etc.]
Further, each component of each of the illustrated devices is a functional concept, and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or part of the device is functionally or physically dispersed / physically distributed in any unit according to various loads and usage conditions. Can be integrated and configured. Further, each processing function performed by each device is realized by a CPU or GPU and a program that is analyzed and executed by the CPU or GPU, or as hardware by wired logic. It can be realized.

Further, among the processes described in the present embodiment, all or part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed can be performed. All or part of it can be done automatically by a known method. In addition, the processing procedure, control procedure, specific name, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified.

[program]
It is also possible to create a program in which the processing executed by the information processing apparatus described in the above embodiment is described in a language that can be executed by a computer. For example, it is possible to create a program in which the processing executed by the information processing apparatus 10 according to the embodiment is described in a language that can be executed by a computer. In this case, the same effect as that of the above embodiment can be obtained by executing the program by the computer. Further, the same processing as that of the above-described embodiment may be realized by recording the program on a computer-readable recording medium, reading the program recorded on the recording medium into the computer, and executing the program.

FIG. 9 is a diagram showing a computer that executes a program. As illustrated in FIG. 9, the computer 1000 has, for example, a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. However, each of these parts is connected by a bus 1080.

The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012, as illustrated in FIG. The ROM 1011 stores, for example, a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090, as illustrated in FIG. The disk drive interface 1040 is connected to the disk drive 1100 as illustrated in FIG. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120, as illustrated in FIG. The video adapter 1060 is connected, for example, to a display 1130, as illustrated in FIG.

Here, as illustrated in FIG. 9, the hard disk drive 1090 stores, for example, the OS 1091, the application program 1092, the program module 1093, and the program data 1094. That is, the above-mentioned program is stored in, for example, the hard disk drive 1090 as a program module in which a command executed by the computer 1000 is described.

Further, the various data described in the above embodiment are stored as program data in, for example, a memory 1010 or a hard disk drive 1090. Then, the CPU 1020 reads the program module 1093 and the program data 1094 stored in the memory 1010 and the hard disk drive 1090 into the RAM 1012 as needed, and executes various processing procedures.

The program module 1093 and program data 1094 related to the program are not limited to the case where they are stored in the hard disk drive 1090, and may be stored in, for example, a removable storage medium and read by the CPU 1020 via a disk drive or the like. .. Alternatively, the program module 1093 and the program data 1094 related to the program are stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.), and are stored via the network interface 1070. It may be read by the CPU 1020.

The above-described embodiments and modifications thereof are included in the inventions described in the claims and the equivalent scope thereof, as are included in the technology disclosed in the present application.

10 Information processing device 11 Communication processing unit 12 Control unit 12a Acquisition unit 12b Preprocessing unit 12c Estimating unit 13 Storage unit 13a Data storage unit 13b Setting information storage unit 13c Model storage unit 121c Setting information reception unit 122c Learning unit 123c Output unit

Claims (6)

An acquisition unit that acquires multiple data to which the sparse estimation method can be applied, and
A learning unit that learns the degree of application of each of the plurality of sparse estimation methods to a model that takes the plurality of data as inputs and applies a plurality of sparse estimation methods to perform a predetermined output.
A learning device characterized by having. The learning unit
It is input to the model in which the degree of application of each of the plurality of sparse estimation methods is set by using the error between the actual observed value and the predicted value output by the model with respect to the input of the plurality of data. A first learning process for learning the relationship between the plurality of data and the predetermined output with respect to the plurality of data, and
The accuracy of each degree of application of the plurality of sparse estimation methods is obtained based on a series of learning results by the learning unit, and each degree of application of the plurality of sparse estimation methods to be applied to the model is set based on each accuracy. The second learning process and
The learning apparatus according to claim 1, wherein the learning apparatus is performed. The learning apparatus according to claim 1 or 2, wherein the plurality of sparse estimation methods include any two or more of LASSO, Elastic Net, Adaptive LASSO, Group LASSO, Fused LASSO, and RIDGE. The learning device according to claim 3, wherein the Fused LASSO is applied by extending to N (> 1) floors. The process of acquiring multiple data to which the sparse estimation method can be applied, and
A step of learning each degree of application of the plurality of sparse estimation methods to a model that takes the plurality of data as inputs and applies a plurality of sparse estimation methods to perform a predetermined output.
A learning method characterized by including. Steps to acquire multiple data to which the sparse estimation method can be applied, and
A step of learning each degree of application of the plurality of sparse estimation methods to a model that takes the plurality of data as inputs and applies a plurality of sparse estimation methods to perform a predetermined output.
A learning program characterized by having a computer execute it.

Images