JP7359829B2 - Machine learning systems and methods - Google Patents

Description

The present invention relates to a machine learning system and a machine learning method.

Conventionally, outlier detection methods that can accurately detect outliers (see, for example, Patent Document 1) and outlier factor estimation methods that can easily infer the causes of outliers (for example, see Patent Document 2) have been proposed. )It has been known.

Japanese Patent Application Publication No. 2015-082190 Patent No. 6719612

By the way, in a machine learning system, if an outlier is included in the objective variable, it becomes difficult to ensure sufficient prediction accuracy using normal machine learning using a loss function such as the mean squared error. Furthermore, even if the value appears to be an outlier mathematically, it may actually be an error during measurement or entry, so there is a problem in that appropriate measures cannot be taken unless the cause is estimated. In particular, when the objective variable is expressed as a ratio of two numbers, such as "cost effectiveness" or "sales per talk time," it is necessary to determine whether the objective variable is an outlier in the numerator or denominator. I can't deal with it without it.

A machine learning system according to an aspect of the present invention includes a first outlier detection unit that performs a first outlier detection process on all records of learning data, with a detection target being an objective variable that is a prediction target of machine learning; a group processing unit that divides all the records into a plurality of groups based on explanatory variables included in the records; and a second outlier that performs a second outlier detection process using the target variable as a detection target for each group. a value detection unit, a cause estimation unit that estimates the cause of the outlier based on the detection results of the first outlier detection unit and the second outlier detection unit, and a cause estimation unit that estimates the cause of the outlier based on the estimation result of the cause estimation unit. A model creation unit that performs machine learning to create a learning model that predicts the target variable.

According to the present invention, the cause estimation of outliers can be further improved, and sufficient prediction accuracy can be ensured in machine learning.

FIG. 1 is a block diagram showing a configuration example of a machine learning system. FIG. 2 is a diagram showing an example of learning data. FIG. 3 is a flowchart showing the overall processing of the machine learning method in the machine learning system. FIG. 4 is a flowchart showing details of the deviation detection process executed in step S10 of FIG. FIG. 5 is a flowchart showing details of the outlier detection process for each group in step S170 of FIG. FIG. 6 is a flowchart showing detailed processing of outlier cause classification in step S1800 of FIG. FIG. 7 is a diagram illustrating grouping of learning data. FIG. 8 is a diagram illustrating an example of a screen display of the outlier cause estimation results. FIG. 9 is a flowchart showing details of the learning model creation process executed in step S20 of FIG. FIG. 10 is a flowchart showing details of the machine learning process in step S220 of FIG. FIG. 11 is a flowchart showing details of the learning model evaluation process in step S30 of FIG. FIG. 12 is a hardware diagram of a computer that implements the machine learning system.

Embodiments of the present invention will be described below with reference to the drawings. The following description and drawings are examples for explaining the present invention, and are omitted and simplified as appropriate for clarity of explanation. Furthermore, in the following description, the same or similar elements and processes may be denoted by the same reference numerals, and redundant explanations may be omitted. The content described below is merely an example of the embodiment of the present invention, and the present invention is not limited to the embodiment described below, and can be implemented in various other embodiments. It is possible.

The machine learning system of this embodiment uses machine learning to predict target variables that are expressed as a ratio of two numerical values, such as "cost effectiveness" and "sales per call time." It is a suitable machine learning system if it includes: FIG. 1 is a block diagram showing a configuration example of a machine learning system 1 according to the present embodiment. Databases related to machine learning include a learning data database DB1 (hereinafter referred to as learning data DB1) in which learning data and evaluation data are stored, and a setting information database DB2 (hereinafter referred to as setting information DB2) in which setting information is stored. ), an outlier information database DB3 (hereinafter referred to as outlier information DB3) in which outlier information is stored, and a model information database DB4 (hereinafter referred to as model information DB4) in which information regarding learning models is stored. We are prepared. It also includes an outlier detection section 10, a learning section 20, and a model evaluation section 30, which perform machine learning processing based on data in these databases.

FIG. 2 shows an example of learning data stored in the learning data DB1, and shows a table regarding the learning data. A table is made up of multiple columns and rows. The columns and rows of a table are called columns and records, respectively. Columns C1 to C5 store data such as customer name, age, occupation, call time, and sales amount, respectively. In this embodiment, customer name, age, occupation, call time, and sales amount will be referred to as variables. Each record R1 to R3 is composed of data for each variable (customer name, age, occupation, call time, and sales amount). For example, in the learning data R1, the customer name is A, the age is 48 years old, the occupation is a housewife, the call time is 600 seconds, and the sales amount is 100,000 yen. Although FIG. 2 only shows three records R1 to R3, the learning data includes many records.

Returning to FIG. 1, the setting information DB2 stores information on the probability distribution of the target variable, information on the digit correction coefficient, information on the loss function, information on missing value complementation, and the like. Furthermore, the outlier information DB 3 stores in advance information regarding the domain of the target variable and the period during which measurement troubles occurred. Furthermore, as will be described later, a series of results from the outlier detection process (for example, outlier cause estimation results, etc.) are also stored in the outlier information DB3. As will be described later, the model information DB 4 stores models obtained through a series of learning, information regarding the models (for example, loss functions used, corrections made, etc.), and the like.

The outlier detection section 10, the learning section 20, and the model evaluation section 30 perform arithmetic processing related to learning model creation, and each performs the processing shown in the flowchart of FIG. 3. FIG. 3 is a flowchart showing the overall processing of the machine learning method in the system. In this embodiment, a case will be explained in which the learning data shown in FIG. 2 is stored in the learning data DB1, and the target variable to be predicted using machine learning is the ratio = (sales amount)/(call time). do. When a target variable is expressed as a ratio, if at least one of the numerical values corresponding to the denominator or numerator of the ratio contains an incorrect value due to an error in measurement or entry, it becomes difficult to ensure the prediction accuracy of the target variable. In particular, errors in entering digits often result in mathematical outliers. In addition, when performing machine learning using each of the numerator and denominator as objective variables, if there are mathematical outliers in them, it is necessary to set a loss function that takes the outliers into account. Here, if the outlier is actually due to a typographical error, it is necessary to correct the typographical error instead of setting the loss function. Therefore, in order to perform machine learning with appropriate settings both when the numerical value expressed as a ratio is used as the objective variable directly, and when the numerator and denominator are individually used as objective variables, it is necessary to set the numerator and denominator separately. It becomes necessary to extract the included outliers and to estimate whether or not the cause of their generation is a typographical error.

Therefore, in this embodiment, we perform a process to detect statistical outliers not only for the target variable (=ratio) that is the prediction target, but also for the denominator of the target variable and the numerator of the target variable indicated by the ratio. Execute. Furthermore, records with similar attributes regarding "age" and "occupation", which are explanatory variables for the objective variable, are grouped together, and statistical outlier detection processing is again performed within each group. Then, the cause of the outlier is estimated based on the results of these two stages of processing, and preprocessing and loss function settings for generating a learning model are performed based on the cause estimation result.

In step S10 of FIG. 3, the outlier detection unit 10 executes the two-step outlier detection process described above to determine what kind of content the outlier is (for example, whether it is a true outlier or a typo). Perform outlier cause estimation to estimate outliers (such as outliers due to

In step S20, the learning unit 20 executes processing, and performs preprocessing and loss function settings for creating a learning model based on the cause estimation results obtained in step S10. Then, based on these settings, predictions are made regarding both the numerator and denominator of the target variable, and predictions regarding the target variable (=ratio). For example, machine learning that corrects typos or machine learning that uses a loss function that is resistant to outliers (robust regression) is applied, and the model with the best accuracy of each is adopted.

In step S30, the model evaluation unit 30 executes evaluation processing. For example, using the test data, the ratio is determined from the numerator/denominator individual prediction model (hereinafter referred to as model 1) determined in step S20. In addition, the value of a prediction model (hereinafter referred to as model 1) regarding the ratio between the two is also calculated using the same test data. Then, between Model 1 and Model 2, the one with higher prediction accuracy is adopted. Note that details of the processing performed by the outlier detection section 10, learning section 20, and model evaluation section 30 will be described later.

<Detailed explanation of outlier detection processing in outlier detection unit 10>
FIG. 4 is a flowchart showing details of the deviation detection process executed in step S10 of FIG. Note that in this embodiment, not only the ratio, which is the objective variable, but also the denominator (=call time) and numerator (=sales) of the ratio are also detected for outlier detection, so they are all referred to as objective variables. I'll decide. Hereinafter, the ratio, denominator, and numerator will be referred to as objective variable (ratio), objective variable (denominator), and objective variable (numerator), respectively.

In the flowchart of FIG. 4, the processes from step S100 to step S180 are performed for each target variable. That is, first perform the processing from step S100 to step S180 regarding the objective variable (denominator), then perform the processing from step S100 to step S180 regarding the objective variable (numerator), and finally perform the processing regarding the objective variable (ratio). , processes from step S100 to step S180 are performed.

In step S100, an outlier detection processing loop for each target variable is started. In step S110, the data of the objective variable column is acquired from the learning data. Note that if the objective variable (denominator) or objective variable (numerator) is the target, data in the call time or sales column of the table in FIG. 2 may be acquired. On the other hand, if the target variable (ratio) is the target variable, read the call time and sales from the table and use the value of (sales)/(call time) calculated from them as the target variable (ratio). .

In step S120, it is checked whether probability distribution information regarding the objective variable of interest, such as information indicating that sales can be approximated by a normal distribution, is registered in the setting information DB2. In step S130, it is determined whether or not there is probability distribution information. If there is (yes), the process proceeds to step S140, and if there is not (no), the process proceeds to step S150. In step S140, outlier detection processing 1A is performed based on the registered probability distribution. For example, if a normal distribution is registered as probability distribution information for the objective variable, outlier detection processing using the Smirnov-Grubbs test is performed. On the other hand, in step S150, an outlier detection process 1B that is applicable even if the following distribution is unknown, such as an outlier detection process that does not assume a specific probability distribution, such as an outlier detection process using an interquartile range, is performed. do.

In step S160, which value of the target variable is an outlier is stored in the outlier information DB3. Step S170 is a routine related to outlier detection processing on a group-by-group basis, and details of the processing will be described later using the flowchart of FIG. In step S180, the outlier detection processing loop for each target variable is completed. When the series of processes from step S100 to step S180 are completed for each of the objective variable (denominator), objective variable (numerator), and objective variable (ratio), the outlier detection process in FIG. 4 is completed. .

(Group-based outlier detection processing)
FIG. 5 is a flowchart showing details of the outlier detection process for each group in step S170 of FIG. In step S1700, all data are sorted into several groups based on explanatory variables using clustering or the like. In the case of the learning data shown in FIG. 2, records include customer name, age, and occupation as explanatory variables, and for example, the records can be grouped by occupation or age. For example, when grouping by occupation, assuming that there are three types of occupation (housewife, office worker, part-time worker) in the data shown in Figure 2, multiple records will be grouped as shown in Figures 7 (a) to (c). Divided into three types of groups. In the examples shown in FIGS. 7A to 7C, the determination results in outlier detection processing 1 (1A, 1B) and outlier detection processing 2, which will be described later, are also listed on the right side of column C5. However, Figures 7(a) and (b) show the outlier detection results regarding the objective variable (denominator) = call time, and Figure 7(c) shows the outlier detection results regarding the objective variable (numerator) = sales. It shows. Here, the grouping is performed with respect to one categorical explanatory variable, but even if the explanatory variable is a continuous value, the grouping may be performed using a plurality of intervals created by dividing the value. In addition, two or more explanatory variables may also be grouped using a method such as clustering.

Note that although the descriptions in Figures 7(a) to (c) are simplified, when outlier detection is performed in group units, each of the objective variable (numerator), objective variable (denominator), and objective variable (ratio) Since outlier detection is performed for .

In step S1710, the outlier information DB 3 is searched to see if there are any notes regarding call time and sales data, such as information such as the domain (for example, the upper limit of call time) or the period of measurement trouble. These are acquired from the outlier information DB3. In step S1720, an outlier detection processing loop for each group (processing from step S1720 to step S1810) is started. In the example of grouping shown in FIGS. 7(a) to (c), there are three types of groups (housewives, office workers, part-time workers), and the processing loop from step S1720 to step S1810 is for the housewife group. , is performed for a group of office workers and a group of part-time workers, respectively.

In step S1730, outlier detection processing 2 regarding the objective variable is performed. In order to distinguish it from the outlier detection process 1 (1A, 1B) using all records of the learning data described above, the outlier detection process for each group will be referred to as the outlier detection process 2. Here, a detailed explanation of the outlier detection process 2 will be omitted, but the same process as steps S110 to S160 in FIG. 4 is performed except that it is limited to records within a group. However, in the outlier detection process 2 in step S1730, outlier detection is performed without assuming a specific probability distribution.

In addition, in FIG. 4, when the outlier detection processing loop is executed for the objective variable, in the outlier detection processing for each group at that time (step S170), the outlier detection processing is executed for the same objective variable. become. For example, if the objective variable (numerator) in the outlier detection processing loop in FIG. 4 is the call time, the objective variable of the outlier detection process 2 in step S1730 also becomes the objective variable (numerator) = call time. That is, the value of column C4 is acquired from the grouped learning data and the outlier detection process is performed.

In step S1740, it is determined whether or not there is a record in which an outlier was detected in the outlier detection process 1 (1A, 1B). If there is (yes), the process advances to step S1750; if there is no record (no), the process proceeds to step S1750. Proceed to S1760. In step S1750, the outlier detection result of the record in which the outlier was detected is updated to "outlier", and then the process advances to step S1760. For example, in the examples shown in FIGS. 7(a) to (c), each record of customer names A1 to A3, B1, C1, and C2 has been detected as an outlier in the outlier detection process 1 (1A, 1B). , the process advances from step S1740 to step S1750, and the outlier detection result of each record is updated to "outlier".

To explain the update timing in more detail, each record of customer names A1 to A3 is a processing loop when the objective variable is call time (=objective variable (denominator)), and the processing is performed for each housewife group. Through the processing in steps S1740 and S1750 in the loop, the outlier detection result regarding call time is updated to "outlier". The record for customer name B1 is a processing loop in which the objective variable is call time (= objective variable (denominator)), and the record related to call time is obtained through the processing of steps S1740 and S1750 in the processing loop for each company employee group. The outlier detection result is updated to "outlier". Each record of customer names C1 and C2 is a processing loop when the objective variable is sales (= objective variable (numerator)), and the sales are calculated by the processing of steps S1740 and S1750 in the processing loop for part-time job groups. The outlier detection result regarding high is updated to "outlier".

In step S1760, it is determined whether or not there is a record in which an outlier was detected in the outlier detection process 2. If there is (yes), the process advances to step S1770, and if there is no record (no), the process advances to step S1780. In step S1770, the outlier detection result of the record in which the outlier was detected is updated to "general error", and the process then proceeds to step S1780. For example, in the example shown in FIGS. 7(a) to (c), each record of customer names B1 and C3 has been detected as an outlier in the outlier detection process 2, so the process advances from step S1760 to step S1770, and each record Update the outlier detection results to "general errors". Note that the outlier detection result for the record of customer name B1 is determined to be an "outlier" in step S1750, but is updated to "general error" again in step S1770.

In step S1780, it is determined whether the data of each record (call time, sales amount) includes data outside the defined range or data measured during a period in which there was a problem with the measuring equipment. The process advances to step S1790, and if there is none (no), the process advances to step S1800. In step S1790, the outlier detection result of the corresponding record is updated to "error due to measurement error", and the process then proceeds to step S1800. For example, if 3,600 seconds is specified as the defined range regarding call time, the data of 4,800 seconds is determined to be a "misprint due to a measurement error." Furthermore, if there is note information indicating that the data was taken during a period when there was a problem with the measuring equipment, it is unconditionally determined to be a "mistake due to a measurement error."

In the case of the example shown in FIGS. 7(a) to 7(c), when the processes from step S1730 to step S1790 regarding the housewife group are performed, the outlier detection results for each record of customer names A1 to A3 are Data related to time is considered an "outlier." Furthermore, when the processes from step S1730 to step S1790 regarding the company employee group are performed, the outlier detection result for the record of customer name B1 is that the data regarding the call time is a "general error." Furthermore, when the processing from step S1730 to step S1790 regarding the group of part-time workers is performed, the outlier detection results for the records with customer names C1 and C2 are such that the data related to sales is determined to be an "outlier", and the data regarding customer name C3 is determined to be an "outlier". Outlier detection results for records indicate that data related to sales is a "general error." Step S1800 is a routine related to outlier cause classification, and details of the process are shown in FIG.

(Detailed explanation of outlier cause classification process)
In step S1802 of FIG. 6, the digit correction coefficient is acquired from the setting information DB2. The digit correction coefficient is used to correct the digits of the data value, that is, the position of the decimal point, by multiplying the data determined ^to be incorrect.For example, 10 ^-5 , ^{10 -4 ,} . Numerical values in 10 times increments are stored in advance in the setting information DB2 as digit correction coefficients. In step S1804, a multiplication process of multiplying the data value determined to be a "general error" by a digit correction coefficient is performed for each of the plurality of obtained digit correction coefficients.

In step S1806, it is determined whether any of the plurality of values after multiplication is within the frequent occurrence range of data values that are not errors or outliers within the same group. An example of the frequent range here is, for example, a range from "quartile" to "third quartile." If the determination in step S1806 is yes, the process advances to step S1808, and if the determination is no, step S1808 is skipped. In step S1808, the outlier detection result is updated from "general error" to "error due to digit error."

Returning to FIG. 5, in step S1810, the outlier detection processing loop for each group is completed. When the group-based outlier detection loop is completed for all groups, the process advances to step S1820. In step S1820, a series of outlier detection results are stored in the outlier information DB3. Note that if the outlier detection result is "an error in writing due to a digit error," the digit correction coefficient at that time is stored in the outlier information DB 3 together with the outlier detection result.

When the process in FIG. 5 is completed, the process returns to FIG. 4 and proceeds to step S180. In step S180, the outlier detection processing loop for each target variable is completed. When the outlier detection processing loop is completed for all target variables, that is, the target variable (denominator), target variable (numerator), and target variable (ratio), the series of processes shown in FIG. 4 ends.

Note that the outlier cause estimation result may be displayed on a screen on a monitor provided in the output device 5700, which will be described later. FIG. 8 shows an example of a screen display of the outlier cause estimation results. Fields with dashed rectangular frames are target variables determined to be outliers and errors.

<Detailed explanation of the learning model creation process in the learning unit 20>
Next, details of the learning model creation process executed in step S20 of FIG. 3 will be explained using the flowchart of FIG. 9. In the flowchart of FIG. 9, the processes from step S200 to step S240 are performed for each target variable unit of the target variable (denominator), target variable (numerator), and target variable (ratio). In the learning model creation process, if an outlier or misprint is detected in the objective variable by the outlier detection process in step S10, preprocessing and loss function correction are performed when creating the learning model based on the obtained cause estimation results. Make settings. On the other hand, if neither an outlier nor an error in the target variable is detected in the outlier detection process, learning is performed as usual. For each learning, multiple learning algorithms may be tried as in normal AutoML, and hyperparameters may be optimized during learning for each algorithm to be tried.

In step S200, a model creation processing loop for each target variable is started. In step S210, it is checked whether information about a misprint of the objective variable or an outlier is registered in the outlier information DB3. Step S220 is a routine related to machine learning processing according to the contents of errors and outliers, and details of the processing are shown in FIG. 10. In addition, in the machine learning process according to the contents of the misprint or outlier in step S220, if the misprint of the target variable or information about the outlier is not registered in the outlier information DB 3, learning is performed as usual. .

(Detailed explanation of machine learning processing according to the contents of errors and outliers)
FIG. 10 is a flowchart showing details of machine learning processing according to the contents of errors and outliers. In step S2200, information on outliers, typos, and digit correction coefficients regarding each objective variable is acquired from the outlier information DB3. In step S2210, the value of the target variable is corrected by multiplying the target variable of "error in writing due to wrong digit" by the corresponding digit correction coefficient. For example, in the record of customer name C3 in FIG. 8, the sales value "30 yen" is determined to be an error due to an incorrect digit, and "10 ² " is registered as the digit correction coefficient in the outlier information DB3. . Therefore, in step S2210, the value "30 yen" is corrected to the value "3,000 yen" obtained by multiplying by "10 ² ".

In step S2210, it is determined whether or not there is a record determined to be an "outlier" in the objective variable. If there is (yes), the process proceeds to step S2230; if there is not (no), the process proceeds to step S2240. move on. If there are outliers that are not errors, perform robust regression. Robust regression is necessary for the following reasons.
- Some learning algorithms based on linear regression, tree models, and neural networks can be executed by replacing the loss function to be optimized.
- An example of a loss function is the mean squared error, but if learning is done to optimize this, learning will be done that preferentially reduces the predicted actual measurement error for outliers. Therefore, the predicted actual measurement error for values that are not outliers is not sufficiently reduced, and a model that is insufficient in predicting values other than outliers is generated.
・Therefore, there are ways to make it easier to generate models that accurately predict values that are not outliers, such as by using a loss function that does not easily increase in value due to the presence of outliers, and these methods are collectively called robust regression. .

Specific examples of loss functions used in robust regression include "Huber-loss," "pseudo-Huber-loss," and "τ-quantile loss." In step S2230, the type of loss function for robust regression to be performed in the model creation process is acquired from the setting information DB2, and then the process advances to step S2240.

In step S2240, it is determined whether or not there is a record in the objective variable that has been determined to be a typographical error other than "error due to digit error". If yes (yes), the process advances to step S2250; if not (no) Then, the process advances to step S2260. In step S2250, the type of missing value complementation to be performed in the model creation process is acquired from the setting information DB2, and the process then proceeds to step S2260.

If there are errors other than "errors due to incorrect digits" such as "general errors" and "errors due to measurement errors," the relevant objective variable is considered to be missing, and the missing value imputation method used as a countermeasure is 1. Create a learning model after at least one trial. Examples of methods for filling in missing values include a method of excluding them from data to be learned, a method of filling with another value, and a method of applying semi-supervised learning. As a method of complementing with another value, for example, there is a method of complementing with a median value, an average value, or a separately specified value. In this case, random noise may be further added to the value to be complemented. There are also completion methods that use matrix decomposition, and completion methods that apply algorithms that can be used for regression, such as missForest.

In step S2260, the setting information DB2 is referred to and a loss function used when robust regression is not performed is acquired from the setting information DB2. In step S2270, machine learning is performed in accordance with the combination of the series of loss function types and missing value completion types acquired as described above. For example, if there are three types of acquired loss functions and three types of missing value complementation, there are nine (=3×3) combinations of loss functions and missing value complementation. Then, in step S2270, machine learning is performed with each combination setting. Note that normal learning when there is no "outlier" is also included as a combination of loss function (=normal squared error) x missing value interpolation (=no interpolation). In step S2280, the model obtained through the series of learning is stored in the model information DB4. As can be seen from FIG. 9, the machine learning in step S2270 is performed for each of the three objective variables (numerator), objective variable (denominator), and objective variable (ratio).

Returning to FIG. 9, in step S230, the models obtained in each learning are evaluated using common test data, and the model with the best accuracy is stored in the model information DB4. In step S240, when the loop of learning processing for each objective variable is completed, the series of processes related to model creation shown in FIG. 9 is completed.

<Detailed explanation of learning model evaluation process>
Details of the learning model evaluation process in step S30 in FIG. 3 will be explained with reference to the flowchart in FIG. 11. In step S300, evaluation data is acquired from the learning data DB1. In step S310, the model with the best accuracy for each of the target variable (numerator), target variable (denominator), and target variable (ratio) stored in the model information DB4 in step S230 is acquired from the model information DB4. Here, each of the acquired models will be referred to as a numerator prediction model, a denominator prediction model, and a ratio prediction model.

In step S320, the values of the numerator, denominator, and ratio are estimated from the numerator prediction model, denominator prediction model, and ratio prediction model, respectively, using the evaluation data acquired in step S300. In step S330, the accuracy of (numerator/denominator) (=accuracy 1) and the accuracy of the ratio (=accuracy 2) are compared to determine whether "accuracy 1>accuracy 2". If it is determined in step S330 that "accuracy 1>accuracy 2", the process proceeds to step S340, where the numerator prediction model and the denominator prediction model are adopted as the best models, and the determination results are stored in the model information DB4. On the other hand, if it is determined in step S330 that "accuracy 1>accuracy 2" is not satisfied, the process proceeds to step S350, where the ratio prediction model is adopted as the best model, and the determination result is stored in the model information DB4.

In addition, in the embodiment described above, the case where the objective variable is expressed as a ratio has been explained, but even if the objective variable is not a ratio, for example, even when the sales amount shown in FIG. The two-stage outlier detection process shown in 4 to 6 can be applied. The target variable in the processing from step S100 to step S180 in the flowchart of FIG. 5 is only the sales amount, and the two-step outlier detection processing shown in FIGS. 4 to 6 is performed with respect to the target variable=sales amount. The learning model creation process in FIGS. 9 and 10 and the learning model evaluation process in FIG. 11 following the outlier detection process are also performed assuming that the target variable is sales. In other words, the two-step outlier detection process, learning model creation process, and learning model evaluation process in this embodiment can be applied to cases where the objective variable is a ratio and when it is not a ratio. It is possible to construct highly accurate learning models.

<Computer that realizes machine learning system 1>
FIG. 12 is a hardware diagram of a computer 5000 that implements the machine learning system 1. A computer 5000 that implements the machine learning system 1 includes a processor 5300 represented by a CPU (Central Processing Unit), a memory 5400 such as a RAM (Random Access Memory), an input device 5600 (for example, a keyboard, a mouse, a touch panel, etc.), and an output. Devices 5700 (eg, a video graphics card connected to an external display monitor) are interconnected through a memory controller 5500.

In the computer 5000, a program for realizing a machine learning system is read out from an external storage device 5800 such as an SSD or HDD via an I/O (Input/Output) controller 5200, and is executed in cooperation with a processor 5300 and a memory 5400. Executed by Thereby, the machine learning system 1 is realized. Alternatively, the program for implementing the machine learning system 1 may be obtained from an external computer through communication via the network interface 5100, or may be obtained by being read from a recording medium by a medium reading device.

According to the embodiment of the present invention described above, the following effects are achieved.

(C1) As shown in FIGS. 1 to 8, the machine learning system 1 includes an outlier detection section 10 and a learning section 20. The outlier detection unit 10 performs a first outlier detection process on all records of the learning data, using the target variable that is the prediction target of machine learning as the detection target. For example, if the target variable is sales, a first outlier detection process is performed in which sales are the detection target. In addition, the outlier detection unit 10 may classify all records R1, R2, R3, etc. into a group of housewives, a group of office workers, a group of part-time workers, etc., based on the explanatory variable of occupation included in records R1 to R3, for example. Perform group processing to divide the data into groups. Furthermore, the outlier detection unit 10 performs a second outlier detection process that targets sales for each of the plurality of groups. Furthermore, the outlier detection unit 10 estimates the cause of the outlier based on the detection results of the first and second outlier detection processes. Then, the learning unit 20 performs machine learning based on the estimation result of the cause of the outlier to create a learning model for predicting sales.

As described above, the first outlier detection process is performed on all records of the training data using sales as the detection target, and the second outlier detection process is performed on each grouped record group using sales as the detection target. The cause of the outlier is estimated based on these two stages of outlier detection processing. As a result, the estimation of the causes of outliers is improved, and it becomes possible to perform appropriate machine learning according to the causes of outliers.

(C2) As shown in Figures 1 to 8, the target variable is expressed as the ratio of the first variable (sales amount) and the second variable (call time) included in records R1 to R3 of the learning data. In this case, the outlier detection unit 10 performs a first outlier detection process that detects any one of the target variable (ratio), sales amount, and call time on all records of the learning data. Do each for each time. Furthermore, the outlier detection unit 10 performs a second outlier detection process for each of the plurality of groups, in which one of the target variable (ratio), sales amount, and call time is detected. Do each. The learning unit 20 then performs machine learning based on the results of estimating the cause of the outlier to create a learning model that predicts the ratio between sales and call time.

As mentioned above, the first outlier detection process is performed on all records of the training data using the ratio, the numerator of the ratio, and the denominator of the ratio as objective variables, and the ratio and the numerator of the ratio are calculated for each grouped record group. A second outlier detection process is performed using the denominator of the ratio and the denominator of the ratio as objective variables, and the cause of the outlier is estimated based on these two stages of outlier detection process. As a result, the causes of outliers can be found for each of the ratio, the numerator of the ratio, and the denominator of the ratio, and it becomes possible to perform appropriate machine learning according to the cause of the outliers.

(C3) As shown in FIG. 5, in estimating the cause of an outlier, the outlier detection unit 10 estimates the detection target detected by the first outlier detection process as an outlier, and detects the second outlier. The detection target detected by the detection process is presumed to be a typo. By estimating the causes of outliers in this way, it is possible to distinguish between true outliers and typos, even if they include things that appear to be outliers mathematically, and to take appropriate measures for each. It becomes possible to perform machine learning.

(C4) As shown in FIGS. 6 and 8, in estimating the cause of an outlier, the outlier detection unit 10 multiplies a plurality of digit correction coefficients of different sizes by the detection target estimated to be an error, If the value after multiplication is within the frequency range of the same detection target that is not determined to be an outlier in the record in the same group, it is assumed that the error is due to a digit error.

For example, in the record where the customer name is C3 in Figure 8, the sales amount of 30 yen is determined to be an outlier, and the value is changed to 10 ^-5 , 10 ^-4 , ..., 10 ⁴ , 10 ⁵ , etc. When multiplied by a plurality of digit correction coefficients in increments of 10 times, 3,000 yen when multiplied by 10 ² falls within the numerical range of sales that are not determined to be outliers. That is, it is estimated that 30 yen is a typographical error due to a digit difference, and it is possible to determine that the cause is a typographical error due to a digit discrepancy, rather than just a simple typographical error.

(C5) As shown in FIGS. 8 and 10, the outlier detection unit 10 multiplies the detection target (for example, the sales amount of 30 yen in FIG. 8) that is estimated to be an error due to a digit error, so that the result is within the frequent occurrence range. The value is multiplied by a digit correction coefficient (=10 ² ) to correct 30 yen to 3,000 yen (step S2210). Then, the learning unit 20 performs machine learning using the corrected sales amount of 3,000 yen instead of the detected sales amount of 30 yen, which is estimated to be a typographical error due to a digit error. As a result, accuracy can be improved compared to the case where errors in writing due to digit errors are not corrected.

(C6) As shown in FIG. 10, when there is a detection target estimated to be an outlier, the learning unit 20 performs machine learning using robust regression to create a learning model. If there are outliers, normal machine learning using mean squared error as a loss function cannot guarantee sufficient prediction accuracy, but machine learning using robust regression as described above can improve prediction accuracy. can be achieved.

(C7) As shown in FIGS. 10 and 11, the learning unit 20 has a numerator prediction model that predicts the target variable (numerator), which is sales, and a denominator prediction model, which predicts the target variable (denominator), which is call time. , and a ratio prediction model that predicts the objective variable (ratio). Then, the machine learning system 1 calculates the predicted values of the numerator prediction model, the denominator prediction model, and the ratio prediction model using the evaluation data, and determines the accuracy of the ratio between the predicted value of the numerator prediction model and the predicted value of the denominator prediction model. , and a model evaluation unit 30 that compares the accuracy of the predicted value of the ratio prediction model and adopts the prediction model with higher accuracy as the learning model.

In this way, in addition to the ratio prediction model, we also consider the numerator prediction model and the denominator prediction model, and calculate the accuracy of the predicted value of the ratio prediction model and the accuracy of the ratio between the predicted value of the numerator prediction model and the predicted value of the denominator prediction model. Since the learning model is determined from a comparison of the above, a more accurate learning model can be obtained.

(C8) As shown in FIGS. 1 to 8, the machine learning method of the present invention performs a first outlier detection process on all records of learning data, using the objective variable that is the prediction target of machine learning as the detection target. , based on the explanatory variable of occupation included in records R1 to R3, all records R1, R2, R3,... are divided into a group of housewives, a group of office workers, a group of part-time workers, and each group of multiple groups is divided. Then, a second outlier detection process is performed using the objective variable as a detection target, and the cause of the outlier is estimated based on the detection results of the first outlier detection process and the second outlier detection process. Perform machine learning based on the causes of outliers to create a learning model that predicts the target variable.

The embodiments described above are merely examples, and the present invention is not limited to these contents as long as the characteristics of the invention are not impaired. Other embodiments considered within the technical spirit of the present invention are also included within the scope of the present invention. Moreover, the above-described embodiments and modified examples can be combined in part or in whole within a mutually compatible range without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 1...Machine learning system, 10...Outlier detection unit, 20...Learning unit, 30...Model evaluation unit, 5000...Computer, 5700...Output device, DB1...Learning data database, DB2...Setting information database, DB3...Outlier information Database, DB4...Model information database

Claims (8)

a first outlier detection unit that performs a first outlier detection process on all records of the learning data, with a target variable that is a prediction target of machine learning as a detection target;
a group processing unit that divides all the records into a plurality of groups based on explanatory variables included in the records;
a second outlier detection unit that performs a second outlier detection process using the objective variable as a detection target for each group;
a cause estimation unit that estimates the cause of the outlier based on the detection results of the first outlier detection unit and the second outlier detection unit;
A machine learning system comprising: a model creation unit that performs machine learning based on the estimation result of the cause estimation unit to create a learning model that predicts the objective variable. The machine learning system according to claim 1,
The target variable is expressed as a ratio of a first variable and a second variable included in the learning data record,
The first outlier detection unit detects any one of the target variable, the first variable, and the second variable as a detection target for all records of the learning data. Performing processing for each of the detection targets,
The second outlier detection unit performs a second outlier detection process in which one of the target variable, the first variable, and the second variable is detected for each group. Do this for each target,
The model creation unit is a machine learning system that performs machine learning based on the estimation result of the cause estimation unit to create a learning model that predicts a ratio between the first variable and the second variable. The machine learning system according to claim 1,
The cause estimation unit is a machine that estimates the detection target detected by the first outlier detection unit to be an outlier, and estimates the detection target detected by the second outlier detection unit to be a typographical error. learning system. The machine learning system according to claim 3,
The cause estimating unit multiplies a plurality of digit correction coefficients of different sizes by the detection target estimated to be a typographical error, and determines that the value after the multiplication is not determined to be an outlier of the record in the same group. A machine learning system that estimates errors due to incorrect digits when the same detection target occurs frequently. The machine learning system according to claim 4,
further comprising a correction unit that corrects the detection target by multiplying the detection target estimated to be an error due to a digit error by the digit correction coefficient whose multiplication result falls within the frequently occurring range;
The model creation unit is a machine learning system that performs machine learning using the detection target corrected by the correction unit instead of the detection target estimated to be a typographical error due to the digit error. The machine learning system according to claim 3,
The model creation unit is a machine learning system that performs machine learning using robust regression to create a learning model when there is a detection target estimated to be an outlier by the cause estimation unit. The machine learning system according to claim 2,
The model creation unit creates a first prediction model that predicts the first variable, a second prediction model that predicts the second variable, and a third prediction model that predicts the target variable. death,
Using the evaluation data, calculate the predicted values of the first predictive model, the second predictive model, and the third predictive model, and calculate the predicted values of the first predictive model and the second predictive model. Machine learning, further comprising a model evaluation unit that compares the accuracy of the ratio with the predicted value and the accuracy of the predicted value of the third prediction model, and adopts the prediction model with higher accuracy as the learning model. system. For all records of the learning data, perform a first outlier detection process using the objective variable that is the prediction target of machine learning as the detection target,
dividing all the records into a plurality of groups based on explanatory variables included in the records;
performing a second outlier detection process using the target variable as a detection target for each group;
Estimating the cause of the outlier based on the detection results of the first outlier detection unit and the second outlier detection unit,
A machine learning method that performs machine learning based on the estimated cause of the outlier to create a learning model that predicts the objective variable.

Images