JP2023087998A - Machine learning system and machine learning method - Google Patents

Abstract

To provide a machine learning system capable of further improving estimation of cause of an outlier and ensuring sufficient prediction accuracy.SOLUTION: A machine learning system 1 comprises an outlier detection unit 10 and a learning unit 20. The outlier detection unit 10 performs a first outlier detection process with an objective variable (for example, sales) as a detection target for all records, divides all the records into a plurality of groups based on explanatory variables, performs a second outlier detection process with the objective variable as the detection target for each group, and estimates cause of the outlier based on results of these two outlier detection processes. The learning unit 20 performs machine learning based on the results of estimating the cause of the outlier, and creates a learning model for predicting the objective variable.SELECTED DRAWING: Figure 1

Description

The present invention relates to machine learning systems and machine learning methods.

Conventionally, an outlier detection method capable of accurately detecting an outlier (see, for example, Patent Document 1) and an outlier factor estimation method capable of easily inferring the cause of an outlier (see, for example, Patent Document 2) )It has been known.

JP 2015-082190 A Japanese Patent No. 6719612

By the way, in a machine learning system, when an outlier is included in the objective variable, it becomes difficult to ensure sufficient prediction accuracy with normal machine learning in which the mean square error or the like is used as a loss function. In addition, even if the value appears to be an outlier from a mathematical point of view, there is a possibility that it was actually an error during measurement or entry, so there is a problem that appropriate countermeasures cannot be taken unless the cause is estimated. In particular, if the objective variable is expressed as a ratio of two numbers, such as "cost-effectiveness" or "sales per call hour," determine whether the numerator or denominator causes the objective variable to become an outlier. I can't deal with it without it.

A machine learning system according to an aspect of the present invention includes a first outlier detection unit that performs a first outlier detection process for detecting an objective variable that is a prediction target of machine learning for all records of learning data; A group processing unit that divides all the records into a plurality of groups based on an explanatory variable included in the record; a value detection unit; a cause estimation unit for estimating a cause of an outlier based on the detection results of the first outlier detection unit and the second outlier detection unit; and based on the estimation result of the cause estimation unit and a model creating unit that performs machine learning to create a learning model for predicting the objective variable.

ADVANTAGE OF THE INVENTION According to this invention, the cause estimation of an outlier can be improved more and sufficient prediction accuracy can be ensured in machine learning.

FIG. 1 is a block diagram showing a configuration example of a machine learning system. FIG. 2 is a diagram showing an example of learning data. FIG. 3 is a flow chart showing the overall processing of the machine learning method in the machine learning system. FIG. 4 is a flow chart showing details of the deviation detection process executed in step S10 of FIG. FIG. 5 is a flowchart showing the details of the outlier detection processing in units of groups in step S170 of FIG. FIG. 6 is a flowchart showing detailed processing for outlier cause classification in step S1800 of FIG. FIG. 7 is a diagram for explaining grouping of learning data. FIG. 8 is a diagram showing an example of a screen display of outlier cause estimation results. FIG. 9 is a flow chart showing details of the learning model creation process executed in step S20 of FIG. FIG. 10 is a flow chart showing details of the machine learning process in step S220 of FIG. FIG. 11 is a flowchart showing details of the learning model evaluation process in step S30 of FIG. FIG. 12 is a hardware diagram of a computer that implements the machine learning system.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following description and drawings are examples for explaining the present invention, and are appropriately omitted and simplified for clarity of explanation. Also, in the following description, the same or similar elements and processes are denoted by the same reference numerals, and redundant description may be omitted. It should be noted that the contents described below merely show an example of the embodiment of the present invention, and the present invention is not limited to the following embodiment, and can be implemented in various other forms. It is possible.

In the machine learning system of the present embodiment, the target variable to be predicted using machine learning is represented by the ratio of two numerical values such as "cost-effectiveness" and "sales per call time". It is a preferred machine learning system when FIG. 1 is a block diagram showing a configuration example of a machine learning system 1 according to this embodiment. As databases related to machine learning, a learning data database DB1 (hereinafter referred to as learning data DB1) that stores learning data, evaluation data, etc., and a setting information database DB2 (hereinafter referred to as setting information DB2) that stores setting information. ), an outlier information database DB3 (hereinafter referred to as outlier information DB3) in which outlier information is stored, and a model information database DB4 (hereinafter referred to as model information DB4) in which information on learning models is stored. I have. It also has an outlier detection unit 10, a learning unit 20, and a model evaluation unit 30 that perform machine learning processing based on the data in these databases.

FIG. 2 shows an example of learning data stored in the learning data DB1, and shows a table related to the learning data. A table consists of multiple columns and rows. Columns and rows of a table are called columns and records, respectively. Columns C1 to C5 store customer name, age, occupation, call duration, and sales data, respectively. In this embodiment, the customer name, age, occupation, call duration, and sales amount are called variables. Each record R1-R3 consists of data of each variable (customer name, age, occupation, call duration and sales). For example, in learning data R1, customer name is A, age is 48, occupation is housewife, call time is 600 seconds, and sales is 100,000 yen. Note that although only three records R1 to R3 are shown in FIG. 2, many records are included in the learning data.

Returning to FIG. 1, the setting information DB2 stores information on the probability distribution of objective variables, information on digit correction coefficients, information on loss functions, information on missing value interpolation, and the like. In addition, the outlier information DB 3 stores in advance information about the domain of the objective variable and the period during which the measurement trouble occurred. In addition, as will be described later, a series of results (for example, outlier cause estimation results, etc.) from the outlier detection process are also stored in the outlier information DB3. As will be described later, the model information DB 4 stores models obtained through a series of learning, information about the models (for example, loss functions used, corrections performed, etc.), and the like.

The outlier detection unit 10, the learning unit 20, and the model evaluation unit 30 perform arithmetic processing related to creation of a learning model, and each perform processing shown in the flowchart of FIG. FIG. 3 is a flow chart showing the overall processing of the machine learning method in the system. In this embodiment, the learning data shown in FIG. 2 is stored in the learning data DB 1, and the target variable to be predicted using machine learning is the ratio=(sales)/(call time). do. When the objective variable is expressed as a ratio, if at least one of the numerical values corresponding to the denominator or numerator of the ratio contains an erroneous value due to an error in measurement or entry, it becomes difficult to ensure the prediction accuracy of the objective variable. In particular, those due to erroneous entries of digits often take mathematical outliers. In addition, when performing machine learning using the numerator and denominator as objective variables, if there are mathematical outliers in them, it is necessary to set a loss function that considers the outliers. Here, if the outlier is actually due to a typographical error, it is necessary to correct the typographical error instead of setting the loss function. Therefore, in order to perform machine learning under appropriate settings for both cases where the numerical value expressed by the ratio is directly used as the target variable and when the numerator and denominator are set as separate target variables, it is necessary to set It is necessary to extract the included outliers and estimate whether or not the cause of their generation is an error in writing.

Therefore, in the present embodiment, a process for detecting statistical outliers is performed not only for the objective variable (=ratio) to be predicted, but also for the denominator and numerator of the objective variable indicated by the ratio. to run. Furthermore, records with similar attributes regarding "age" and "occupation", which are explanatory variables for the objective variable, are grouped together, and statistical outlier detection processing is performed again within each group. Based on the results of these two stages of processing, the cause of the outliers is estimated, and preprocessing and loss functions are set for learning model generation based on the result of the cause estimation.

In step S10 of FIG. 3, the process by the outlier detection unit 10 is executed, the two-step outlier detection process described above is performed, and the content of the outlier (for example, a true outlier or an error) is detected. Perform outlier cause estimation to estimate outliers due to

In step S20, processing by the learning unit 20 is executed, and based on the cause estimation result obtained in step S10, preprocessing and loss function setting are performed when creating a learning model. Then, based on these settings, both predictions for the numerator and denominator of the objective variable and predictions for the objective variable (=ratio) are performed. For example, machine learning that corrects typos and machine learning that uses a loss function that is strong against outliers (robust regression) is applied, and the model with the best accuracy for each is adopted.

In step S30, evaluation processing by the model evaluation unit 30 is executed. For example, using test data, the ratio is obtained from the individual numerator/denominator prediction model (hereinafter referred to as model 1) obtained in step S20. In addition, the value of a predictive model (hereafter referred to as model 1) regarding the ratio of the two is also calculated using the same test data. Then, of Model 1 and Model 2, the model with the higher prediction accuracy is adopted. Details of the processing performed by the outlier detection unit 10, the learning unit 20, and the model evaluation unit 30 will be described later.

<Detailed Description of Outlier Detection Processing in Outlier Detection Unit 10>
FIG. 4 is a flow chart showing details of the deviation detection process executed in step S10 of FIG. In the present embodiment, not only the ratio, which is the objective variable, but also the denominator (=call time) and numerator (=sales) of the ratio are targets for outlier detection, so they are all referred to as objective variables. to decide. Hereinafter, the ratio, denominator, and numerator are referred to as objective variable (ratio), objective variable (denominator), and objective variable (numerator), respectively.

In the flowchart of FIG. 4, the processing from step S100 to step S180 is performed for each objective variable unit. That is, first, the process from step S100 to step S180 is performed for the objective variable (denominator), then the process from step S100 to step S180 is performed for the objective variable (numerator), and finally the objective variable (ratio) is , the processing from step S100 to step S180 is performed.

In step S100, an outlier detection processing loop for each objective variable is started. In step S110, the data of the objective variable column is obtained from the learning data. When the object variable (denominator) or the object variable (numerator) is the object, the data in the call time or sales column of the table in FIG. 2 can be acquired. On the other hand, if the objective variable (ratio) is the target, the call duration and sales are read from the table, and the value of (sales)/(call duration) calculated from them is used as the objective variable (ratio). .

In step S120, it is checked whether probability distribution information regarding the objective variable of interest, for example, information that sales can be approximated by a normal distribution, is registered in the setting information DB2. In step S130, it is determined whether or not there is probability distribution information. If there is (yes), the process proceeds to step S140, and if there is no (no), the process proceeds to step S150. In step S140, an outlier detection process 1A is performed on the premise of the registered probability distribution. For example, when a normal distribution is registered as probability distribution information for the objective variable, outlier detection processing using the Smirnov-Grubbs test is performed. On the other hand, in step S150, an outlier detection process 1B that can be applied even if the distribution to follow is unknown, such as an outlier detection process that does not assume a specific probability distribution, for example, an outlier detection process that uses an interquartile range, is performed. do.

In step S160, which value of the objective variable is the outlier is stored in the outlier information DB3. Step S170 is a routine relating to outlier detection processing in units of groups, and the details of the processing will be described later using the flowchart of FIG. In step S180, the outlier detection processing loop for each objective variable is completed. Then, when a series of processes from step S100 to step S180 are completed for each of the objective variable (denominator), objective variable (numerator) and objective variable (ratio), the outlier detection process of FIG. 4 is terminated. .

(Outlier detection processing for each group)
FIG. 5 is a flowchart showing the details of the outlier detection processing in units of groups in step S170 of FIG. In step S1700, clustering or the like is used to sort all data into several groups for each explanatory variable. In the case of the learning data shown in FIG. 2, the record has customer name, age, and occupation as explanatory variables, and can be grouped by occupation and age, for example. For example, when grouping by occupation, assuming that there are three types of occupation (housewife, office worker, part-time worker) in the data shown in FIG. They are divided into three groups. In the examples shown in FIGS. 7A to 7C, the determination results of outlier detection processing 1 (1A, 1B) and outlier detection processing 2, which will be described later, are also described on the right side of column C5. However, FIGS. 7A and 7B show outlier detection results related to the objective variable (denominator)=call time, and FIG. 7C shows outlier detection results related to the objective variable (numerator)=sales. showing. Here, grouping is performed with respect to one categorical explanatory variable, but even with continuous value explanatory variables, grouping may be performed using a plurality of intervals created by dividing the value. In addition, two or more explanatory variables may also be grouped using a technique such as clustering.

Although the descriptions in FIGS. 7A to 7C are simplified, even when outlier detection is performed on a group-by-group basis, the objective variable (numerator), objective variable (denominator), and objective variable (ratio) Since outlier detection is performed with respect to .

In step S1710, the outlier information DB 3 is searched for remarks on the call time and sales data, such as information on the domain (for example, upper limit of call time) and measurement trouble period. They are acquired from the outlier information DB3. In step S1720, an outlier detection processing loop (processing from step S1720 to step S1810) is started for each group. In the example of grouping shown in FIGS. 7A to 7C, the groups are divided into three types (housewives, office workers, and part-time workers). , for the group of office workers and the group of part-time workers, respectively.

In step S1730, outlier detection processing 2 for the objective variable is performed. In addition, in order to distinguish from the outlier detection processing 1 (1A, 1B) using all the records of the learning data described above, the outlier detection processing for each group will be referred to as the outlier detection processing 2. FIG. Here, a detailed description of the outlier detection process 2 is omitted, but the same processes as those of steps S110 to S160 in FIG. 4 are performed except that the records are limited to the records in the group. However, in outlier detection processing 2 in step S1730, outlier detection is performed without a specific probability distribution.

In FIG. 4, when the outlier detection processing loop is executed for the objective variable, the outlier detection processing is executed for the same objective variable in the group unit outlier detection processing (step S170) at that time. become. For example, if the outlier detection processing loop in FIG. 4 has the objective variable (numerator)=call time, the objective variable of the outlier detection process 2 in step S1730 is also the objective variable (numerator)=call time. That is, outlier detection processing is performed by acquiring the value of column C4 from the grouped learning data.

In step S1740, it is determined whether or not there is a record for which an outlier has been detected in outlier detection processing 1 (1A, 1B). Proceed to S1760. In step S1750, the outlier detection result of the record in which the outlier was detected is updated to "outlier", and then the process proceeds to step S1760. For example, in the examples shown in FIGS. 7A to 7C, each record of customer names A1 to A3, B1, C1, and C2 is detected as an outlier by outlier detection processing 1 (1A, 1B). , the process advances from step S1740 to step S1750 to update the outlier detection result of each record to "outlier".

To explain the update timing in more detail, each record of customer names A1 to A3 is a processing loop in which the objective variable is the call time (=objective variable (denominator)), and is processed in housewife group units. Through the processing of steps S1740 and S1750 in the loop, the outlier detection result regarding call duration is updated to "outlier". The record of customer name B1 is a processing loop in which the objective variable is the call duration (=objective variable (denominator)), and is processed in steps S1740 and S1750 in the processing loop for each office worker group. The outlier detection result is updated to "outlier". Each record of customer names C1 and C2 is processed in steps S1740 and S1750 in a processing loop in which the objective variable is sales (=objective variable (numerator)) and in the processing loop for each part-time job group. The outlier detection result for high is updated to "outlier".

In step S1760, it is determined whether or not there is a record for which an outlier has been detected in outlier detection processing 2. If there is (yes), the process proceeds to step S1770, and if not (no), the process proceeds to step S1780. In step S1770, the outlier detection result of the record in which the outlier was detected is updated to "general error", and then the process proceeds to step S1780. For example, in the examples shown in FIGS. 7A to 7C, the records of customer names B1 and C3 are detected as outliers in the outlier detection process 2, so the process proceeds from step S1760 to step S1770, and each record Update outlier detection result to "common typo". Note that the outlier detection result for the record of customer name B1 is "outlier" in step S1750, but is updated again to "general error" in step S1770.

In step S1780, it is determined whether or not the data (call time, sales) of each record includes data outside the defined range or data measured during the period when there was trouble with the measuring equipment. The process proceeds to step S1790, and if not (no), the process proceeds to step S1800. In step S1790, the outlier detection result of the corresponding record is updated to "error due to measurement error", and then the process proceeds to step S1800. For example, if 3600 seconds is specified as the domain for call duration, data of 4800 seconds is determined to be "error due to measurement error". In addition, if there is remark information indicating that it was taken during a period when there was trouble with the measuring equipment, it is unconditionally determined to be "error due to measurement error".

In the example shown in FIGS. 7A to 7C, when the processing from step S1730 to step S1790 regarding the group of housewives is performed, the outlier detection result for each record of customer names A1 to A3 is Data related to time are considered "outliers". Further, when the processing from step S1730 to step S1790 regarding the group of office workers is performed, the outlier detection result for the record of customer name B1 is that the data regarding call duration is "general error". Furthermore, when the processing from step S1730 to step S1790 regarding the group of part-time workers is performed, the outlier detection results for the records of customer names C1 and C2 are such that the data regarding the sales amount is an "outlier", and the data regarding customer name C3 is an "outlier". Outlier detection results for the record include data on sales as "common typos." Step S1800 is a routine for outlier cause classification, and the details of the processing are shown in FIG.

(Detailed description of outlier cause classification processing)
In step S1802 in FIG. 6, the digit correction coefficient is obtained from the setting information DB2. The digit correction coefficient is used to correct the ^digit of the data value, ^that is, the position of the decimal point by multiplying the data determined to ^be erroneous ^. Numerical values in 10-fold increments are stored in advance in the setting information DB2 as digit correction coefficients. In step S1804, a multiplication process of multiplying the data value determined as "general error" by the digit correction coefficient is performed for each of the plurality of acquired digit correction coefficients.

In step S1806, it is determined whether or not any of the multiple values after multiplication is within the frequent appearance range of data values that are not errors or outliers in the same group. As an example of the frequency range here, for example, there is a range from "quartile point" to "third quartile point". If the determination in step S1806 is yes, the process advances to step S1808, and if the determination is no, step S1808 is skipped. In step S1808, the outlier detection result is updated from "general error" to "error due to digit error".

Returning to FIG. 5, in step S1810, the outlier detection processing loop for each group is completed. Then, when the outlier detection loop for each group is completed for all groups, the process proceeds to step S1820. In step S1820, a series of outlier detection results are stored in the outlier information DB3. If the outlier detection result is "error due to digit error", the digit correction coefficient at that time is stored in the outlier information DB3 together with the outlier detection result.

After completing the process of FIG. 5, the process returns to FIG. 4 and proceeds to step S180. In step S180, the outlier detection processing loop for each objective variable is completed. Then, when the outlier detection processing loop is completed for all objective variables, ie, the objective variable (denominator), the objective variable (numerator), and the objective variable (ratio), the series of processes in FIG. 4 ends.

Note that the outlier cause estimation result may be displayed on a monitor provided in the output device 5700, which will be described later. FIG. 8 shows an example of a screen display of outlier cause estimation results. Fields enclosed in dashed rectangles are the target variables that were identified as outliers and typos.

<Detailed Description of Learning Model Creation Processing in Learning Unit 20>
Next, the details of the learning model creation process executed in step S20 of FIG. 3 will be described using the flowchart of FIG. In the flowchart of FIG. 9, the processing from step S200 to step S240 is performed for each objective variable unit of objective variable (denominator), objective variable (numerator) and objective variable (ratio). In the learning model creation process, when an outlier or an error is detected in the target variable by the outlier detection process in step S10, preprocessing and loss function are performed when the learning model is created based on the obtained cause estimation result. Make settings. On the other hand, if neither an outlier nor an error is detected in the objective variable in the outlier detection process, normal learning is performed. In each learning, as in normal AutoML, multiple learning algorithms may be tried, and hyperparameters may be optimized in learning for each algorithm to be tried.

In step S200, a model creation processing loop for each objective variable is started. In step S210, it is checked whether or not the outlier information DB3 contains information about an error in the objective variable or an outlier. Step S220 is a routine relating to machine learning processing according to the content of errors and outliers, and details of the processing are shown in FIG. In addition, in the machine learning process according to the contents of the clerical error and the outlier in step S220, if the information of the clerical error of the objective variable or the outlier is not registered in the outlier information DB 3, learning is performed as usual. .

(Detailed explanation of machine learning processing according to the content of errors and outliers)
FIG. 10 is a flow chart showing the details of the machine learning process according to the content of errors and outliers. In step S2200, information on outliers, writing errors, and digit correction coefficients for each objective variable is obtained from the outlier information DB3. In step S2210, the value of the objective variable "error due to wrong digit" is multiplied by the corresponding digit correction coefficient to correct the value of the objective variable. For example, in the record of the customer name C3 in FIG. 8, the sales value of "30 yen" is determined to be a digit error, and "10 ² " is registered in the outlier information DB 3 as the digit correction coefficient. . Therefore, in step S2210, the value "30 yen" is corrected to the value "3,000 yen" obtained by multiplying "10 ² ".

In step S2210, it is determined whether or not there is a record determined to be an "outlier" in the objective variable. If yes (yes), the process proceeds to step S2230; otherwise (no), the process proceeds to step S2240. move on. If there are non-error outliers, perform robust regression. Robust regression is necessary for the following reasons.
• Some learning algorithms based on linear regression, tree models, and neural nets can be run by replacing the loss function to be optimized.
・One example of the loss function is the mean squared error, but learning to optimize this results in learning that preferentially reduces the predicted actual measurement error of outliers. As a result, the prediction measurement error of values that are not outliers is not sufficiently small, and a model that does not sufficiently predict values other than outliers is generated.
・For this reason, there is a method that makes it easier to generate a model that accurately predicts values that are not outliers, such as by using a loss function that is less likely to increase due to the presence of outliers. .

Specific examples of loss functions used in robust regression include "Huber-loss", "pseudo-Huber-loss", and "τ-quantile loss". In step S2230, the type of loss function for robust regression to be executed in the model creation process is acquired from the setting information DB2, and then the process proceeds to step S2240.

In step S2240, it is determined whether or not there is a record determined to be a clerical error other than "error due to digit error" in the objective variable. If yes (yes), the process proceeds to step S2250; to step S2260. In step S2250, the type of missing value interpolation to be performed in the model creation process is obtained from the setting information DB2, and then the process proceeds to step S2260.

If there are "general errors" and "errors due to measurement errors" other than "errors due to incorrect digits", the target variable is regarded as a missing value, and the missing value imputation method as a countermeasure is set to 1. Try one or more to create a learning model. Missing value imputation methods include, for example, a method of excluding from data to be learned, a method of complementing with another value, a method of applying semi-supervised learning, and the like. As a method of interpolating with another value, for example, there is a method of interpolating with a median value, an average value, or a separately specified value. In this case, random noise may be added to the interpolated value. There are also interpolation methods using matrix decomposition and interpolation methods that apply algorithms that can be used for regression such as missForest.

In step S2260, the setting information DB2 is referenced, and the loss function used when robust regression is not performed is acquired from the setting information DB2. In step S2270, machine learning is performed according to the combination of the series of loss function types and missing value imputation types obtained as described above. For example, if there are three types of loss functions and missing value imputations, respectively, there are nine (=3×3) combinations of loss functions and missing value imputations. Then, in step S2270, machine learning is performed for each combination setting. It should be noted that normal learning without "outliers" is also included as a combination of loss function (=normal squared error)×missing value interpolation (=no interpolation). In step S2280, the model obtained by a series of learning is stored in model information DB4. As can be seen from FIG. 9, the machine learning in step S2270 is performed for each of three target variables (numerator), target variable (denominator), and target variable (ratio).

Returning to FIG. 9, in step S230, the models obtained by each learning are evaluated using common test data, and the model with the best accuracy is stored in the model information DB4. In step S240, when the learning process loop for each objective variable is completed for each objective variable, the series of processes related to model creation shown in FIG. 9 ends.

<Detailed description of learning model evaluation processing>
Details of the learning model evaluation process in step S30 of FIG. 3 will be described with reference to the flowchart of FIG. In step S300, evaluation data is obtained from the learning data DB1. In step S310, the best-precision model for each of the objective variable (numerator), objective variable (denominator), and objective variable (ratio) stored in the model information DB4 in step S230 is obtained from the model information DB4. Here, each acquired model is called a numerator prediction model, a denominator prediction model, and a ratio prediction model.

In step S320, the evaluation data acquired in step S300 are used to estimate the values of the numerator, denominator, and ratio from the numerator prediction model, denominator prediction model, and ratio prediction model, respectively. In step S330, the accuracy of (numerator/denominator) (=accuracy 1) and the accuracy of ratio (=accuracy 2) are compared to determine whether or not "accuracy 1>accuracy 2". If it is determined in step S330 that "accuracy 1>accuracy 2", the process proceeds to step S340, adopts the numerator prediction model and the denominator prediction model as the best models, and saves the decision results in the model information DB4. On the other hand, if it is determined in step S330 that "accuracy 1>accuracy 2" does not hold, the process proceeds to step S350, adopts the ratio prediction model as the best model, and saves the determination result in the model information DB4.

In the above-described embodiment, the case where the objective variable is expressed as a ratio has been described. A two-step outlier detection process shown in 4 to 6 can be applied. The target variable in the processing from step S100 to step S180 in the flowchart of FIG. 5 is only the sales amount, and with respect to the target variable=sales amount, two-step outlier detection processing shown in FIGS. 4 to 6 is performed. 9 and 10 following the outlier detection process and the learning model evaluation process of FIG. 11 are also performed with sales as the objective variable. That is, the two-stage outlier detection process, learning model creation process, and learning model evaluation process in the present embodiment can be applied to both cases in which the objective variable is a ratio and a case in which it is not a ratio. A highly accurate learning model can be constructed.

<Computer Realizing Machine Learning System 1>
FIG. 12 is a hardware diagram of a computer 5000 that implements the machine learning system 1. As shown in FIG. A computer 5000 that realizes the machine learning system 1 includes a processor 5300 represented by a CPU (Central Processing Unit), a memory 5400 such as a RAM (Random Access Memory), an input device 5600 (for example, a keyboard, a mouse, a touch panel, etc.), and an output Devices 5700 (eg, a video graphics card connected to an external display monitor) are interconnected via memory controller 5500 .

In the computer 5000, a program for realizing a machine learning system is read from an external storage device 5800 such as an SSD or HDD via an I/O (Input/Output) controller 5200, and cooperation between a processor 5300 and a memory 5400 is performed. Executed by Thereby, the machine learning system 1 is realized. Alternatively, the program for realizing the machine learning system 1 may be acquired from an external computer through communication via the network interface 5100, or read from a recording medium by a medium reader.

According to the embodiment of the present invention described above, the following effects are obtained.

(C1) As shown in FIGS. 1 to 8, the machine learning system 1 includes an outlier detection unit 10 and a learning unit 20. FIG. The outlier detection unit 10 performs a first outlier detection process on all the records of the learning data, with the objective variable, which is the prediction target of machine learning, as the detection target. For example, if the target variable is sales, a first outlier detection process is performed with sales as the detection target. Further, the outlier detection unit 10 divides all the records R1, R2, R3, . Perform group processing to divide into Furthermore, the outlier detection unit 10 performs a second outlier detection process for each of the plurality of groups, with the sales amount as the detection target. Furthermore, the outlier detection unit 10 estimates the cause of the outlier based on the detection results of the first and second outlier detection processes. Then, the learning unit 20 performs machine learning based on the result of estimating the cause of the outlier to create a learning model for predicting sales.

As described above, the first outlier detection process is performed on all records of the learning data with the sales amount as the detection target, and the second outlier detection process is performed with the sales amount as the detection target for each group of grouped records. Based on these two-step outlier detection processing, the cause of the outlier is estimated. As a result, outlier cause estimation is improved, and appropriate machine learning can be performed according to the cause of the outlier.

(C2) As shown in FIGS. 1 to 8, the objective variable is expressed as a ratio between the first variable (sales) and the second variable (call time) included in the learning data records R1 to R3. In this case, the outlier detection unit 10 performs the first outlier detection process for detecting any one of the objective variable (ratio), the sales amount, and the call time for all records of the learning data. each time. Furthermore, the outlier detection unit 10 performs a second outlier detection process for each detection target of any one of the objective variable (ratio), the sales amount, and the call time for each of the plurality of groups. Do each. Then, the learning unit 20 performs machine learning based on the results of estimating the cause of the outliers, and creates a learning model for predicting the ratio of sales to call duration.

As described above, the first outlier detection process is performed with the ratio, the numerator of the ratio, and the denominator of the ratio for all records of the learning data as objective variables, and the ratio and the numerator of the ratio are performed for each grouped record group. and the denominator of the ratio as objective variables, the second outlier detection process is performed, and the cause of the outlier is estimated based on these two-step outlier detection processes. As a result, the causes of outliers can be found for each of the ratio, the numerator of the ratio, and the denominator of the ratio, and appropriate machine learning can be performed according to the cause of the outliers.

(C3) As shown in FIG. 5, in estimating the cause of an outlier, the outlier detection unit 10 estimates the detection target detected by the first outlier detection process as an outlier, A detection target detected by the detection process is presumed to be a writing error. By estimating the cause of such outliers, it is possible to distinguish between true outliers and typographical errors, even when some outliers appear mathematically. Machine learning becomes possible.

(C4) As shown in FIGS. 6 and 8, in estimating the cause of an outlier, the outlier detection unit 10 multiplies a plurality of digit correction coefficients of different magnitudes by detection targets that are estimated to be erroneous, If the multiplied value is within the frequency range of the same detection target that is not determined to be an outlier in the same group, it is assumed to be a writing error due to a digit error.

For ^{example ,} in ^the record whose customer name ^is C3 in FIG. When multiplied by a plurality of digit correction coefficients in 10-fold increments, 3,000 yen when multiplied by ¹⁰² falls within the numerical range of sales that is not determined as an outlier. That is, 30 yen is presumed to be a clerical error due to an order of magnitude difference, and it is possible to determine that the cause is not just a clerical error but a clerical error due to an order of magnitude difference.

(C5) As shown in FIGS. 8 and 10, the outlier detection unit 10 determines that the multiplication result is within the frequent appearance range for the detection target (for example, the sales amount of 30 yen in FIG. 30 yen is corrected to 3,000 yen by multiplying by a digit correction coefficient (=10 ² ) (step S2210). Then, the learning unit 20 performs machine learning using the corrected sales amount of 3,000 yen instead of the sales amount of 30 yen, which is the detection target and is estimated to be a writing error due to a digit error. As a result, it is possible to improve accuracy compared to the case where writing errors due to digit errors are not corrected.

(C6) As shown in FIG. 10, when there is a detection target that is estimated to be an outlier, the learning unit 20 performs machine learning using robust regression to create a learning model. If there are outliers, it is not possible to ensure sufficient prediction accuracy with normal machine learning that uses the mean squared error as a loss function, but as described above, machine learning using robust regression improves prediction accuracy. can be achieved.

(C7) As shown in FIGS. 10 and 11, the learning unit 20 creates a numerator prediction model that predicts the target variable (numerator) that is sales, and a denominator prediction model that predicts the target variable (denominator) that is call duration. , and a ratio prediction model that predicts the target variable (ratio). Then, the machine learning system 1 obtains predicted values of the numerator prediction model, the denominator prediction model, and the ratio prediction model using the evaluation data, and calculates the accuracy of the ratio between the prediction value of the numerator prediction model and the prediction value of the denominator prediction model. , and the accuracy of the prediction values of the specific prediction models, and adopts the prediction model with the higher accuracy as the learning model.

Thus, in addition to the ratio prediction model, the numerator and denominator prediction models are also considered, and the accuracy of the ratio prediction model and the ratio of the prediction values of the numerator and denominator prediction models are calculated. Since the learning model is determined from the comparison of , a more highly accurate learning model can be obtained.

(C8) As shown in FIGS. 1 to 8, the machine learning method of the present invention performs a first outlier detection process on all records of the learning data, with the objective variable, which is the prediction target of machine learning, as the detection target. , all records R1, R2, R3, . Then, a second outlier detection process is performed with the objective variable as a detection target, the cause of the outlier is estimated based on the detection results of the first outlier detection process and the second outlier detection process, and the estimated cause of the outlier is Perform machine learning based on the sources of the outliers to create a learning model that predicts the target variable.

Each embodiment described above is merely an example, and the present invention is not limited to these contents as long as the features of the invention are not impaired. Other aspects conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention. Moreover, the above embodiments and modifications can be combined in whole or in part without departing from the gist of the present invention and within a mutually compatible range.

DESCRIPTION OF SYMBOLS 1... Machine learning system 10... Outlier detection part 20... Learning part 30... Model evaluation part 5000... Computer 5700... Output device DB1... Learning data database DB2... Setting information database DB3... Outlier information Database, DB4... Model information database

Claims (8)

a first outlier detection unit that performs a first outlier detection process for all records of learning data, with an objective variable that is a prediction target of machine learning as a detection target;
a group processing unit that divides all the records into a plurality of groups based on explanatory variables included in the records;
a second outlier detection unit that performs a second outlier detection process with the objective variable as a detection target for each of the groups;
a cause estimating unit that estimates the cause of the outlier based on the detection results of the first outlier detecting unit and the second outlier detecting unit;
A machine learning system comprising: a model creating unit that creates a learning model for predicting the objective variable by performing machine learning based on the estimation result of the cause estimating unit. In the machine learning system of claim 1,
The objective variable is expressed as a ratio of a first variable and a second variable included in the learning data record,
The first outlier detection unit performs the first outlier detection for detecting any one of the objective variable, the first variable, and the second variable for all records of the learning data. performing processing for each of the detection targets,
The second outlier detection unit performs a second outlier detection process for detecting any one of the objective variable, the first variable, and the second variable for each of the groups. Do it for each target,
The machine learning system, wherein the model creating unit performs machine learning based on the estimation result of the cause estimating unit to create a learning model for predicting the ratio between the first variable and the second variable. In the machine learning system of claim 1,
The cause estimation unit estimates the detection target detected by the first outlier detection unit as an outlier, and estimates the detection target detected by the second outlier detection unit as an error. learning system. In the machine learning system according to claim 3,
The cause estimating unit multiplies a plurality of digit correction coefficients of different magnitudes by the detection target estimated to be the erroneous writing, and the multiplied values are not determined to be outliers of the records in the same group. A machine learning system that presumes that there is a typographical error due to a digit mistake when the same detection target is within the frequent occurrence range. In the machine learning system according to claim 4,
A correction unit that multiplies the detection target estimated to be a writing error due to the digit error by the digit correction coefficient whose multiplication result is within the frequency range to correct the detection target,
The machine learning system, wherein the model creation unit performs machine learning using the detection target corrected by the correction unit instead of the detection target estimated to be a writing error due to the digit error. In the machine learning system according to claim 3,
The machine learning system, wherein the model creation unit creates a learning model by performing machine learning based on robust regression when there is a detection target estimated to be an outlier by the cause estimation unit. In the machine learning system according to claim 2,
The model creation unit creates a first prediction model for predicting the first variable, a second prediction model for predicting the second variable, and a third prediction model for predicting the objective variable. death,
Predicted values of the first prediction model, the second prediction model, and the third prediction model are obtained using the evaluation data, and the prediction values of the first prediction model and the predictions of the second prediction model are obtained. machine learning, further comprising a model evaluation unit that compares the accuracy of the ratio with the value and the accuracy of the predicted value of the third prediction model, and adopts the prediction model with higher accuracy as the learning model. system. performing a first outlier detection process for all records of the learning data, with the target variable, which is the prediction target of machine learning, as the detection target;
dividing all the records into a plurality of groups based on explanatory variables contained in the records;
performing a second outlier detection process with the objective variable as a detection target for each of the groups;
estimating the cause of the outlier based on the detection results of the first outlier detecting unit and the second outlier detecting unit;
A machine learning method, wherein machine learning is performed based on the estimated causes of the outliers to create a learning model that predicts the objective variable.

Images