JP2021093020A - Information processing apparatus, information processing method, and program - Google Patents

Abstract

To efficiently extract data giving harmful affection and good affection on prediction accuracy.SOLUTION: An information processing apparatus according to an embodiment of the present invention has an acquiring unit, a learning unit, and an output control unit. The acquiring unit acquires an input data for use in input to a model for outputting an inference result based on the input of an input data including time-series data having values continuously changed in response to a position and a correct answer data indicating a correct answer of the inference by the model. The learning unit learns a model by using first input data selected from the input data and the correct answer data. The output control unit outputs a degree of contribution of the first input data to the inference result of the learned model. The learning unit further learns the model by using third input data based on second input data designated in response to the output degree of contribution of the input data and the correct answer model.SELECTED DRAWING: Figure 1

Description

Embodiments of the present invention relate to information processing devices, information processing methods and programs.

A technique for predicting the demand for electricity in a designated area from meteorological data has been proposed by using a model created by machine learning for each area where electricity is used. As the meteorological data, data provided by the Japan Meteorological Agency can be used for the area where the facility such as the meteorological station is installed. In addition, a technique for predicting meteorological data in areas where meteorological data is not provided has been proposed from meteorological data provided to a limited area. By using the predicted meteorological data, it is possible to realize demand forecasting in a wide range of areas with higher accuracy.

In the above prediction technology, it is desirable to extract data that has a bad influence on the accuracy of prediction and data that has a good influence, and to use data that has a good influence without using data that has a bad influence. ..

JP-A-2019-087027

However, in the prior art, there are cases where data having a bad influence on the accuracy of prediction and data having a good influence cannot be efficiently extracted.

The information processing device of the embodiment includes an acquisition unit, a learning unit, and an output control unit. The acquisition unit inputs input data including time-series data whose value changes continuously according to the position and outputs the inference result. Input data to be input to the model and correct answer data indicating the correct answer of the inference by the model are input. get. The learning unit learns the model using the first input data selected from the input data and the correct answer data. The output control unit outputs the contribution of the first input data to the inference result by the learned model. The learning unit further learns the model using the third input data based on the second input data specified according to the output contribution of the input data and the correct answer data.

FIG. 1 is a block diagram showing an example of the configuration of the information processing apparatus according to the embodiment. FIG. 2 is a diagram showing an example of a data structure of event data. FIG. 3 is a diagram showing an example of items representing the weather that can be included in the weather data. FIG. 4 is a diagram showing an example of a data structure of geographic data. FIG. 5 is a diagram showing an example of a data structure of event data. FIG. 6 is a diagram showing an example of a data structure of additional data. FIG. 7 is a flowchart showing an example of the learning process in the embodiment. FIG. 8 is a flowchart showing an example of inference processing in the embodiment. FIG. 9 is a flowchart showing an example of the explanatory variable extraction process. FIG. 10 is a diagram for explaining an example of a method of creating the frequency distribution R1. FIG. 11 is a diagram for explaining an example of a method of creating the frequency distribution R1. FIG. 12 is a diagram for explaining an example of a method of creating the frequency distribution R1. FIG. 13 is a diagram showing an example of the frequency distribution R2. FIG. 14 is a flowchart showing another example of the explanatory variable extraction process. FIG. 15 is a diagram showing an example of a display screen for displaying the inference result. FIG. 16 is a diagram showing an example of a display screen for displaying the inference result. FIG. 17 is a diagram showing an example of a display screen for displaying the inference result. FIG. 18 is a diagram showing an example of a display screen for displaying the inference result. FIG. 19 is a diagram showing an example of a display screen for displaying the inference result. FIG. 20 is an explanatory diagram showing a hardware configuration example of the information processing apparatus according to the embodiment.

A preferred embodiment of the information processing apparatus according to the present invention will be described in detail below with reference to the accompanying drawings.

In the following, an information system that infers (predicts) the occurrence of a certain event from input data including meteorological data (including predicted meteorological data) will be described as an example. The input data is not limited to meteorological data. Other time-series data whose value continuously changes according to the position (position on the map, etc.) may be used as input data. The inference object may be any object. For example, the inference target may be whether or not an event occurs, or may be the amount of occurrence of an event.

The above information processing system can be interpreted as a system that infers (predicts) the objective variable from one or more explanatory variables. In the case of power demand forecasting, meteorological data corresponds to the explanatory variable and power demand corresponds to the objective variable.

The higher the relationship between the event data (event data) that serves as the objective variable and the meteorological data that serves as the explanatory variable, the higher the prediction accuracy of the machine learning model. On the other hand, when the time-series regional trends are taken into consideration, the weather data of the area to be predicted and the area far away may adversely affect the prediction accuracy. In the past, knowledge about what kind of data influences forecasts has been accumulated as an analyst's rule of thumb, and demand forecasting work has sometimes been personalized.

In the present embodiment, among a large number of explanatory variables (such as meteorological data), the explanatory variables that affect the accuracy of prediction are visualized, and the desired explanatory variables can be excluded based on a quantitative index, and the analyst. It is possible to specify explanatory variables that have a positive effect on prediction accuracy, based on the empirical rules of.

For example, when forecasting power demand for the entire area of Japan, the amount of data is extremely large, making it difficult for analysts to analyze manually, and even when using a computer, the calculation cost is extremely high. Will be expensive. According to this embodiment, the types and attributes of explanatory variables that affect the accuracy of inference (prediction) by the machine learning model are visualized, and the explanatory variables that have a positive effect on the prediction accuracy are specified in consideration of the conventional knowledge ( (Selection) is possible. Therefore, the calculation cost can be reduced.

FIG. 1 is a block diagram showing an example of the configuration of the information processing apparatus 100 according to the present embodiment. As shown in FIG. 1, the information processing apparatus 100 includes an acquisition unit 101, a coding unit 102, an extraction unit 103, a learning unit 104, an output control unit 105, an inference unit 111, a display unit 131, and the like. It includes an event data storage unit 121, a weather data storage unit 122, an additional data storage unit 123, a geographic data storage unit 124, a feature amount storage unit 125, a model storage unit 126, and an extraction information storage unit 127. ing.

The acquisition unit 101 acquires various data used in various processes executed by the information processing apparatus 100. For example, the acquisition unit 101 acquires input data, correct answer data, geographic data, and the like to be input to the machine learning model used for inference.

The acquisition method of each data by the acquisition unit 101 may be any method. For example, a method of acquiring data from an external device via a network, a method of reading data stored in a storage medium, and the like can be applied. The network is a LAN (local area network), the Internet, or the like, but may be any other network. The network may be either a wired network or a wireless network. The acquisition method may be changed according to the data to be acquired. For example, the acquisition unit 101 may be configured to acquire event data from the server device, acquire the weather data from the weather prediction system that provides the weather data, and acquire additional data (described later) from the web scraping system.

In the following, a case where the input data includes meteorological data as time-series data whose value continuously changes according to the position will be described as an example. The input data may include any data that may affect the inference, in addition to meteorological data and the like. Data used in addition to meteorological data may be referred to as additional data below. Input data (weather data, additional data) correspond to explanatory variables.

The correct answer data is data used when learning a machine learning model, and is data representing the correct answer of inference by the machine learning model. For example, data representing the time and position where the event to be predicted occurred in the past can be the correct answer data. In the following, the correct answer data may be referred to as event data.

Geographical data is data that represents the location of an area where an event can occur. Examples of data structures for meteorological data, event data, and geographic data will be described later.

The coding unit 102 encodes (encodes) the input data (input data, correct answer data, etc.) and outputs the feature amount that is the result of the coding. It can be interpreted that the coding unit 102 performs a process of converting the input data into a format that is easy to use in the subsequent process. For example, when using temporally sparse data, it is necessary to convert this data into temporally continuous data so that temporally continuous events can be predicted. The coding unit 102 encodes the data by, for example, the following method.
-One-hot Encoding: The category value of an arbitrary event is represented by data (features) in a format that can be easily interpreted by a machine learning model.
-Count Encoding: The feature quantity is the number of occurrences of a category within an arbitrary period.
-Consolidation Encoding: Coding into features while eliminating items such as notational fluctuations that exist in the data.
-Interaction: The relationship between features is used as a new feature.
-Trend lines Data sets are processed into arbitrary time-series trend data, and trend values are expressed as features.

The coding method is not limited to the above. The coding unit 102 may use the above-mentioned plurality of methods in combination.

The extraction unit 103 extracts explanatory variables used in the learning process of the machine learning model and the inference process by the machine learning model from the plurality of explanatory variables (input data encoded by the feature amount). For example, the extraction unit 103 extracts an explanatory variable that is more correlated with the objective variable (event data) from the plurality of explanatory variables.

The learning unit 104 learns a machine learning model. The machine learning model is a model that inputs input data (weather data, etc.) and outputs inference results (event occurrence, etc.). The machine learning model may be of any type, and for example, a model such as a random forest, a binary tree, and a neural network can be applied. The learning unit 104 may execute the learning process by any learning method used in the machine learning model to be applied. For example, the learning unit 104 learns a machine learning model using explanatory variables (first input data) selected (extracted) from a plurality of explanatory variables and event data corresponding to correct answer data.

The inference unit 111 executes inference by a machine learning model. For example, the inference unit 111 newly inputs input data to the machine learning model learned by the learning unit 104 and executes inference. The input data used for inference is, for example, the same explanatory variable as the explanatory variable selected (extracted) at the time of learning from among a plurality of explanatory variables (input data encoded by the feature amount).

The output control unit 105 controls the output of data to an output device such as the display unit 131. For example, the output control unit 105 causes the display unit 131 to display the inference result by the machine learning model learned by the learning unit 104. In the present embodiment, the output control unit 105 controls the process of visualizing the explanatory variables that contribute to the inference by the learned machine learning model at the time of learning by the learning unit 104. For example, the output control unit 105 displays the contribution of each explanatory variable to the inference result by the learned machine learning model on the display unit 131.

As the contribution output method, various methods can be applied depending on the machine learning model used. When a decision tree is used as a machine learning model, the output control unit 105 can apply a visualization method called dtreeviz. dtreeviz is an OSS (Open Source Software) library that enables visualization of features inside decision trees. By visualizing the degree of contribution, it is possible to confirm how a certain explanatory variable behaves in the machine learning model and contribute to the prediction result, and to clarify the explanatory variable that has a peculiar influence. It will be possible.

The user can further specify (select) the explanatory variables used for learning by referring to the displayed contribution. The extraction unit 103 further extracts the explanatory variable (second input data) designated in this way and other explanatory variables based on the designated explanatory variable. Further, the learning unit 104 repeats the process of learning the machine learning model using the extracted explanatory variables and the event data. By such processing, it becomes possible to efficiently extract data having a bad influence on the accuracy of prediction and data having a good influence.

The display unit 131 is a display device such as a liquid crystal display that displays data. The display unit 131 displays, for example, the inference result by the machine learning model according to the control of the output control unit 105.

The event data storage unit 121 stores, for example, the event data acquired by the acquisition unit 101. FIG. 2 is a diagram showing an example of a data structure of event data. As shown in FIG. 2, the event data includes an ID, an occurrence date and time, a latitude, and a longitude. The ID is information that identifies event data. The date and time of occurrence represents the date and time when the event occurred (year, month, day, time, etc.). Latitude and longitude are information for identifying the position where the event occurred. Event data may include other information that can identify where the event occurred, instead of latitude and longitude. For example, the name of the area where the event occurred (such as the city name) and the name of the facility where the event occurred are other examples of information that can identify the location where the event occurred.

FIG. 2 is an example of event data showing whether or not a certain event has occurred. Event data including the amount of occurrence of a certain event (for example, when predicting power demand, the amount of power demand generated) may be used.

Returning to FIG. 1, the meteorological data storage unit 122 stores, for example, the meteorological data acquired by the acquisition unit 101. Meteorological data includes values for meteorological items such as temperature, wind speed, and precipitation, for example by region and by date and time. As the meteorological data, data provided by the Japan Meteorological Agency or the like may be used, or meteorological data predicted from the provided data may be used. By using the predicted meteorological data, it is possible to avoid problems such as inability to make accurate predictions in areas where the meteorological data is sparse, and to execute predictions with higher accuracy in a wider area.

FIG. 3 is a diagram showing an example of items related to the weather that can be included in the weather data. As shown in FIG. 3, meteorological data may include not only commonly known items such as temperature, wind speed, and precipitation, but also many other items. According to the present embodiment, among these items, it is possible to efficiently find an item that has a positive effect on inference by a machine learning model.

Returning to FIG. 1, the additional data storage unit 123 stores, for example, the additional data acquired by the acquisition unit 101. As described above, the additional data is data that can be added as input data in addition to the meteorological data. No additional data is required. The additional data may be any data and data structure. For example, the presence or absence of events such as long holidays (Golden Week, Silver Week, Obon holidays, year-end and New Year holidays, etc.) can be used as additional data.

The geographic data storage unit 124 stores, for example, the geographic data acquired by the acquisition unit 101. FIG. 4 is a diagram showing an example of a data structure of geographic data. The geographical data of FIG. 4 is data that defines position information such as latitude and longitude for each region to be predicted. As shown in FIG. 4, the geographical data includes a prefecture code, an ID, a latitude, a longitude, and a region name. The prefecture code is information that identifies the prefecture in Japan. The ID is information that identifies the area. The area name represents the name of the area.

The geographic data is referred to, for example, when the output control unit 105 displays the inference result on the map, and when the coding unit 102 encodes the data into regional data. FIG. 5 is a diagram showing an example of the data structure of the event data after being encoded into the feature amount. FIG. 5 shows, for example, an example of a feature amount in which event data indicating the occurrence of an event represented by latitude and longitude as shown in FIG. 2 is encoded so as to represent the number of occurrences of an event in each region. As shown in FIG. 5, the coded event data (feature amount) includes the number of occurrences of events for each date and each region.

Returning to FIG. 1, the feature amount storage unit 125 stores, for example, the feature amount encoded by the coding unit 102. For example, the feature amount storage unit 125 stores the coded event data described in FIG. 5 and the coded additional data as shown in FIG. FIG. 6 is a diagram showing an example of a data structure of additional data after being encoded into a feature amount.

FIG. 6 shows, for example, when additional data indicating the period (date range) of a vacation (an example of an event) is acquired, this additional data is used for whether or not an event occurs for each region and for each date (1: Occurs). , 0: does not occur) is shown as an example of coding.

Returning to FIG. 1, the model storage unit 126 stores information representing the machine learning model.

The extraction information storage unit 127 stores the extraction information indicating the conditions for extracting the explanatory variables used for inference from the plurality of explanatory variables. For example, the extraction information storage unit 127 stores information for specifying an explanatory variable extracted when learning a machine learning model capable of predicting with higher accuracy as extraction information.

Each of the above units (acquisition unit 101, coding unit 102, extraction unit 103, learning unit 104, output control unit 105, and inference unit 111) is realized by, for example, one or more processors. For example, each of the above parts may be realized by causing a processor such as a CPU (Central Processing Unit) to execute a program, that is, by software. Each of the above parts may be realized by a processor such as a dedicated IC (Integrated Circuit), that is, hardware. Each of the above parts may be realized by using software and hardware in combination. When a plurality of processors are used, each processor may realize one of each part, or may realize two or more of each part.

Each of the above storage units (event data storage unit 121, meteorological data storage unit 122, additional data storage unit 123, geographic data storage unit 124, feature quantity storage unit 125, model storage unit 126, extraction information storage unit 127) is a flash memory. , Memory card, RAM (Random Access Memory), HDD (Hard Disk Drive), and any commonly used storage medium such as an optical disk. Each storage unit may be physically different storage media, or may be realized as different storage areas of physically the same storage medium. Further, each of the storage units may be realized by a plurality of physically different storage media. Each of the storage units may be realized by a plurality of physically different storage media.

Note that FIG. 1 shows an example in which one information processing device 100 includes a function for executing learning processing and a function for executing inference processing, but the two functions are configured to be executed by different devices. You may. The information processing device 100 may be a device that operates in a cloud environment. Further, a part of each part shown in FIG. 1 may be configured to be executed by an external device of the information processing device 100. For example, the display unit 131 may be provided in a terminal device such as a personal computer, a smartphone, and a tablet, and the output control unit 105 may be configured to output information to the terminal device. The external device may be a device that operates in a cloud environment.

Next, the learning process of the machine learning model by the information processing apparatus 100 according to the present embodiment configured as described above will be described. FIG. 7 is a flowchart showing an example of the learning process in the present embodiment.

The acquisition unit 101 acquires event data, additional data, meteorological data, and geographic data (step S101). The acquisition unit 101 does not have to acquire additional data when it is not needed. The coding unit 102 executes a coding process using each of the acquired data (step S102). For example, the coding unit 102 encodes at least the correct answer data (event data) and the input data (weather data, additional data) into feature quantities. The input data encoded in the feature quantity is used as an explanatory variable. The event data encoded by the feature quantity is used as the objective variable.

Next, the explanatory variable extraction process is executed (step S103). In the explanatory variable extraction process, explanatory variables that contribute to improving the inference accuracy of the machine learning model are extracted from the acquired multiple explanatory variables (input data encoded by the features), and machine learning is performed using the extracted explanatory variables. This is the process of learning a model. The details of the explanatory variable extraction process will be described later.

The extraction unit 103 stores in the extraction information storage unit 127 the extraction information that identifies the explanatory variables extracted when the machine learning model capable of predicting with higher accuracy is learned in the explanatory variable extraction process. Further, the learning unit 104 stores information representing the learned machine learning model learned by the explanatory variable extraction process in the model storage unit 126 (step S104).

The output control unit 105 displays information about the trained machine learning model on, for example, the display unit 131 (step S105). For example, the output control unit 105 refers to the extracted information and displays information indicating the explanatory variables used for learning the machine learning model. The output control unit 105 may display information indicating the parameters of the machine learning model.

Next, the inference processing using the machine learning model by the information processing apparatus 100 according to the present embodiment will be described. FIG. 8 is a flowchart showing an example of inference processing in the present embodiment.

Since steps S201 and S202 are the same as steps S101 and S102 in FIG. 7, description thereof will be omitted.

The acquisition unit 101 reads the extracted information from the extracted information storage unit 127, and reads the learned machine learning model information from the model storage unit 126 (step S203). The extraction unit 103 extracts (selects) an explanatory variable used for inference from a plurality of encoded feature variables (explanatory variables) using the extracted extraction information (step S204).

The inference unit 111 executes the inference process by inputting encoded input data (weather data, additional data) into the read machine learning model (step S205). The output control unit 105 displays the inference result of the inference process on, for example, the display unit 131 (step S206). A specific example of how to display the inference result will be described later.

Next, the details of the explanatory variable extraction process in step S103 will be described. FIG. 9 is a flowchart showing an example of the explanatory variable extraction process.

The explanatory variable extraction process may be executed only when the event to be predicted correlates with the meteorological data. For example, the extraction unit 103 calculates a correlation coefficient between event data and meteorological data, and when the absolute value of the calculated correlation coefficient is equal to or greater than a threshold value (for example, 0.2), it is determined that there is a correlation between the two. judge. The extraction unit 103 sets an arbitrary unit time, and calculates the correlation coefficient by using the moving average of each data within this unit time. The unit time at which this correlation coefficient can be equal to or greater than the threshold value serves as a guideline for the time particle size for which the prediction is effective. For example, this unit time can also be used as the time particle size when calculating the correlation coefficient in the subsequent processing.

In the explanatory variable extraction process, first, the extraction unit 103 selects an explanatory variable to be used for learning from a plurality of explanatory variables (input data encoded by the feature amount) (step S301). For example, the extraction unit 103 randomly selects an appropriate number of explanatory variables from a plurality of explanatory variables using a uniform random number or the like.

Next, the learning unit 104 learns a machine learning model for inferring the input event data (objective variable) from the selected explanatory variable (step S302). The inference unit 111 executes an inference process using the learned machine learning model (step S303). In the inference process, input data different from the input data used in the learning process may be used.

In addition, the analyst's manual prediction using the selected explanatory variables may be executed in parallel. The prediction results by the analyst can be used as a reference when adding explanatory variables used for learning, as will be described later.

The output control unit 105 displays an explanatory variable having a high contribution to the inference result by the machine learning model on the display unit 131 by using a visualization method such as dtreeviz (step S304).

The user can specify the explanatory variables to be used for the next learning by referring to the displayed contribution. The user may specify the explanatory variables to be used for the next learning by referring to the prediction result by the analyst and the information of the explanatory variables on which the analyst bases the prediction. Instead of the user specifying the explanatory variables, the extraction unit 103 may extract a fixed number of explanatory variables (for example, the top 10%) in descending order of contribution.

Not only the specified (extracted) explanatory variable but also other explanatory variables that correlate with the specified explanatory variable may be configured to be used in the next learning. For example, the extraction unit 103 extracts another explanatory variable that correlates with the designated explanatory variable (step S305). For example, the extraction unit 103 compares the absolute value of the correlation coefficient between the two with the threshold value, and determines that the two are correlated when the absolute value of the correlation coefficient exceeds the threshold value. The explanatory variables specified and extracted in this way are used as the explanatory variables used below.

The extraction unit 103 further extracts explanatory variables similar to each explanatory variable included in the explanatory variable group based on the similarity of the relationship with the meteorological data.

First, the extraction unit 103 creates relational information (first relational information) representing the relation between each explanatory variable included in the explanatory variable group and the meteorological data. The relationship information is, for example, a frequency distribution R1 representing the number of times an index value representing the relationship between the explanatory variable and the meteorological data appears in each of a plurality of periods included in the designated period (step S306).

10 to 12 are diagrams for explaining an example of a method of creating the frequency distribution R1. FIG. 10 shows how the correlation coefficient (an example of an index value representing the relationship) between a certain explanatory variable and the meteorological data changes within a specified period. In FIG. 10, the horizontal axis represents time and the vertical axis represents the correlation coefficient. FIG. 11 shows an example of a frequency distribution representing the number of occurrences of the correlation coefficient within this designated period. FIG. 12 is an example in which the frequency distributions for 10 explanatory variables are superimposed. In FIG. 12, the frequency distribution for one explanatory variable is represented by one polygonal line.

Note that FIG. 12 corresponds to the explanatory variable group including 10 explanatory variables, but the number of explanatory variables included in the explanatory variable group is not limited to 10. On the other hand, when the number of explanatory variables is large (for example, when the number exceeds the threshold value), the extraction unit 103 performs clustering that integrates frequency distributions that are similar to each other into one distribution, and adjusts the number to an appropriate number. You may. When clustering is performed, the extraction unit 103 sets the representative vector of each cluster as the frequency distribution R1. The representative vector is, for example, a vector having the average value of the elements of each vector as an element.

Returning to FIG. 9, the extraction unit 103 creates relational information (second relational information) representing the relation between all the explanatory variable groups and the meteorological data. Similar to the above, the relationship information is, for example, a frequency distribution R2 representing the number of times an index value (for example, a correlation coefficient) representing the relationship between the explanatory variable and the meteorological data appears in each of a plurality of periods included in the designated period. (Step S307).

FIG. 13 is a diagram showing an example of the frequency distribution R2. FIG. 13 shows an example of the frequency distribution R2 created for all the explanatory variables by the same method as the frequency distribution R1.

Returning to FIG. 9, the extraction unit 103 specifies (extracts) an explanatory variable corresponding to the frequency distribution R2 that matches or is similar to the frequency distribution R1 as an explanatory variable (third input data) used for the next learning. For example, the extraction unit 103 specifies an explanatory variable corresponding to the frequency distribution R2 that approximates the frequency distribution R1 (step S308). The extraction unit 103 obtains the frequency distribution R2 having the closest distance (vector distance) to the frequency distribution R1 by using, for example, the k-nearest neighbor algorithm (KNN), and describes the frequency distribution R2 corresponding to the obtained frequency distribution R2. Identify the variable.

Further machine learning models are trained using the explanatory variables identified in this way. That is, the learning unit 104 learns a machine learning model for inferring the input event data (objective variable) from the specified explanatory variable (step S309). The inference unit 111 executes an inference process using the learned machine learning model (step S310).

The learning unit 104 determines whether or not the inference accuracy is improved (step S311). For example, the learning unit 104 determines whether or not the accuracy when inferred using the machine learning model in step S310 is greater than the accuracy of the prediction result by the analyst. Even if the learning unit 104 determines whether the accuracy is improved from the inference result at the time of the previous inference by the machine learning model (for example, step S303 or step S309 executed immediately before the case where step S310 is repeatedly executed). Good.

When the inference accuracy is improved (step S311: Yes), the explanatory variable extraction process is terminated. When the inference accuracy is not improved (step S311: No), the learning unit 104 determines whether or not the number of times of repeating the process (number of processes) from step S306 to step S311 has reached the upper limit value (step). S312). Instead of determining whether the number of processes has reached the upper limit, it may be configured to determine whether the end of the process is specified by the user. When the upper limit value has not been reached (step S312: No), the extraction unit 103 further adds an explanatory variable (step S313).

For example, when the analyst uses an explanatory variable that is not included in the currently used explanatory variable as the basis for prediction, the user specifies the explanatory variable to be added as an explanatory variable. The extraction unit 103 adds the specified explanatory variable. After that, the processes after step S306 are repeatedly executed for the explanatory variable group including the added explanatory variable.

When it is determined that the upper limit value has been reached (step S312: Yes), the extraction unit 103 selects an explanatory variable by yet another method. For example, the extraction unit 103 selects explanatory variables by a method called Backward Elimination (step S314). The learning unit 104 learns a machine learning model for inferring the input event data (objective variable) from the selected explanatory variable (step S315).

Backward Elimination is a method of first creating a model that includes all features (explanatory variables) and then sequentially deleting features that are judged to be insignificant. Backward Elimination clarifies the explanatory variables to be deleted by broadly visualizing the relationship between the explanatory variables such as meteorological data and event information (an example of additional data) and the objective variable. As a method of deleting the explanatory variable, a method of deleting the explanatory variable specified by the user based on the degree of influence (importance) and the like, and a method of deleting the explanatory variable by the extraction unit 103 based on the comparison result between the degree of influence and the threshold value (hereinafter, Boruta) explained in the above can be applied.

Boruta, which is an example of Backward Elimination, is a method of creating false features and comparing their importance in order to extract effective features from a large number of features. In Boruta, for example, an existing feature (Original Data) is copied, a sample in each column is shuffled to create a fake feature (Shadow Feature), and the existing feature and the fake feature are combined to make a random feature. Train the forest. From the highest importance of false features, it is possible to obtain an indication of the importance of existing features that do not contribute. That is, an existing feature that is less important than the most important of the false features is judged to be a feature if it is not effective.

Due to the nature of random forests, the importance of features changes each time they are trained. Therefore, it is necessary to obtain a large number of samples and then perform a statistical test, which increases the calculation cost to apply to big data.

In FIG. 9, an example of comparing the accuracy of the prediction result by the analyst has been described. Assuming that there is no analyst, the explanatory variables may be extracted without using the prediction result by the analyst. FIG. 14 is a flowchart showing an example of the explanatory variable extraction process in the case of such a configuration.

Since steps S401 to S410 are the same as steps S301 to S310 in FIG. 9, description thereof will be omitted. In the example of FIG. 14, it is not necessary to perform the prediction by the analyst in parallel with, for example, the process of step S403.

The user can specify the explanatory variables to be used for the next learning by referring to the contribution displayed in step S404. In the example of FIG. 14, the user cannot specify the explanatory variable with reference to the prediction result by the analyst.

In step S411, the learning unit 104 determines whether or not the inference accuracy is improved (step S411). For example, the learning unit 104 determines whether or not the accuracy when inferred using the machine learning model in step S410 is greater than the accuracy of the inference result in step S403.

When the inference accuracy is improved (step S411: Yes), the explanatory variable extraction process is terminated. When the inference accuracy is not improved (step S411: No), the extraction unit 103 selects an explanatory variable by another method. For example, the extraction unit 103 selects an explanatory variable by Backward Elimination (step S412). The learning unit 104 learns a machine learning model for inferring the input event data (objective variable) from the selected explanatory variable (step S413).

As described above, the index value representing the relationship between the explanatory variable and the meteorological data, which is calculated when the frequency distribution is created in steps S306 and S307, is, for example, the correlation coefficient between the explanatory variable and the meteorological data. The index value is not limited to the correlation coefficient, and the following index values may be used. In addition, the relationship between the explanatory variables and the meteorological data may be evaluated using a plurality of index values. As the number of indicators to be adopted and the number of explanatory variables increase, the evaluation accuracy of the relationship improves, but the calculation time increases. Therefore, it is desirable to use an appropriate number of indexes in consideration of these.
-Errors (root mean square error (RMSE), mean absolute error (MAE), etc.):
Evaluate the Euclidean distance between variables and evaluate the relationship based on the magnitude of the error.
・ Data shaping:
Draw moving averages or envelopes according to variable fluctuations and evaluate their relationships with correlation coefficients, errors, and other methods.
・ Phase change frequency:
The relationship is evaluated by the number of extremums whose demand has changed from an increase to a decrease and the number of extremums whose demand has changed from a decrease to an increase over a certain period of time.
・ Phase change interval:
The relationship is evaluated by the time interval distribution of the extreme value when the demand changes from increase to decrease and the time of the extreme value when the demand changes from decrease to increase in a certain time width.
・ Histogram density estimation (Peristimulus Time Histogram):
The frequency of event occurrence in a certain time width is expressed by the number of times, and the relationship is evaluated by focusing on the number of events that occurred within a certain period. For example, a frequency distribution is created based on how many spikes occur during a unit period.
-Ignition interval (interspike interval):
Create a frequency distribution (for example, the frequency distribution of the length of the spike interval) based on the frequency of sudden fluctuations over the entire period, and focus on the period from the occurrence of the previous spike to the occurrence of the next spike. Evaluate the relationship.

Next, an example of a display method for displaying the inference result in step S206 or the like will be described. 15 to 19 are examples of display screens for displaying inference results.

As shown in FIG. 15, the display screen includes a selection field 1501 and marks 1511, 1512, 1513 displayed on the map. The selection column 1501 is a column for selecting an explanatory variable that has contributed to the inference result. In the selection field 1501 of FIG. 15, temperature, wind speed, and rainfall / snowfall can be selected as explanatory variables. These explanatory variables are examples, and other explanatory variables may be added. For example, it may be configured so that the explanatory variable to be displayed in the selection field 1501 can be specified on another specification screen.

The mark 1511 is a symbol for indicating the occurrence position of the event predicted to occur. A position where a mark having the same shape as the mark 1511 is displayed means a position where one event is predicted to occur.

The marks 1512 and 1513 have numerical values written inside the circular symbols. A mark having such a shape means that it is predicted that a number of events corresponding to a numerical value will occur within the range including the position where the mark is displayed. That is, a mark having such a shape corresponds to a mark obtained by aggregating a plurality of marks 1511. The output control unit 105 may change the display mode (color, etc.) of the mark according to the number of occurrences. For example, the output control unit 105 may display the mark colors as green, yellow, and red when the number of occurrences is one digit, two digits, three digits or more, respectively.

FIG. 16 shows an example of a display screen displayed when only temperature is selected as an explanatory variable. As shown in the selection field 1601, in this example, only the air temperature (derived from the air temperature) is selected as the explanatory variable. In this case, the prediction result for the event predicted by the selected explanatory variable (temperature) is displayed on the map. In the example of FIG. 16, the marks 1511 and 1512 are displayed, and the marks 1513 are not displayed because they are not derived from the temperature.

The display screen may be enlarged or reduced according to the user's designation or the like. FIG. 17 shows an example of an enlarged display screen. As shown in FIG. 17, when the mark 1701 is selected, the output control unit 105 may display detailed information about the event corresponding to the mark 1701.

The above display screen is an example, and the display method of the inference result is not limited to these. In FIGS. 15 and 16, the number of occurrences is displayed numerically, but a display screen may be used in which the display mode (size, etc.) of the mark is changed according to the number of occurrences. 18 and 19 are views showing an example of a display screen configured in this way.

In the selection field 1801 of FIG. 18, all three explanatory variables are selected. In such a case, a display screen for displaying a circular mark having a radius corresponding to the number of occurrences is displayed at the occurrence position of the event predicted by the three explanatory variables. In the selection field 1901 of FIG. 19, one explanatory variable (derived from temperature) is selected. In such a case, a display screen for displaying a circular mark having a radius corresponding to the number of occurrences is displayed at the occurrence position of the event predicted by one selected explanatory variable.

When there is data to be compared such as a prediction result by an existing method and a prediction result by an analyst, the output control unit 105 displays the data to be compared and the prediction result by the present embodiment in comparison with each other. The display screen may be displayed.

By using the display screen as described above, the user can grasp what kind of explanatory variables contribute to the prediction.

(Application example)
The information processing device of this embodiment can be applied to the following systems.
(Application example 1) System for predicting traffic congestion on roads There are various factors that cause traffic congestion, and one of them is traffic demand. The traffic demand is the number of vehicles passing through the road in each time zone, and refers to the traffic volume when there is no limit to the traffic volume (traffic capacity) that can pass through the road. For example, when a road section designed assuming a traffic capacity of 50 vehicles per minute is flooded with more vehicles, it often becomes a bottleneck and congestion occurs.

Traffic demand has been statistically found to be strongly associated with meteorological data. Therefore, it is possible to make a prediction from the meteorological data according to the above embodiment. In addition, by making numerical corrections based on conventional traffic simulations for changes in traffic demand due to new road openings and regional events, etc., traffic demand forecasts for each date and time and for each road section ("x month") At the time of y day z, the traffic demand of the road section j is ○ vehicles / minute. "

By predicting the sections with high and low traffic demand, it is possible to present the optimum required route and analyze the factors with high and low traffic demand. Therefore, it is possible to determine measures to alleviate traffic congestion of road operators and to support the behavior of drivers.

(Application example 2) Incoming call prediction system in an insurance business call center For example, in the call center business of a business operator handling automobile insurance, it is required to optimize the staffing of operators. By forecasting the demand for the number of incoming calls that can occur by hour and region, it is possible to reduce costs by curbing idle human resources and improve the accuracy of management plans by refining employment plans. In addition, by predicting the types of incoming calls that can occur in each region, it is possible to precisely allocate after-sales service personnel who can handle the cases in advance. As a result, the time from the occurrence of incoming call to the service can be shortened, and customer satisfaction can be improved.

(Application example 3) Forecast information provision system in insurance companies By predicting the types of incoming calls that can occur by time and region, "engine non-stop" is given to automobile insurance users traveling in the relevant region at that time. Please be careful about the occurrence of. " In addition, for events that are expected to occur with high probability (puncture, overheating, collision, etc.), actions that should be taken to avoid end users, such as "Be careful of foreign objects on the road while driving". It becomes possible to provide information.

(Application example 4) Incoming call prediction system that continuously improves the model in insurance companies Accumulates weather information in areas where there are many accidents and breakdowns, and vehicle types that are often used in that area from incoming call information, and also Data obtained by hearing the end user's purpose of use of the vehicle in the region and time zone can be newly added as additional data. The additional data added in this way can be utilized for improving the accuracy, and the prediction accuracy of the machine learning model can be continuously improved.

As described above, according to the above embodiment, it is possible to more efficiently extract data having a bad influence on the accuracy of prediction and data having a good influence. As a result, a prediction system capable of predicting events with higher accuracy can be realized.

Next, the hardware configuration of the information processing apparatus according to the embodiment will be described with reference to FIG. FIG. 20 is an explanatory diagram showing a hardware configuration example of the information processing apparatus according to the embodiment.

The information processing device according to the embodiment connects to a control device such as a CPU (Central Processing Unit) 51 and a storage device such as a ROM (Read Only Memory) 52 or a RAM (Random Access Memory) 53 to communicate with each other. It is provided with a communication I / F 54 for performing communication and a bus 61 for connecting each unit.

The program executed by the information processing apparatus according to the embodiment is provided by being incorporated in ROM 52 or the like in advance.

The program executed by the information processing apparatus according to the embodiment is a file in an installable format or an executable format, and is a CD-ROM (Compact Disk Read Only Memory), a flexible disk (FD), or a CD-R (Compact Disk Recordable). ), DVD (Digital Versatile Disk), or the like, which may be recorded on a computer-readable recording medium and provided as a computer program product.

Further, the program executed by the information processing apparatus according to the embodiment may be stored on a computer connected to a network such as the Internet and provided by downloading via the network. Further, the program executed by the information processing apparatus according to the embodiment may be configured to be provided or distributed via a network such as the Internet.

The program executed by the information processing apparatus according to the embodiment can make the computer function as each part of the information processing apparatus described above. This computer can read a program from a computer-readable storage medium onto the main storage device and execute the program by the CPU 51.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

100 Information processing device 101 Acquisition unit 102 Coding unit 103 Extraction unit 104 Learning unit 105 Output control unit 111 Reasoning unit 121 Event data storage unit 122 Meteorological data storage unit 123 Additional data storage unit 124 Geographical data storage unit 125 Feature quantity storage unit 126 Model storage unit 127 Extraction information storage unit 131 Display unit

Claims (10)

Acquire the input data to be input to the model that outputs the inference result by inputting the input data including the time-series data whose value continuously changes according to the position, and the correct answer data representing the correct answer of the inference by the model. Acquisition department and
A learning unit that learns the model using the first input data selected from the input data and the correct answer data.
It is provided with an output control unit that outputs the contribution of the first input data to the inference result by the learned model.
The learning unit further learns the model using the third input data based on the second input data designated according to the output degree of contribution of the input data and the correct answer data.
Information processing device. The first relationship information representing the relationship between the second input data and the time series data matches or resembles the second relationship information representing the relationship between the third input data and the time series data.
The information processing device according to claim 1. The first relationship information representing the relationship between the second input data and the data including the data correlating with the second input data and the time-series data represents the relationship between the third input data and the time-series data. 2 Matches or resembles related information,
The information processing device according to claim 2. The first relationship information is a frequency distribution representing the number of times that an index value representing the relationship between the second input data and the time series data appears in each of a plurality of periods included in the designated period.
The second relationship information is a frequency distribution representing the number of times that an index value representing the relationship between the third input data and the time series data appears in each of a plurality of periods included in the designated period.
The information processing device according to claim 2 or 3. The learning unit determines whether the first relational information and the second relational information match or are similar to each other based on the distance between the plurality of frequency distributions.
The information processing device according to claim 4. Further, an extraction unit for randomly selecting the first input data from the input data is provided.
The learning unit learns the model using the first input data selected by the extraction unit and the correct answer data.
The information processing device according to claim 1. The time-series data is predetermined regional meteorological data.
The information processing device according to claim 1. The output control unit further displays the inference result of the learned model on the display device.
The information processing device according to claim 1. Acquire the input data to be input to the model that outputs the inference result by inputting the input data including the time-series data whose value continuously changes according to the position, and the correct answer data representing the correct answer of the inference by the model. Acquisition steps and
A first learning step of learning the model using the first input data selected from the input data and the correct answer data, and
An output control step that outputs the contribution of the first input data to the inference result by the learned model, and
A second learning step of further learning the model using the third input data based on the second input data specified according to the output degree of contribution among the input data and the correct answer data.
Information processing methods including. Computer,
Acquire the input data to be input to the model that outputs the inference result by inputting the input data including the time-series data whose value continuously changes according to the position, and the correct answer data representing the correct answer of the inference by the model. Acquisition department and
A learning unit that learns the model using the first input data selected from the input data and the correct answer data.
It functions as an output control unit that outputs the contribution of the first input data to the inference result by the learned model.
The learning unit further learns the model using the third input data based on the second input data designated according to the output degree of contribution of the input data and the correct answer data.
program.

Images