JP7481909B2 - Feature generation method and feature generation device - Google Patents

Description

The present invention relates to a feature generation device and a feature generation method that perform machine learning using features generated from time-series data.

For example, Patent Document 1 is known as a technology for generating a machine learning model from time-series data. Patent Document 1 discloses a technology for MIL (Multiple Instance Learning) for the purpose of failure prediction from time-series data (e.g., sensor values and event logs) of a manufacturing device, in which multiple subsets of negative bags are extracted, a classifier is generated that is trained together with a positive bag for each subset, and a feature value having a large average weight (applied to each feature value) of the classifier group is preferentially selected, and a failure prediction model is trained using the feature value as an input.

Patent Document 2 also discloses a technology for detecting side effects in pharmaceuticals, which targets the medical event history of each patient, learns combinations of diseases that occur within a specific period from medication, time series patterns of other medical events (e.g., hospitalization and medical expenses), and combinations of known medications and side effects (positive/negative), and scores whether or not a certain medical event history is a case of side effect occurrence.

Patent Literature 3 also discloses a technology for labeling training data, which starts with a few main features, and then presents additional features useful for labeling to an expert, who is asked to select them, in a process that is repeated several times, gradually increasing the number of features and improving the recall of the labels, and assigning labels to all positive examples with an appropriate number of features.

US Patent Application Publication No. 2015/0227838 US Patent Application Publication No. 2017/0083670 International Publication No. 2019/045759

The above conventional example does not take into consideration the narrowing down of features to be learned without manual intervention. This causes a problem in that when explanatory variables are synthesized by multiplying features, the number of combinations of explanatory variables can become enormous.

When generating a machine learning model that predicts the occurrence of a target event (positive example) from time series data, features that serve as input data for machine learning are generated from positive and negative examples. For time series data of positive examples, the analysis period is set to the date (or date and time) when the target event occurred as the base date, and a specified period from the base date is set as the analysis period.

On the other hand, the time series data for negative cases has the same analysis period as the positive cases, but since the target event does not occur, the above-mentioned conventional example has a problem in that it does not take into consideration how to determine the reference date for negative cases.

The present invention has been made in consideration of the above problems, and aims to prevent the amount of input data for machine learning from becoming too large when generating a machine learning model that predicts the occurrence of a target event from time series data, and to determine the reference date for negative example time series data.

The present invention is a feature generation method in which a computer having a processor and a memory receives time series data and generates features to be input data to a machine learning unit that predicts the occurrence of a target event, the method including a time series data input step in which the computer receives a plurality of time series data including values and timestamps, a target event occurrence data input step in which the computer receives target event occurrence data including a timestamp at which the target event occurred, a feature calculation definition input step in which the computer receives a feature calculation definition that defines the content for calculating the feature of the time series data, and a feature calculation step in which the computer converts the time series data into a positive example time series data by referring to the target event occurrence data. a positive example reference date determination step in which the computer determines a positive example reference date, which is a reference date in the positive example time series data; a positive example feature calculation step in which the computer calculates positive example features from a combination of the positive example time series data and the positive example reference date based on the feature calculation definition; a negative example reference date determination step in which the computer determines a negative example reference date using the positive example reference date, the positive example features, and the negative example time series data as inputs; and a negative example feature calculation step in which the computer calculates negative example features from a combination of the negative example time series data and the negative example reference date based on the feature calculation definition.

Therefore, in the present invention, by calculating the cumulative value of feature importance starting from the most important feature, and gradually eliminating features with less importance based on a threshold value for the cumulative value, it is possible to narrow down the important features and prevent the number of combinations of features (explanatory variables) from becoming too large. In addition, it is possible to determine the reference date for negative examples from the time series data of negative examples, using the proximity to the features of positive examples as an indicator.

Details of at least one implementation of the subject matter disclosed herein are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the disclosed subject matter will become apparent from the following disclosure, drawings, and claims.

FIG. 1 is a block diagram illustrating an example of a configuration of a time-course data analysis device according to a first embodiment of the present invention. FIG. 2 illustrates an example of processing performed by the time-course data analysis device according to the first embodiment of the present invention. 1 illustrates an example of a feature amount importance cumulative value graph according to the first embodiment of the present invention. 11 is a flowchart illustrating an example of processing performed by the feature selection unit according to the first embodiment of the present invention. FIG. 11 illustrates the first embodiment of the present invention and is a diagram illustrating an example of a sliding process of a reference date that is performed by the temporal feature generating unit. FIG. 11 illustrates an example of a process for determining a negative example base date that is performed by the negative example base date determination unit according to the first embodiment of the present invention. FIG. 2 illustrates an example of a configuration of a negative example base date determination unit according to the first embodiment of the present invention. 11 is a flowchart illustrating an example of a process performed by the negative example base date determination unit according to the first embodiment of the present invention. FIG. 11 illustrates an example of a process performed by the time-course data analysis device according to the second embodiment of the present invention. FIG. 11 illustrates an example of a feedback process of importance performed by the longitudinal data analysis device according to the second embodiment of the present invention. 11 is a flowchart illustrating an example of a process performed by a time-varying feature quantity generating unit and a feature selecting unit according to a second embodiment of the present invention. FIG. 13 illustrates an example of a process performed by the negative example base date determination unit according to the third embodiment of the present invention. 13 is a graph illustrating a relationship between time-series data and a warning period according to a fourth embodiment of the present invention. FIG. 13 illustrates an example of a process performed by the negative example base date determining unit according to the fourth embodiment of the present invention. 13 is a flowchart illustrating a modified example of the process performed by the negative example base date determination unit according to the fourth embodiment of the present invention.

The following describes an embodiment of the present invention with reference to the attached drawings.

Figure 1 shows a first embodiment of the present invention, and is a block diagram showing an example of the configuration of a time-course data analysis device 1.

The time-course data analysis device 1 is a computer including a processor 2, a memory 3, a storage device 4, an input device 5, an output device 6, and a communication device 7.

The longitudinal data analysis device 1 of this embodiment uses, for example, the account balance of a financial institution as the time series data 102 for learning, and the occurrence of a default (bad debt) as the target event, and shows an example of generating a machine learning model that predicts the occurrence of a default from the time series trend of the account balance.

The time series data 102 is not limited to the account balance, and the target event is not limited to the occurrence of a default on a debt. For example, the occurrence of a target event such as a failure may be predicted from a time series of a physical quantity.

In this embodiment, the time series data 102 includes a value (balance), a timestamp (date), and a pre-set identifier in one record for each account identifier.

The memory 3 of the longitudinal data analysis device 1 stores a longitudinal feature generation unit 110 that calculates features from pre-collected time series data 102, a feature selection unit 150 that narrows down the features, and a machine learning unit 160 that performs machine learning to generate a predictive model.

The functional units of the temporal feature generation unit 110, the feature selection unit 150, and the machine learning unit 160 are loaded into memory 3 as programs.

The processor 2 operates as a functional unit that provides a specific function by executing processing according to the program of each functional unit. For example, the processor 2 functions as a feature selection unit 150 by executing processing according to a feature selection program. The same applies to other programs. Furthermore, the processor 2 also operates as a functional unit that provides each function of the multiple processes executed by each program. Computers and computer systems are devices and systems that include these functional units.

The temporal feature generation unit 110 includes a time series data division unit 111, a positive example reference date determination unit 114, a negative example reference date determination unit 119, and a feature calculation unit 116. The time series data division unit 111 divides the time series data 102 into time series data of positive examples in which a target event (default) has occurred, and time series data of negative examples in which a target event has not occurred.

The positive example reference date determination unit 114 determines the date and time when the target event occurred as the positive example reference date by referring to the target event occurrence time data 101. The negative example reference date determination unit 119 determines the reference date for the negative example time series data based on the feature amount calculated from the negative example time series data, as described below. The feature amount calculation unit 116 calculates the feature amount from each of the positive example time series data and the negative example time series data, as described below.

The feature selection unit 150 includes a feature importance calculation unit 151 and a feature accumulation threshold determination unit 153. As described below, the feature importance calculation unit 151 calculates an index indicating the degree to which an increase or decrease in the value of a feature affects the predicted value of the model generated by the machine learning unit 160 as importance. The feature importance calculation unit 151 can employ, for example, a configuration that combines LightGBM and SHAP (Shapley Additive exPlanations).

In the feature importance calculation unit 151, the prediction model generated by LightGBM predicts the presence or absence (1, 0) of the target event, and SHAP calculates the importance of the extent to which each feature influenced the prediction result.

The feature accumulation threshold determination unit 153 determines the features to be excluded from the learning target using the cumulative importance value calculated by the feature importance calculation unit 151 and a predetermined threshold Th1, and narrows down the number of combinations of objective variables when synthesizing objective variables from the product of the features.

The machine learning unit 160 performs machine learning using the features of the positive and negative examples narrowed down by the feature selection unit 150 as input data, and generates a model that predicts with high accuracy the occurrence of a target event that occurs infrequently.

The storage device 4 stores target event occurrence time data 101, time series data 102, positive example reference date 115, negative example reference date 120, positive example feature amount 118, negative example feature amount 121, feature amount calculation definition 117, first feature amount list 122, feature amount importance 152, and second feature amount list 154.

The target event occurrence time data 101 includes the date when a preset target event occurred in the time series data 102, the account identifier, and the account balance in one record (not shown). The time series data 102 includes the date, balance, identifier, etc. for each account identifier in one record (not shown).

In the positive example reference date 115, the account identifier output by the positive example reference date determination unit 114 and the reference date of the positive example time series data are stored in one record. In the negative example reference date 120, the account identifier output by the negative example reference date determination unit 119 and the reference date of the negative example time series data are stored in one record. The determination of each reference date will be described later.

In the positive example feature 118, the feature of the positive example time series data 112 calculated by the feature calculation unit 116 and the identifier of the time series data 102 are stored in one record. The positive example feature 118 is composed of the feature of the time series data 102 specified in the feature calculation definition 117.

In the negative example feature 121, the feature of the negative example time series data 113 calculated by the feature calculation unit 116 and the identifier of the time series data 102 are stored in one record. Note that the negative example feature 121 is composed of the feature of the time series data 102 specified in the feature calculation definition 117.

The feature calculation definition 117 stores the processing period of the time series data 102, the type of feature, and the calculation method specified in the feature calculation method user setting 103. In this embodiment, for example, statistics such as the average, maximum, minimum, variance, standard deviation, maximum-minimum, and coefficient of variation are used as the feature of the time series data 102.

The first feature list 122 includes a list of the time series data 102 to be learned, which is specified in the feature calculation method user setting 103. The first feature list 122 includes, for example, an identifier and a timestamp of the time series data 102. The first feature list 122 is not limited to this, and may be any data that allows the correspondence between the calculated positive example features 118 and negative example features 121 and the time series data 102 specified in the feature calculation definition 117 to be identified.

In addition, the first feature list 122 in this embodiment stores a list of positive example features 118 and negative example features 121 before being narrowed down by the feature selection unit 150.

In the feature importance 152, the feature identifier and the feature importance calculated by the feature importance calculation unit 151 are stored in one record. The feature identifier stores values corresponding to the feature identifiers of the positive example feature 118 and the negative example feature 121.

In the second feature list 154, a list of features narrowed down by the feature selection unit 150 is generated. The positive example features 118 and negative example features 121 listed in the second feature list 154 are input to the machine learning unit 160.

The machine learning unit 160 can employ, for example, AT/PRC (AI Technology/Prediction of Rare Case) or a known or publicly known machine learning device.

The input device 5 is, for example, a keyboard, a mouse, or a touch panel. The output device 6 is a display. The communication device 7 is connected to a network (not shown) and transmits and receives information.

Figure 2 is a diagram showing an example of processing performed by the longitudinal data analysis device 1. The longitudinal data analysis device 1 accepts feature calculation method user settings 103 via the input device 5 and the communication device 7.

The feature calculation method user settings 103 include, for example, the type of statistics to be used as features, the designation of the time series data 102 to be the subject of machine learning, and the designation of the target event occurrence time data 101. The designation of the time series data 102 can include the period to be learned (hereinafter referred to as the learning period), account attributes (industry type, etc.), and account identifiers.

In the temporal feature generation unit 110, first, the time series data division unit 111 reads the target event occurrence time data 101 and divides the time series data 102 into positive example time series data 112 where the target event has occurred and negative example time series data 113 where the target event has not occurred.

Next, the positive example reference date determination unit 114 of the temporal feature generation unit 110 acquires the account identifier of the target event occurrence time data 101 and the occurrence time (or date or timestamp) of the target event, and outputs the occurrence date of the target event as the positive example reference date 115.

The negative example reference date determination unit 119 of the temporal feature generation unit 110 acquires the positive example reference date 115, the positive example features 118, the negative example time series data 113, and the learning period of the feature calculation method user settings 103, and determines the negative example reference date 120 as described below.

In this embodiment, the method for determining the negative example reference date 120 is as follows: the negative example reference date determination unit 119 slides the negative example reference date 120 by a predetermined unit from the positive example reference date 115 to extract the negative example time series data 113 for the learning period, and causes the feature calculation unit 116 to calculate the feature amounts within the learning period as the negative example feature amounts for each reference date.

The negative example base date determination unit 119 causes the feature calculation unit 116 to calculate feature amounts for multiple pre-set statistical periods while sliding the candidates for the negative example base date by day, week, or month, and performs clustering for each candidate for the negative example base date with the base date slid to calculate the feature amounts for each negative example base date.

The negative example reference date determination unit 119 also calculates the features of the positive example time series data 112 for one or more positive example reference dates 115 over multiple statistical periods similar to the negative example reference date candidates, and calculates the positive example feature values for each reference date by clustering the features for each positive example reference date 115.

Then, the negative example reference date determination unit 119 arranges the negative example reference date feature quantities and the positive example reference date feature quantities in a specified feature space, and determines the reference date candidate corresponding to the negative example reference date feature quantities that are closest to the positive example reference date 115 feature quantities as the negative example reference date 120.

The negative example reference date determination unit 119 generates each reference date feature in multiple dimensions, and the distance between the reference date feature of the positive example and the negative example can be, for example, a geometric distance (e.g., Euclidean distance, etc.). The processing of the negative example reference date determination unit 119 will be described in detail later.

Next, the feature calculation unit 116 accepts the positive example reference date 115 and the positive example time series data 112, calculates the positive example features 118 according to the feature calculation definition 117, and outputs them to the negative example reference date determination unit 119, the feature selection unit 150, and the machine learning unit 160.

The feature calculation unit 116 also accepts the negative example reference date 120 and the negative example time series data 113, calculates the negative example features 121 according to the feature calculation definition 117, and outputs them to the feature selection unit 150 and the machine learning unit 160.

In addition, the temporal feature generation unit 110 generates a list of the positive example time series data 112 and the negative example time series data 113 contained in the positive example features 118 and the negative example features 121, and outputs the list as a first feature list 122.

Next, in the feature selection unit 150, the feature importance calculation unit 151 receives the positive example features 118, the negative example features 121, and the first feature list 122, and calculates the feature importance 152 for each feature. In this embodiment, a large importance value is assigned to a feature that has a large influence on the prediction result of the prediction model generated by LightGBM.

Next, in the feature selection unit 150, the feature accumulation threshold determination unit 153 sorts the first feature list 122 in descending order of the feature importance 152 value. Then, the feature accumulation threshold determination unit 153 accumulates values starting from the feature importance with the largest feature importance 152 value, and stores the features (positive example features 118 and negative example features 121) until the accumulated value reaches a predetermined threshold Th1 in the second feature list 154 as features to be learned. In addition, the feature accumulation threshold determination unit 153 deletes other features (non-accumulated positive example features 118 and negative example features 121) from the first feature list 122.

As a result, the feature accumulation threshold determination unit 153 narrows down the positive example features 118 and negative example features 121 calculated by the temporal feature generation unit 110 to those with high feature importance, thereby reducing the number of features to be learned. Note that the threshold Th1 can be a preset value such as the ratio of the accumulated value of the feature importance 152 or the ratio of the number of features.

The feature amount cumulative threshold determination unit 153 also inputs the generated second feature amount list 154 to the feature amount importance calculation unit 151 to generate the feature amount importance 152 again, and the feature amount cumulative threshold determination unit 153 performs a loop process to further narrow down the features.

In this loop process from the feature importance calculation unit 151 to generating the second feature list 154, if there is remaining (unprocessed) data in the first feature list 122 when the feature accumulation threshold determination unit 153 detects that the accumulated value of the feature importance 152 has reached the threshold Th1, the remaining (unprocessed) data is deleted and the feature importance 152 is calculated again, and the narrowing down loop can be performed until there is no remaining data in the first feature list 122 when the accumulated value reaches the threshold Th1.

Alternatively, the feature accumulation threshold determination unit 153 can repeat the process until the number of features in the second feature list 154 reaches a predetermined threshold Th2. The predetermined threshold Th2 may be a preset value, such as a ratio (e.g., 60% or less) to the sum of the number of features of the positive example features 118 and the negative example features 121.

As described above, the longitudinal data analysis device 1 of this embodiment determines the reference date that is the negative example reference date feature closest to the positive example feature 118, as the negative example reference date 120, making it possible to accurately set the reference date for the negative example time series data 113 in which no target event has occurred.

In other words, the longitudinal data analysis device 1 can perform learning using significant features by having the machine learning unit 160 compare features of positive examples and features of negative examples that have similar combinations of explanatory variables.

The longitudinal data analysis device 1 then calculates the feature importance of the positive example features 118 and the negative example features 121, and learns features whose feature importance is from the maximum value to a predetermined threshold value Th1. By deleting other features, it becomes possible to reduce the number of features to be input to the machine learning unit 160 and provide significant features to the machine learning unit 160.

Figure 3 is an example of a feature importance cumulative value graph 301. The feature importance cumulative threshold determination unit 153 sorts the first feature list 122 in descending order of feature importance, calculates the cumulative value of feature importance as the feature importance cumulative value, and calculates the number of accumulated features as the number of features.

The feature importance cumulative value graph 301 in Figure 3 shows an example in which the vertical axis represents the feature importance cumulative value and the horizontal axis represents the number of features, and the threshold Th1 is a ratio of the feature importance cumulative value (e.g., 90%). In the example shown, features corresponding to importance exceeding the threshold Th1 are deleted, and features corresponding to importance equal to or less than the threshold Th1 are stored in the second feature list 154. Note that the threshold Th1 is not limited to the feature importance cumulative value, and may be a ratio to the number of features.

Figure 4 is a flowchart showing an example of the processing performed by the feature selection unit 150. This processing starts after the positive example features 118, the negative example features 121, and the first feature list 122 are output from the temporal feature generation unit 110 (401).

First, the feature importance calculation unit 151 acquires the positive example features 118, the negative example features 121, and the first feature list 122 from the temporal feature generation unit 110 (402). The feature importance calculation unit 151 calculates the importance of the positive example features 118 and the negative example features 121 listed in the first feature list 122 (403).

As described above, the feature importance calculation unit 151 combines LightGBM and SHAP to predict the presence or absence of a target event by applying the features in the first feature list 122 to the prediction model generated by LightGBM, and calculates the importance of the extent to which each feature affected the prediction result. The feature importance calculation unit 151 then stores the calculated importance and the feature identifier in feature importance 152.

Next, the process proceeds to the feature amount accumulation threshold determination unit 153 (404). The feature amount accumulation threshold determination unit 153 acquires the feature amount importance 152 and the first feature amount list 122, and sorts the first feature amount list 122 in descending order of the feature amount importance 152 value (405).

Next, in steps 406 to 409, the feature accumulation threshold determination unit 153 accumulates the values of the feature importance 152 in order from the top of the first feature list 122, and executes a loop process until the accumulated value reaches a predetermined threshold Th1.

The feature accumulation threshold determination unit 153 obtains the importance of the feature importance 152 from the top of the first feature list 122, which has been sorted in descending order of importance, and accumulates them sequentially (407).

The feature accumulation threshold determination unit 153 determines whether the accumulated value has reached a predetermined threshold Th1 (408), and if it has reached the threshold Th1, ends the loop processing and proceeds to step 410, and if it has not reached the threshold Th1, proceeds to step 409 and repeats the loop processing.

Next, the feature accumulation threshold determination unit 153 determines whether there are any remaining (unprocessed) features in the first feature list 122 (410), and if there are, proceeds to step 411, and if there are no remaining features, proceeds to step 412. Note that the remaining number of features in the first feature list 122 indicates the features to be deleted shown in FIG. 3, and corresponds to the portion where the feature importance accumulation value exceeds the threshold Th1.

In step 411, the feature amount accumulation threshold determination unit 153 deletes the feature amounts that exceed the threshold Th1 in the first feature amount list 122, and updates the first feature amount list 122. Then, the feature amount accumulation threshold determination unit 153 returns to step 403 and repeats the above process.

On the other hand, in step 412, since the cumulative feature importance value is equal to or less than the threshold value Th1 and the number of features in the first feature list 122 is reduced, the contents of the first feature list 122 (feature identifiers) are output as the second feature list 154.

By the above process, the feature selection unit 150 can reduce the features whose cumulative feature importance value exceeds the threshold value Th1, generate a second feature list 154 composed of features with high importance, and input it to the machine learning unit 160.

Next, the processing of the negative example reference date determination unit 119 of the temporal feature generation unit 110 will be described with reference to Figures 5A and 5B.

Figure 5A is a diagram showing an example of the sliding process of the reference date performed by the negative example reference date determination unit 119 of the temporal feature generation unit 110. Figure 5B is a diagram showing an example of the process of determining the negative example reference date performed by the negative example reference date determination unit 119.

Figure 5A shows the relationship between observed values (e.g., balances) and time (or dates) as negative example time series data 113.

The negative example reference date determination unit 119 sets a preset date (e.g., positive example reference date 115) as the first reference date 1, and sets multiple preset statistical periods such as the past one month, three months, six months, and twelve months from reference date 1. The conditions for determining reference date 1 may be specified by the user in the feature calculation method user settings 103 from multiple positive example reference dates 115, or other conditions may be used.

Then, the negative example base date determination unit 119 sets the date obtained by adding (or subtracting) a preset sliding width (predetermined date interval) to (or from) base date 1 as base date 2, and sets multiple preset statistical periods, such as 1 month to 12 months.

Similarly, the negative example reference date determination unit 119 sets reference dates 3 to N shifted by a predetermined sliding width, and sets multiple statistical periods in the same manner as above. The negative example reference date determination unit 119 sets reference dates 1 to N so that the above statistical periods cover the entire period of the negative example time series data 113.

In the illustrated example, the reference dates 1 to N are shifted from the past reference date 1 to the present, but the present invention is not limited to this and may be shifted in the opposite direction. The negative example reference date determination unit 119 sets the reference dates 1 to N and multiple statistical periods so that the multiple statistical periods and reference dates cover the entire period of one negative example time series data. Next, the negative example reference date determination unit 119 causes the feature calculation unit 116 to calculate the feature amount of the negative example time series data 113 for each statistical period of the reference dates 1 to N, and clusters the feature amounts for the multiple statistical periods for each reference date to calculate the reference date-specific statistics of negative examples and associate them with the reference dates 1 to N.

The negative example reference date determination unit 119 also sets multiple predefined statistical periods for each positive example reference date 115, causes the feature calculation unit 116 to calculate the feature amounts of the positive example time series data 112, and calculates the feature amounts by reference date of the positive examples by aggregating the feature amounts of each statistical period for each positive example reference date 115.

The negative example reference date determination unit 119 places the reference date feature amounts of the negative examples and the reference date feature amounts of the positive examples in the feature space 602 shown in FIG. 5B, and calculates the geometric distance between the reference date feature amounts of the negative examples and the reference date feature amounts of the positive examples (reference date or positive example in the figure). Note that in the illustrated example, a two-dimensional space of feature amounts A and B is shown, but a feature space may be set according to the number of dimensions of the feature amounts.

Then, the negative example reference date determination unit 119 selects the reference date feature of the negative example that is closest to the reference date feature of the positive example from among the reference date feature of the negative example corresponding to reference date 1 to reference date N, and determines the reference date corresponding to the reference date feature of the negative example as the negative example reference date 120.

In the illustrated feature space 602, the geometric distance between the reference date feature of the positive example corresponding to positive example 2 (reference date 2 of the positive example) and the reference date feature of the negative example corresponding to reference date 5 is the shortest, so an example is shown in which reference date 5 is determined as the negative example reference date 120.

Figure 6 is a diagram showing an example of the configuration of the negative example reference date determination unit 119. The negative example reference date determination unit 119 includes a reference date sliding unit 802 that generates multiple reference dates from reference date 1 to reference date N as reference date candidates for negative examples, and a feature space shortest distance search unit 810 that calculates reference date feature values 804 from the feature values for each reference date, and determines the reference date of the negative example reference date feature values 804 that is closest to the positive example reference date feature values as the negative example reference date 120.

The statistical period for the reference date for negative and positive examples is a number of predetermined statistical periods, such as the above-mentioned 1 month, 3 months, 6 months, and 12 months.

The negative example reference date determination unit 119 acquires one piece of negative example time series data 801 from the negative example time series data 113 specified in the feature calculation method user setting 103, determines reference date 1 from the above-mentioned specified conditions, and inputs reference date 1 to the reference date sliding unit 802.

The reference date sliding unit 802 generates a predetermined number of reference dates 2 to N with a preset sliding width. The negative example reference date determination unit 119 sets multiple preset statistical periods for each of the generated reference dates 1 to N, and inputs the negative example time series data 801 for each statistical period for each negative example reference date to the feature calculation unit 116 to calculate the negative example features.

The negative example reference date determination unit 119 accepts the negative example features for reference dates 1 to N calculated by the feature calculation unit 116 as reference date-specific features 804, and inputs them to the feature space shortest distance search unit 810.

The feature space shortest distance search unit 810 receives the positive example features 118 and the target event occurrence time data 101 as input, and causes the feature calculation unit 116 to calculate the reference date features of the positive examples for multiple statistical periods, similar to the reference date features 804 of the negative examples. The feature space shortest distance search unit 810 places the reference date features 804 of the negative examples and the calculated reference date features of the positive examples in the feature space 602 (see FIG. 5B), and calculates the geometric distance between each of the reference date features.

Then, the feature space shortest distance search unit 810 selects the negative example reference date feature (negative example feature 806) that is closest to the positive example reference date feature from among the negative example reference date features corresponding to reference date 1 to reference date N (805), and determines the reference date corresponding to the reference date feature 804 as the negative example reference date 120.

The feature space shortest distance search unit 810 can also output the negative example reference date 120 and the negative example feature 121 for each piece of negative example time series data 801 to be processed.

Figure 7 is a flowchart showing an example of processing performed by the negative example base date determination unit 119. This processing is started after the negative example base date determination unit 119 receives the negative example time series data 113, the positive example base date 115, and the positive example features 118.

The negative example reference date determination unit 119 selects one of the negative example time series data 113 to set it as the negative example time series data 801, and determines the positive example reference date 115 as the initial reference date 1 (901). Then, in a loop of steps 902 to 905, the negative example reference date determination unit 119 shifts the reference date by a predetermined sliding width and causes the feature calculation unit 116 to calculate the feature of the negative example.

In step 903, the negative example reference date determination unit 119 inputs the current reference date N, multiple pre-set statistical periods, and the negative example time series data 801 to the feature calculation unit 116 to calculate the negative example features.

In step 904, the negative example reference date determination unit 119 acquires the negative example features for each of the multiple statistical periods from the feature calculation unit 116, performs a predetermined statistical process (e.g., averaging), and stores the results as the reference date feature 804.

Next, the negative example reference date determination unit 119 returns to step 902 (905), shifts the reference date N by the sliding width, and repeats the above process until the end of the negative example time series data 801, thereby calculating the reference date-specific feature quantities 804 for each of reference dates 1 to N.

When the end of the negative example time series data 801 is reached, the negative example reference date determination unit 119 ends the loop of steps 902 to 905 and proceeds to step 906.

In step 906, the feature space shortest distance search unit 810 of the negative example reference date determination unit 119 causes the feature calculation unit 116 to calculate the feature of the positive example time series data 112 for a predetermined number of statistical periods from the positive example reference date 115, as described above, and clusters the feature of the statistical period for each positive example reference date 115 to obtain the feature for each positive example reference date.

Then, the feature space shortest distance search unit 810 places the negative example reference date feature 804 and the positive example reference date feature in the feature space, and calculates the geometric distance between each reference date feature. The feature space shortest distance search unit 810 then determines and outputs the reference date corresponding to the negative example reference date feature that is the smallest distance from the reference date feature of the positive example reference date 115 as the negative example reference date 120 (907).

By the above process, the temporal feature generation unit 110 calculates the feature amounts for each reference date of multiple negative examples by shifting the reference date N from the negative example time series data 113, and determines the negative example reference date 120 that is the starting point for calculating the feature amounts, using the close geometric distance to the reference date feature amounts of the positive example time series data 112 as an indicator.

As a result, the longitudinal data analysis device 1 is able to provide a highly accurate risk estimation model by having the machine learning unit 160 learn the positive example time series data 112 and the negative example time series data 113, which have similar combinations of explanatory variables, in the negative example time series data 113 in which the target event does not occur.

As described above, the longitudinal data analysis device 1 of Example 1 determines the negative example reference date determination unit 119 based on the closeness of the reference date features of the positive example features 118 from the negative example time series data 113 as an indicator, and selects important features by repeating the process of calculating cumulative values starting from the most important features and gradually eliminating features with less importance, thereby generating learning data for the machine learning unit 160.

This makes it possible to reduce the number of features to be learned by the machine learning unit 160, while generating a highly accurate machine learning model with reduced computational load by using highly important features (second feature list 154) and negative example reference dates 120 that have indices close to the positive example reference date features of the positive example features 118. The longitudinal data analysis device 1 of this embodiment makes it possible to generate a model that predicts with high accuracy the occurrence of a target event that occurs infrequently.

Figure 8 shows Example 2 of the present invention, and is a diagram showing an example of processing performed by the longitudinal data analysis device 1. In Example 1, an example was shown in which importance was used inside the feature selection unit 150, but in Example 2, an example is shown in which the importance of the feature calculated by the feature selection unit 150 is fed back to the longitudinal feature generation unit 110, and the longitudinal feature generation unit 110 notifies an update of the feature calculation definition 117 based on the importance of the feature.

The longitudinal data analysis device 1 of Example 2 adds a feature calculation definition update unit 201 to the longitudinal feature generation unit 110, adds a minimum feature number determination unit 202 and a previously output feature list 203 to the feature selection unit 150, and feeds back the feature importance 152 calculated by the feature importance calculation unit 151 to the feature calculation definition update unit 201 of the longitudinal feature generation unit 110 to the configuration of Example 1. The other configurations are the same as those of Example 1.

If there is a bias in the importance of each statistical period, the feature calculation definition update unit 201 notifies a change to the previously set statistical period. For example, if the importance of statistical periods of 1 month and 3 months is relatively greater than the importance of statistical periods of 6 months and 12 months, a notification is issued to add new statistical periods of "2 months" and "4 months."

In other words, the feature calculation definition update unit 201 makes it possible to detect greater importance by changing the number of statistical periods and the interval. The feature calculation definition update unit 201 may display the importance of each statistical period on the output device 6 and output a notification encouraging the user to change the number of statistical periods and the interval, or may notify the user of the longitudinal data analysis device 1 to review the statistical periods if a bias in importance is detected.

Alternatively, when the feature calculation definition update unit 201 detects a bias in importance across multiple statistical periods, it may update the feature calculation definition 117 to automatically change the statistical period.

When the second feature list 154 is output from the feature accumulation threshold determination unit 153, the minimum feature number determination unit 202 of the feature selection unit 150 compares the number of features (number of records) in the previous second feature list 154 stored in the previously output feature list 203 with the number of features (number of records) in the current second feature list 154.

If the number of features in the current second feature list 154 is smaller, the minimum feature number determination unit 202 determines that there is still room to reduce the number of features, and instructs the feature calculation definition update unit 201 to update the feature calculation definition 117 and calculate new features. The minimum feature number determination unit 202 also stores the latest second feature list 154 in the previous output feature list 203.

Figure 9 shows an example of the importance feedback process performed by the longitudinal data analysis device 1.

In the illustrated example, four periods of 1 month, 3 months, 6 months, and 12 months are preset as predetermined statistical periods in the feature calculation definition 117 of the temporal feature generation unit 110. In addition, an example is shown in which the average value is set as a statistical quantity as a condition for calculating the feature quantities of the time-series data 102.

The feature calculation unit 116 accepts the positive example time series data 112 and the negative example time series data 113 according to the statistical period of the feature calculation definition 117, causes the feature calculation unit 116 to calculate the positive example features 118 and the negative example features 121, and outputs the first feature list 122 and the statistical period 1171 to the feature selection unit 150.

When the feature importance calculation unit 151 of the feature selection unit 150 receives the positive example features 118, the negative example features 121, the first feature list 122, and the statistical period 1171, it calculates the importance of each feature and outputs it as feature importance 152.

Feature importance 152 stores the importance of multiple statistical periods for one reference date. In the example shown, the importance of the one-month average is 0.4, the importance of the three-month average is 0.5, and the importance of the six-month average and the twelve-month average is 0.1.

The feature calculation definition update unit 201, which receives feedback on the feature importance 152 from the feature importance calculation unit 151, detects that the importance of the one-month average and three-month average has increased.

The feature calculation definition update unit 201 subdivides the vicinity of the statistical period with a high importance value, and updates the feature calculation definition 117 by adding the two-month average between the one-month average and the three-month average, and the four-month average one month after the three-month average.

The feature calculation unit 116 recalculates the positive example features 118, the negative example features 121, and the first feature list 122 based on the updated feature calculation definition 117, and outputs them to the feature selection unit 150.

Figure 10 is a flowchart showing an example of processing performed by the temporal feature generation unit 110 and the feature selection unit 150. This processing begins when the temporal feature generation unit 110 receives the positive example time series data 112, the negative example time series data 113, the target event occurrence time data 101, and the feature calculation definition 117 (501).

The temporal feature generation unit 110 generates positive example features 118, negative example features 121, and a first feature list 122 from the input positive example time series data 112, negative example time series data 113, target event occurrence time data 101, and feature calculation definition 117 (502).

The feature importance calculation unit 151 of the feature selection unit 150 calculates the importance of each feature from the positive example features 118 and the negative example features 121, and outputs it as feature importance 152. Next, the feature accumulation threshold determination unit 153 sorts the first feature list 122 in descending order of importance value, selects features whose importance reaches the above-mentioned threshold Th1, and generates and outputs the second feature list 154 (503).

The minimum feature number determination unit 202 determines whether the number of features of the previous second feature list 154 stored in the previous output feature list 203 is greater than the number of features of the new second feature list 154 (504).

If the number of features in the previously output feature list 203 is greater, the minimum feature number determination unit 202 determines that there is still room to reduce the number of features and proceeds to step 505; otherwise, it proceeds to step 506.

In step 505, the feature calculation definition update unit 201 updates the feature calculation definition 117 as described above based on the feature importance 152, returns to step 502, calculates new features, and repeats the above process.

On the other hand, in step 506, the minimum feature number determination unit 202 outputs the second feature list 154 of the previously output feature list 203 as a result, and ends the process.

As described above, in the longitudinal data analysis device 1 of Example 2, the importance calculated by the feature importance calculation unit 151 is fed back to the feature calculation definition update unit 201 of the longitudinal feature generation unit 110, making it possible to suggest updating the feature calculation definition 117 in order to calculate a new feature.

In the above, an example has been shown in which the feature calculation definition update unit 201 changes the statistical period, but this is not limited to this, and the method of calculating the statistics may also be changed.

Fig. 11 illustrates a third embodiment of the present invention, and shows an example of processing performed by the negative example base date determination unit 119. In the third embodiment, the negative example base date determination unit 119 calculates a selection probability according to the occurrence frequency of the target event (positive example base date 115), and determines the negative example base date 120 based on the selection probability.

In this embodiment, an example is shown in which the negative example base date determination unit 119 includes a negative example base date selection unit 1102 and a positive example base date frequency distribution 1103.

The negative example base date determination unit 119 receives the positive example base date 115 output by the positive example base date determination unit 114, calculates a frequency distribution, and calculates a positive example base date frequency distribution 1103. The negative example base date selection unit 1102 receives as input one piece of negative example time series data 1101 selected from the negative example time series data 113, calculates a selection probability according to the occurrence frequency of the positive example base date 115 (positive example base date frequency distribution 1103), and determines a negative example base date 1104 for the negative example time series data 1101 based on the selection probability. The selection probability may be approximated by a well-known method such as a binomial distribution or a Poisson distribution.

The negative example base date determination unit 119 writes the determined negative example base date 1104 to the negative example base date 120 of the storage device 4. The negative example base date determination unit 119 calculates the negative example base date 1104 for each of the negative example time series data 113 to be processed and stores it in the negative example base date 120.

By the above processing, the longitudinal data analysis device 1 is able to determine the negative example reference date 120 of the negative example time series data 113 with the same probability distribution as the occurrence frequency of the positive example reference date 115, and the machine learning unit 160 is able to generate a model that predicts with high accuracy the occurrence of a target event that occurs infrequently.

Figures 12 to 14 show a fourth embodiment of the present invention. In this embodiment, an example is shown in which a negative example reference date 120 is determined from an event related to the occurrence of a target event (hereinafter, an important event). The negative example reference date determination unit 119 of this embodiment determines the period from the occurrence date of the important event to the positive example reference date 115 on which the target event occurred as a predictive period, calculates the predictive period for each piece of positive example time series data 112, and determines the negative example reference date 120 based on the results of statistical processing such as the frequency distribution of the calculated predictive periods.

In this embodiment, the important events are events that are preset when the target event is a default, such as the execution date of a loan contract, the execution date of a large amount of borrowing, or the date when the overdraft exceeds a specified amount.

The longitudinal data analysis device 1 calculates the period from the date on which these important events occurred to the positive example reference date 115 as the predictive period, calculates the predictive period for each of the multiple positive example time series data 112, and calculates the frequency distribution of the predictive periods of these positive examples. The longitudinal data analysis device 1 then calculates the negative example reference date 120 and the predictive periods of the negative examples of the negative example time series data 113 based on the frequency distribution of the predictive periods of the positive examples.

Figure 12 is a graph showing the relationship between the feature (important feature in the figure) calculated from the positive example time series data 112 and the warning period. Positive example time series data 701 is a graph showing the relationship between the feature and time of data selected from the positive example time series data 112. As the feature, for example, statistics of the borrowing balance and overdraft balance (e.g., average, maximum, minimum, variance, standard deviation, maximum-minimum, coefficient of variation, etc.) are used.

In the illustrated example, the date on which the target event occurred as described above is the positive case reference date 115, the date on which an important event related to the target event occurred is the important event occurrence date, and the period from the positive case reference date 115 to the important event occurrence date is the predictive period. Furthermore, in this embodiment, a specified period in the past from the positive case reference date 115 is set as the statistical period.

In the illustrated example, the day on which the feature value exceeds the threshold value Th3 is set as the important event occurrence date, but as described above, if there is important event occurrence data with a clear date and time, such as the execution date of a loan or the borrowing date, the occurrence data of the important event may be set as the important event occurrence date. The threshold value Th3 may be a preset value or ratio, such as 90% of the maximum feature value.

FIG. 13 is a diagram showing an example of processing performed by the negative example reference date determination unit 119. The negative example reference date determination unit 119 of Example 4 includes an important feature threshold excess search unit 1002, a predictive period determination unit 1004, and an addition unit 1007. The other configurations of the longitudinal data analysis device 1 of Example 4 are the same as those of Example 1 or Example 2.

The important feature threshold excess search unit 1002 regards the data received from the negative example time series data 113 as negative example time series data 1001, and refers to the items for determining important events set in the feature calculation definition 117, and causes the feature calculation unit 116 to calculate the negative example feature for the item.

The important feature threshold excess search unit 1002 compares the negative example feature calculated by the feature calculation unit 116 with a predetermined threshold Th4 from the past to the present of the time series of the negative example time series data 1001, and outputs the day on which the negative example feature first exceeds the threshold Th4 as the important event occurrence date 1003.

The warning period determination unit 1004 inputs the positive case feature amount 118 and the positive case reference date 115, compares them with a preset threshold value Th3, extracts the date on which the important event occurred for the positive case, and calculates the period between the positive case reference date 115 and the date on which the important event occurred as the warning period.

Then, the predictive period determination unit 1004 calculates the predictive period for each of the multiple positive example features 118, and further calculates the frequency distribution of the predictive periods and stores it as a positive example predictive period frequency distribution 1005.

Then, the predictive period determination unit 1004 probabilistically determines the predictive period 1006 so as to match the frequency distribution of the predictive periods of positive cases in the positive case predictive period frequency distribution 1005, and outputs it to the addition unit 1007.

The addition unit 1007 adds the warning period 1006 from the warning period determination unit 1004 to the negative example important event occurrence date 1003 to generate the negative example reference date 1008. The addition unit 1007 calculates the negative example reference date 1008 for each of the input negative example time series data 1001 and stores it in the negative example reference date 120.

The warning period determination unit 1004 may set the inverse of the period from the occurrence date of the important event to the occurrence date of the target event in the positive case as the new objective function.

Figure 14 is a flowchart showing a modified example of the process performed by the negative example reference date determination unit 119. In the illustrated example, the temporal feature generation unit 110 receives feedback of feature importance 152 from the feature selection unit 150 as shown in Figure 8 of the second embodiment, but is not limited to this.

The negative example reference date determination unit 119 starts processing by accepting the positive example features 118 of the specified positive example time series data 112 (S1301). It is determined whether or not any of the feature importances 152 fed back from the feature selection unit 150 exceeds a predetermined threshold Th5 (S1302). If any of the feature importances 152 exceeds the predetermined threshold Th5, the process proceeds to step S1303; otherwise, the process ends.

In step S1303, the negative example reference date determination unit 119 determines a threshold value Th3 for the current important feature amount for the received feature amount, as shown in FIG. 12. The threshold value Th3 can be set at a predetermined ratio to the maximum value of the important feature amount, as shown in FIG. 12.

In steps S1304 to S1307, the negative example reference date determination unit 119 repeats the process for each piece of positive example time series data 112 received, treating the positive example features 118 as important features.

In step S1305, if the negative example reference date determination unit 119 determines from the positive example feature amount 118 that there is a day that exceeds the threshold value Th3, it acquires this as the important event occurrence date. When the negative example reference date determination unit 119 acquires the important event occurrence date, it acquires the positive example reference date 115 and calculates the period from the positive example reference date 115 to the important event occurrence date as the warning period (1306).

When the negative example reference date determination unit 119 has completed the processing of steps S1304 to S1307 for all of the received positive example time series data 112, in step S1308, it calculates the frequency distribution of the predictive periods of positive examples and generates the positive example predictive period frequency distribution 1005.

In steps S1309 to S1313, the negative example reference date determination unit 119 repeats the process for each piece of negative example time series data 113 received. In step S1310, the negative example reference date determination unit 119 selects one negative example feature 121 and obtains the day on which the important feature (statistics of the loan balance and overdraft balance) exceeds a predetermined threshold value Th3 as the important event occurrence date.

In step S1311, the negative example reference date determination unit 119 refers to the positive example predictive period frequency distribution 1005 and determines the predictive period of the negative example so as to match the frequency distribution of the predictive period of the positive example. That is, the predictive period is treated as a random variable, the frequency distribution is regarded as a probability distribution, and the predictive period of each negative example is probabilistically selected according to the distribution. In step S1312, the negative example reference date determination unit 119 adds the predictive period of the negative example to the important event occurrence date to calculate the negative example reference date 120.

The negative example reference date determination unit 119 repeats steps S1309 to S1313 for all of the received negative example time series data 113.

By the above process, the longitudinal data analysis device 1 can determine the negative example reference date 120 of the negative example time series data 113 using the proximity to the features of the positive examples as an indicator, and the machine learning unit 160 can generate a model that predicts with high accuracy the occurrence of a target event that occurs infrequently.

<Conclusion>
As described above, each of the above embodiments can be configured as follows.

(1) A feature generation method in which a computer (time-course data analysis device 1) having a processor 2 and a memory 3 receives time series data (102) and generates features to be input data to a machine learning unit (160) that predicts the occurrence of a target event, the method including: a time series data (102) input step in which the computer receives a plurality of time series data (102) including values and timestamps; a target event occurrence data input step in which the computer receives target event occurrence data (target event occurrence time data 101) including a timestamp at which the target event occurred; a feature calculation definition input step in which the computer receives a feature calculation definition (117) that defines the content for calculating the feature of the time series data (102); and a feature calculation definition input step in which the computer refers to the target event occurrence data (101) and divides the time series data (102) into positive example time series data (112) and negative example time series data (113). a positive example reference date determination step in which the computer determines a positive example reference date (115) that is a reference date in the positive example time series data (112); a positive example feature calculation step in which the computer calculates a positive example feature (118) from a combination of the positive example time series data (112) and the positive example reference date (115) based on the feature calculation definition (117); a negative example reference date determination step in which the computer determines a negative example reference date (120) using the positive example reference date (115), the positive example feature (118), and the negative example time series data (113) as input; and a negative example feature calculation step in which the computer calculates a negative example feature (121) from a combination of the negative example time series data (113) and the negative example reference date (120) based on the feature calculation definition (117).

With the above configuration, the longitudinal data analysis device 1 can determine the negative example reference date determination unit 119 using the closeness of the reference date feature of the positive example feature 118 from the negative example time series data 113 as an index, thereby making it possible to determine a reference date in the negative example time series data 113 on which no target event has occurred. This enables the longitudinal data analysis device 1 to generate a model that predicts with high accuracy the occurrence of a target event that occurs infrequently.

(2) The feature generation method according to (1) above, further comprising: a time-course feature generation step in which the computer generates a list of the positive example features (118) and the negative example features (121) as a first feature list (122) and outputs the positive example features (118), the negative example features (121), and the first feature list (122); a feature importance calculation step in which the computer calculates feature importance (152) of the positive example features (118) and the negative example features (121) listed in the first feature list (122); and a feature accumulation threshold determination step in which the computer accumulates the feature importance (152) in descending order, and stores the positive example features (118) and the negative example features (121) until the accumulated value reaches a predetermined threshold Th1 in a second feature list (154) as features to be learned.

With the above configuration, by repeatedly calculating the cumulative value starting from the feature importance value with the highest and gradually eliminating features with lower importance, it is possible to select important features, thereby reducing the number of features to be trained by the machine learning unit 160 while still performing training using features with higher importance (second feature list 154), thereby making it possible to generate a model that can make highly accurate predictions.

(3) The feature generation method described in (1) above, wherein the feature accumulation threshold determination step is characterized in that, if there is unprocessed data in the first feature list (122) when the accumulated value reaches a predetermined threshold Th1, the unprocessed data is deleted and the feature importance (152) is calculated again in the feature importance calculation step, and the feature importance calculation step and the narrowing down by the feature accumulation threshold determination unit step are repeated until there is no unprocessed data in the first feature list (122) when the accumulated value of the feature importance (152) reaches the threshold Th1.

With the above configuration, the longitudinal data analysis device 1 can generate a highly accurate prediction model by selecting important features, thereby reducing the number of features to be learned by the machine learning unit 160, while still performing learning using features with high importance (second feature list 154).

(4) The feature generation method according to (2) above, further comprising a feature calculation update step in which the computer inputs the calculated feature importance (152) and changes the feature calculation definition (117) according to the value of the feature importance (152).

With the above configuration, in the longitudinal data analysis device 1, the importance calculated by the feature importance calculation unit 151 is fed back to the feature calculation definition update unit 201 of the longitudinal feature generation unit 110, making it possible to suggest updating the feature calculation definition 117 in order to calculate new features.

(5) In the feature generation method described in (1) above, the negative example reference date determination step includes a reference sliding step of setting a first reference date as a preset reference date and setting multiple reference dates from the first reference date to an Nth reference date by shifting the multiple reference dates at intervals of a predetermined number of days, a statistical period setting step of setting multiple statistical periods that are preset for each of the first reference date to the Nth reference date, and a negative example time series data setting step of calculating a feature of the negative example time series data (113) for each statistical period for each of the first reference date to the Nth reference date, and calculating a negative example reference date feature (804) for each reference date. A feature generation method comprising: a step of calculating a feature amount for each positive example reference date; a step of calculating a feature amount for each positive example reference date (115) by calculating a feature amount of the positive example time series data (112) for each of the plurality of statistical periods for each of the positive example reference dates (115) to calculate a feature amount for each positive example reference date (115); and a step of arranging the negative example reference date feature amount and the positive example reference date feature amount in a predetermined feature amount space, calculating the distance between each reference date, and determining the reference date of the negative example reference date feature amount that is closest to any of the positive example reference date feature amounts as the negative example reference date (120).

The above configuration enables the longitudinal data analysis device 1 to determine the negative example reference date determination unit 119 from the negative example time series data 113 using as an index the proximity of the reference date feature of the positive example feature 118. This enables the longitudinal data analysis device 1 to generate a model that predicts with high accuracy the occurrence of a target event that occurs infrequently.

(6) The feature generation method described in (5) above, wherein the negative example reference date determination step is characterized in that the first reference date to the Nth reference date and the statistical periods are set so that the multiple statistical periods cover the entire period of the negative example time series data (113).

With the above configuration, the longitudinal data analysis device 1 can determine the negative example reference date determination unit 119 using the closeness of the reference date feature of the positive example feature 118 from the negative example time series data 113 as an index. As a result, the longitudinal data analysis device 1 can determine the occurrence of a low frequency target event as a high frequency. (7) A feature generation method according to the above (1), wherein the negative example reference date determination step obtains an occurrence date of an important event related to the occurrence of the target event for each of the positive example time series data (112) as an important event occurrence date (1003), sets a period from the important event occurrence date (1003) to the positive example reference date (115) as a predictive period, calculates a positive example predictive period frequency distribution for each of the positive example time series data (112), and determines a negative example reference date (120) of the negative example time series data (113) so as to have the same probability distribution as the positive example predictive period frequency distribution.

With the above configuration, the longitudinal data analysis device 1 can calculate the warning period from the important event occurrence date of the positive example feature 118 to the target event occurrence time, set the important event occurrence date related to the occurrence of the target event in the negative example time series data 113, and determine the negative example reference date 120 by adding the warning period.

(8) The feature generation method according to (1) above, wherein the negative example reference date determination step calculates a frequency distribution (1005) of the positive example reference dates (115) and determines a negative example reference date (120) for each piece of negative example time series data (113) with the same probability distribution as the frequency distribution (1005).

The above configuration enables the longitudinal data analysis device 1 to determine the negative example reference date 120 of the negative example time series data 113 with the same probability distribution as the occurrence frequency of the positive example reference date 115, and the machine learning unit 160 to generate a model that predicts with high accuracy the occurrence of a target event that occurs infrequently.

(9) The feature generation method described in (5) above, further including a feature calculation definition (117) update step in which the computer receives the feature importance (152) and updates the feature calculation definition (117), the feature importance (152) calculation step calculates the feature importance (152) from the feature for each of the multiple different statistical periods, and the feature calculation definition (117) update step receives the feature importance (152) for each of the multiple different statistical periods, and notifies the addition of a new statistical period if there is a statistical period in which the feature importance (152) is greater than the other statistical periods.

With the above configuration, the longitudinal data analysis device 1 can suggest updating the feature calculation definition 117 to calculate new features by feeding back the importance calculated by the feature importance calculation unit 151 to the feature calculation definition update unit 201 of the longitudinal feature generation unit 110.

(10) The feature generation method described in (7) above, wherein the negative example reference date determination step includes the steps of: acquiring an occurrence date of an important event related to the occurrence of a target event for each of the positive example time series data (112) as an important event occurrence date (1003); calculating a period from the important event occurrence date (1003) to the positive example reference date (115) as a predictive period; calculating a frequency distribution of a positive example predictive period for each of the positive example time series data (112); and determining a predictive period from the positive example predictive period frequency distribution (1005); and calculating features for each of the negative example time series data (113); A feature generation method comprising the steps of: calculating feature importance (152) from the feature; accumulating the feature importance (152) in descending order of value; and calculating negative example feature (121) until the accumulated value reaches a predetermined threshold value Th1; calculating the day on which the feature first exceeds a predetermined threshold value Th4 from the past to the present in the time series of the negative example time series data (113) as the important event occurrence date (1003); and adding the predictive period calculated from the positive example predictive period frequency distribution (1005) to the important event occurrence date (1003) to calculate the negative example reference date (120).

With the above configuration, the longitudinal data analysis device 1 can calculate the warning period from the important event occurrence date of the positive example feature 118 to the time of occurrence of the target event, calculate the important event occurrence date related to the occurrence of the target event from the negative example feature 121, and determine the negative example reference date 120 by adding the warning period to the important event occurrence date of the negative example.

The present invention is not limited to the above-described embodiments, but includes various modified examples. For example, the above-described embodiments are described in detail to clearly explain the present invention, and are not necessarily limited to those having all of the configurations described. It is also possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. In addition, the addition, deletion, or replacement of part of the configuration of each embodiment with other configurations can be applied alone or in combination.

The above configurations, functions, processing units, and processing means may be realized in part or in whole in hardware, for example by designing them as integrated circuits. The above configurations and functions may be realized in software by a processor interpreting and executing a program that realizes each function. Information on the programs, tables, files, etc. that realize each function may be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

In addition, the control lines and information lines shown are those considered necessary for the explanation, and not all control lines and information lines on the product are necessarily shown. In reality, it can be assumed that almost all components are interconnected.

1 Longitudinal data analysis device 2 Processor 3 Memory 4 Storage device 101 Target event occurrence time data 102 Time series data 103 Feature calculation method user setting 110 Longitudinal feature generation unit 111 Time series data division unit 112 Positive example time series data 113 Negative example time series data 114 Positive example reference date determination unit 115 Positive example reference date 116 Feature calculation unit 117 Feature calculation definition 118 Positive example feature 119 Negative example reference date determination unit 120 Negative example reference date 121 Negative example feature 122 First feature list 150 Feature selection unit 151 Feature importance calculation unit 152 Feature importance 153 Feature accumulation threshold determination unit 154 Second feature list 160 Machine learning unit

Claims (20)

A feature generation method in which a computer having a processor and a memory receives time-series data and generates features to be input data to a machine learning unit that predicts an occurrence of a target event, comprising the steps of:
A time series data input step in which the computer receives a plurality of time series data including values and timestamps;
a target event occurrence data input step in which the computer receives target event occurrence data including a timestamp of when the target event occurs;
a feature calculation definition input step of the computer receiving a feature calculation definition that defines details of calculating the feature of the time series data;
a division step in which the computer divides the time series data into positive example time series data and negative example time series data by referring to target event occurrence data;
a positive example reference date determination step in which the computer determines a positive example reference date, which is a reference date in the positive example time series data;
a positive example feature calculation step of calculating a positive example feature from a combination of the positive example time-series data and the positive example reference date based on the feature calculation definition;
a negative example reference date determination step in which the computer determines a negative example reference date using the positive example reference date, the positive example feature amount, and the negative example time-series data as input;
a negative example feature calculation step of calculating negative example features from a combination of the negative example time-series data and the negative example reference date based on the feature calculation definition by the computer. 2. The feature generating method according to claim 1,
a time-course feature generating step of generating a list of the positive example features and the negative example features as a first feature list and outputting the positive example features, the negative example features, and the first feature list by the computer;
a feature importance calculation step in which the computer calculates feature importance of the positive example feature and the negative example feature listed in the first feature list;
the computer accumulates the feature importance values in descending order, and stores the positive example features and negative example features until the accumulated value reaches a predetermined threshold Th1 in a second feature list as features to be learned. 3. The feature generating method according to claim 2,
The feature amount accumulation threshold determination step includes:
a feature generation method characterized in that, if unprocessed data is present in the first feature list when the cumulative value reaches a predetermined threshold Th1, the unprocessed data is deleted and then the feature importance is calculated again in the feature importance calculation step, and the feature importance calculation step and the narrowing down step are repeated until there is no unprocessed data in the first feature list when the cumulative value of the feature importance reaches a predetermined threshold Th1. 3. The feature generating method according to claim 2,
a feature calculation update step in which the computer inputs the calculated feature importance and changes the feature calculation definition in accordance with the value of the feature importance. 2. The feature generating method according to claim 1,
The negative example reference date determination step includes:
a reference sliding step of setting a first reference date as a preset reference date and setting a plurality of reference dates from the first reference date to an Nth reference date at intervals of a predetermined number of days;
a statistical period setting step of setting a plurality of preset statistical periods for each of the first reference date and the Nth reference date;
a negative example reference date feature amount calculation step of calculating a feature amount of negative example time series data for each statistical period from the first reference date to the Nth reference date, and calculating a negative example reference date feature amount for each reference date;
a positive case reference date feature value calculation step of calculating a feature value of positive case time series data for each of the plurality of statistical periods for each of the positive case reference dates, and calculating a positive case reference date feature value for each of the positive case reference dates;
and determining, as the negative example reference date, a reference date for which the negative example reference date feature is closest to any one of the positive example reference date features, by arranging the negative example reference date feature and the positive example reference date feature in a predetermined feature space, and calculating a distance between each reference date. The feature generating method according to claim 5,
The negative example reference date determination step includes:
a first reference date to an Nth reference date and the statistical periods are set so that the statistical periods cover an entire period of negative example time series data. 2. The feature generating method according to claim 1,
The negative example reference date determination step includes:
a date of occurrence of an important event related to the occurrence of a target event for each of the positive example time series data is obtained as an important event occurrence date, a period from the important event occurrence date to the positive example reference date is set as a predictive period, a frequency distribution of the positive example predictive period is calculated for each of the positive example time series data, and a negative example reference date for the negative example time series data is determined so as to have the same probability distribution as the frequency distribution of the positive example predictive period. 2. The feature generating method according to claim 1,
The negative example reference date determination step includes:
a frequency distribution of the positive example reference date is calculated, and a negative example reference date is determined for each of the negative example time series data with the same probability distribution as the frequency distribution. The feature generating method according to claim 5,
a time-course feature generating step of generating a list of the positive example features and the negative example features as a first feature list and outputting the positive example features, the negative example features, and the first feature list by the computer;
a feature importance calculation step in which the computer calculates feature importance of the positive example feature and the negative example feature listed in the first feature list;
a feature accumulation threshold determination step in which the computer accumulates the feature importance values in descending order, and stores the positive example feature values and negative example feature values until the accumulated value reaches a predetermined threshold Th1 in a second feature list as feature values to be learned;
The computer further includes a feature quantity calculation definition updating step of updating the feature quantity calculation definition by inputting the calculated feature quantity importance,
The feature importance calculation step includes:
calculating the feature importance from the feature for each of the plurality of different statistical periods;
The feature amount calculation definition update step includes:
a feature generation method comprising: receiving the feature importance for each of the plurality of different statistical periods; and, if there is a statistical period in which the feature importance is greater than the other statistical periods, notifying the addition of a new statistical period. The feature generation method according to claim 7,
The negative example reference date determination step includes:
acquiring an occurrence date of an important event related to the occurrence of a target event for each of the positive case time series data as an important event occurrence date, calculating a period from the important event occurrence date to the positive case reference date as a predictive period, calculating a frequency distribution of the positive case predictive period for each of the positive case time series data, and determining a predictive period from the positive case predictive period frequency distribution;
calculating a feature amount for each of the negative example time-series data, calculating a feature amount importance from the feature amount, and accumulating the feature amount importance in descending order of value to calculate negative example feature amounts until an accumulated value reaches a predetermined threshold value Th1;
calculating, as an important event occurrence date, a date on which a feature amount from the past to the present of the time series of the negative example time series data first exceeds a predetermined threshold value Th4;
and calculating a negative example reference date by adding the predictive period calculated from the frequency distribution of the positive example predictive period to the important event occurrence date. A feature generation device including a processor and a memory, which receives time-series data and generates features to be input data to a machine learning unit that predicts an occurrence of a target event,
a time-series feature generation unit that receives a plurality of time-series data including values and timestamps, target event occurrence data including a timestamp when the target event occurs, and a feature calculation definition that defines content for calculating features of the time-series data, and outputs positive example features, negative example features, and a first feature list from the time-series data;
a feature selection unit that receives the positive example feature, the negative example feature, and the first feature list, and generates a second feature list that specifies the positive example feature and the negative example feature of a learning target,
The temporal feature generation unit
a time series data division unit that divides the time series data into positive example time series data and negative example time series data by referring to the target event occurrence data;
A positive case reference date determination unit that determines a positive case reference date, which is a reference date in the positive case time series data;
a feature amount calculation unit that calculates a positive example feature amount based on the feature amount calculation definition from a combination of the positive example time series data and the positive example reference date;
a negative example reference date determination unit that determines a negative example reference date using the positive example reference date, the positive example feature amount, and the negative example time-series data as inputs,
The feature amount calculation unit
a feature generating device for calculating negative example features based on the feature calculation definition from a combination of the negative example time-series data and the negative example reference date. The feature generating device according to claim 11,
The temporal feature generation unit
generating a list of the positive example features and the negative example features as a first feature list, and outputting the positive example features, the negative example features, and the first feature list;
The feature selection unit :
a feature importance calculation unit that calculates feature importance of the positive example feature and the negative example feature listed in the first feature list;
and a feature accumulation threshold determination unit that accumulates the feature importance values in descending order, and stores the positive example features and the negative example features until the accumulated value reaches a predetermined threshold Th1 in a second feature list as features to be learned. The feature generating device according to claim 12,
The feature amount accumulation threshold determination unit
a feature generation device characterized in that, if unprocessed data exists in the first feature list when the cumulative value reaches a predetermined threshold Th1, the unprocessed data is deleted and the feature importance calculation unit calculates the feature importance again, and the narrowing down is repeated by the feature importance calculation unit and the feature accumulation threshold determination unit until there is no unprocessed data in the first feature list when the cumulative value of the feature importance reaches the threshold Th1. The feature generating device according to claim 12,
a feature calculation definition update unit that receives the calculated feature importance and changes the feature calculation definition in accordance with the value of the feature importance. The feature generating device according to claim 11,
The negative example reference date determination unit
a feature generation device comprising: a first reference date set as a predetermined reference date; a plurality of reference dates set from the first reference date to an Nth reference date at intervals of a predetermined number of days; a plurality of predetermined statistical periods set for each of the periods from the first reference date to the Nth reference date; a feature amount of negative example time-series data for each statistical period set for each of the periods from the first reference date to the Nth reference date to calculate a negative example reference date feature amount for each reference date; a feature amount of positive example time-series data for each of the plurality of statistical periods set for each of the positive example reference dates to calculate a positive example reference date feature amount for each positive example reference date; The feature generating device according to claim 15,
The negative example reference date determination unit
a first reference date to an N-th reference date and the statistical periods are set so that the plurality of statistical periods covers an entire period of the negative example time series data. The feature generating device according to claim 11,
The negative example reference date determination unit
a date of occurrence of an important event related to the occurrence of a target event for each of the positive example time series data is obtained as an important event occurrence date, a period from the important event occurrence date to the positive example reference date is set as a predictive period, a frequency distribution of the positive example predictive period is calculated for each of the positive example time series data, and a negative example reference date for the negative example time series data is determined so as to have the same probability distribution as the frequency distribution of the positive example predictive period. The feature generating device according to claim 11,
The negative example reference date determination unit
a frequency distribution of the positive example reference date is calculated, and a negative example reference date is determined for each of the negative example time series data with the same probability distribution as the frequency distribution. The feature generating device according to claim 15,
The temporal feature generation unit
generating a list of the positive example features and the negative example features as a first feature list, and outputting the positive example features, the negative example features, and the first feature list;
The feature selection unit:
a feature importance calculation unit that calculates feature importance of the positive example feature and the negative example feature listed in the first feature list;
a feature accumulation threshold determination unit that accumulates the feature importance values in descending order and stores the positive example feature values and the negative example feature values in a second feature list until the accumulated value reaches a predetermined threshold Th1 as feature values to be learned,
the feature generation device further includes a feature calculation definition update unit that updates the feature calculation definition by inputting the calculated feature importance,
The feature importance calculation unit
calculating the feature importance from the feature for each of the plurality of different statistical periods;
The feature amount calculation definition update unit is
and receiving the feature importance for each of the plurality of different statistical periods, and if there is a statistical period in which the feature importance is greater than the other statistical periods, notifying the addition of a new statistical period. The feature generation method according to claim 17,
The negative example reference date determination unit
a predictive period determination unit that obtains an occurrence date of an important event related to the occurrence of a target event for each of the positive case time series data as an important event occurrence date, calculates a period from the important event occurrence date to the positive case reference date as a predictive period, calculates a positive case predictive period frequency distribution for each of the positive case time series data, and determines a predictive period from the positive case predictive period frequency distribution;
an important feature search unit that calculates a feature for each of the negative example time-series data, calculates a feature importance from the feature, accumulates the feature importance in descending order of value, calculates negative example features until an accumulated value reaches a predetermined threshold Th1, calculates a day on which a feature exceeds a predetermined threshold Th4 for the first time from the past to the present in the time series of the negative example time-series data as an important event occurrence date, and calculates a negative example reference date by adding a predictive period calculated from the frequency distribution of a predictive period of a positive example to the important event occurrence date.

Images