JP5071851B2 - Prediction device using time information, prediction method, prediction program, and recording medium recording the program - Google Patents

Description

  The present invention relates to a prediction device, a prediction method, a prediction program, and a recording medium storing the program for predicting an event that will occur in the future by learning learning data indicating an event whose occurrence order is given in advance.

  2. Description of the Related Art Conventionally, a technique related to a prediction problem for predicting an event occurring in the future by learning learning data indicating an event in which an occurrence order is given in advance is known. In this prediction problem, for example, an input sample (input data) x represented by Expression (101) and an output sample (output data) y represented by Expression (102) are assumed. Here, X represents an input sample set, and Y represents an output sample set. The input sample x and the output sample y do not necessarily have a causal relationship. Then, the time t when the input / output sample (x, y) represented by the equation (103) is generated is defined by the equation (104). Here, T indicates the latest time, for example. And a prediction apparatus learns the learning data shown, for example by Formula (105) in a learning stage. Here, n indicates an index, and N indicates the number of learning data. Then, in the prediction stage, the prediction device predicts the output y of the sample (x, t) whose output y is unknown, as shown by Expression (106).

  When this prediction problem is applied to, for example, store-side purchase prediction, the output sample y indicates a purchased product, the input sample x indicates the purchase history at that time, and an input / output sample (x, y) is generated. The time t at that time indicates the purchase time. The index n indicates, for example, a purchase number or a purchaser (user), and the learning data number N indicates, for example, the number of purchases or the number of purchasers (number of users). Table 1 shows a specific example of data (purchase data) in the case of purchase forecast. Here, the purchase history is the product purchased one time ago. The purchase history and purchased products (input / output samples) are time series data. Note that x, y, and t are all discrete.

  Many of such time-series data have properties that change with time. As shown in the equation (107), the input / output (probability) distribution P at time t and the input / output (probability) distribution P at time t ′ are different from the time-series data whose properties change with time. Such time series data. In addition, although purchase data was mentioned as an example of time series data, time series data includes, for example, news article data, paper data, Web surfing data, and the like.

  In order to explain a specific example of time-series data whose properties change with time, purchase data is considered here. For example, in the forecast of product sales, in reality, sudden events such as the announcement of new products and cancellation of sales occur due to the management strategies and circumstances of manufacturers and stores. Therefore, by reflecting such latest information in the prediction, the predicted product candidates (output product candidates) change daily. In addition, the output distribution (predicted probability distribution) of each product also changes due to changes in global events such as seasons, trends, social and economic environments.

If it is possible to predict with high accuracy a product that the user will purchase next from the purchase data, the product can be recommended to the user. 2. Description of the Related Art Conventionally, for example, a recommendation technique with high prediction accuracy in consideration of a purchase order for products such as online shopping is known (for example, see Non-Patent Document 1).
Tomoharu Iwata, Takeshi Yamada, Nobuo Ueda, “Collaborative Filtering Considering Purchase Order”, Artificial Intelligence and Knowledge Processing Study Group, AI2007-3,13-18,2007

  Although the technology disclosed in Non-Patent Document 1 has high prediction accuracy, it does not consider time information, that is, purchase time, and aims to keep calculation cost low and to increase prediction accuracy of the latest data. It is not something to do. However, in a prediction problem in time-series data whose properties change with time, high prediction accuracy is often desired for data at the latest time, not high prediction accuracy for all learning data. This is because, for example, in the case of data whose property change is abrupt, even a model with high prediction accuracy for data at past times may not be used for predicting the future.

  Therefore, the present invention has been made in view of the above problems, and a prediction apparatus and a prediction method capable of predicting data at the latest time with high accuracy in time-series data whose properties change with time. An object is to provide a prediction program and a recording medium on which the program is recorded.

The present invention has been made to solve the above problems, and a prediction apparatus according to the present invention includes an input unit that inputs sequence data that is a sequence of learning sample data and prediction sample data, and a calculation unit. Storage means for storing the input series data and the prediction sample data and storing a calculation processing result by the calculation means, and an output means for outputting a prediction result by the calculation means, The learning sample data includes first observation data whose properties change according to time or region, second observation data whose properties change according to time or region in relation to the first observation data, and the first And index information indicating discrete values of time or area information corresponding to the second observation data, and data having three elements, and the prediction sample data, Of the three elements constituting the learning sample data, when the second observation data is unknown and the first observation data and the index information are known, the calculation means In order to approximate the true distribution of the series data at a predetermined discrete value, a predetermined mixing ratio mixed with an empirical distribution at each discrete value of the index information and a predetermined learning value in the series data A posterior probability estimating means for estimating posterior probabilities for the index information of the first and second observation data, on the condition of the first and second observation data of the sample data, and the estimated posterior probabilities Utilizing the estimation ratio so as to maximize the likelihood for the sequence data at the predetermined discrete value of the index information. A mixture ratio estimating means for determining the weight of the index information from the mixture ratio when the likelihood is maximized, a weight writing means for writing the determined weight of the index information in the storage means, and the storage The weight of the determined index information is read from the means, and the objective function based on the expected error using the weight and the series data are used to calculate the predetermined index information from the prediction sample data. A model construction unit that builds a model that predicts the unknown second observation data at discrete values and writes the model to the storage unit, and reads the built model and the prediction sample data from the storage unit, and the construction The prediction sample data in the predetermined discrete value of the index information based on the model and the prediction sample data. And prediction processing means for performing processing for predicting known second observation data.

According to such a configuration, the prediction device, as to minimize the expected error approximating the expected error obtained from series data related to the prediction target, to construct a model in predetermined discrete values of indicators information, constructed Based on the obtained model and the prediction sample data, a process for predicting unknown second observation data at a predetermined discrete value of the index information is performed. Here, the predetermined reference is, for example, time or area. A discrete index based on the standard of time indicates a specific time in a time having a one-way characteristic from the past to the future, and the series data is time series data. The discrete value at this time can use a predetermined unit such as date, hour, minute, and second, and the predetermined discrete value can be set to the latest time, for example. In addition, when the predetermined standard is an area, examples of the standard include the local environment, people living in the place, living, etc., and the index at this time is, for example, a position or distance And various statistics. As the discrete values at this time, position coordinates, various distance units, predetermined units used for various statistics and trend surveys, digital notation such as “0, 1”, and the like can be used.

For example, when the latest time is set as a discrete value determined in advance using time-series data with reference to time, the prediction device estimates the weight of time information as the weight of index information . By using the weight of the time information, useful information for learning a model that matches the data at the latest time can be acquired from past data other than the data at the latest time. In addition, with the weight of this time information, the experience error at the latest time can be used as an accurate approximation to the expected error at the latest time. Therefore, the prediction device predicts the unknown data at the latest time of the sample from the model constructed using the weights estimated in this way and the sample whose output (second observation data) is unknown with high accuracy. It becomes possible.

Further, according to such a configuration, the prediction apparatus includes a thing posterior probability estimation means is provided with the mixing ratio estimating means, the posterior probability estimation means, E step in the EM algorithm (Expectation-Maximization algorithm) (Expectation step ) And the mixture ratio estimation means performs M steps (Maximization step), thereby obtaining a global optimum solution for the mixture ratio and determining the weight from the obtained mixture ratio.

In order to solve the above problem, the prediction apparatus according to the present invention includes an input unit that inputs sequence data that is a sequence of learning sample data and the sample data for prediction, a calculation unit, and the input sequence data. And storage means for storing the calculation processing result by the calculation means, and output means for outputting the prediction result by the calculation means, and the learning sample data is stored in time or Corresponding to first observation data whose properties change according to the region, second observation data whose properties change according to time or region in relation to the first observation data, and the first and second observation data. When the second observation data y is given, the data has three elements, index information indicating discrete values of time or area information. The distribution of one observation data x is a sequence of learning sample data that is similar in discrete values of the different index information, and the prediction sample data is selected from among the three elements that constitute the learning sample data. When the second observation data is unknown and the first observation data and the index information are known, the calculation means calculates a true distribution of the series data in a predetermined discrete value of the index information. In order to approximate, on the condition of the predetermined mixture ratio mixed in the empirical distribution in each discrete value of the index information and the second observation data of the predetermined learning sample data in the series data, the second Using the posterior probability estimating means for estimating the posterior probability for the indicator information of the observation data, respectively, and using the estimated posterior probabilities, So as to maximize the likelihood for the sequence data in the discrete value said predetermined the serial index information, and estimates the mixture ratio, the likelihood of the index information from the mixing ratio when maximized A mixture ratio estimating means for determining a weight; a weight writing means for writing the weight of the determined index information into the storage means; and reading the weight of the determined index information from the storage means and using the weight A model for predicting the unknown second observation data at the predetermined discrete value of the index information is constructed from the prediction sample data using the objective function based on the expected error and the series data. A model construction means for writing to the storage means; reading the constructed model and the prediction sample data from the storage means; Prediction processing means for performing a process of predicting the unknown second observation data of the prediction sample data at the predetermined discrete value of the index information based on the measurement sample data , To do .

  According to the prediction device having such a configuration, the prediction device uses the second observation data instead of the conditional distribution of the first and second observation data when estimating the mixture ratio using the EM algorithm to determine the weight. Therefore, the calculation is relatively easy and the processing load can be reduced.

  In the prediction device according to the present invention, the model construction means minimizes an objective function based on the weight of the estimated index information, the hyperparameter of the weight, and the error function in the approximated expected error. Model parameter estimating means for estimating a model at the predetermined discrete value by learning a model based on the sequence data and the estimated weight, and applied to the estimated model Hyperparameter estimating means for estimating the hyperparameter so as to minimize the error function, wherein the model parameter estimating means uses the hyperparameter when the error function is minimized. It is preferable to determine the estimated model as a model to be constructed.

  According to such a configuration, since the model construction unit includes the model parameter estimation unit and the hyper parameter estimation unit, the prediction apparatus includes the weight estimated by the weight estimation unit using the hyper parameter that minimizes the error function. It is possible to build an accurate model that stabilizes.

In order to solve the above-described problem, the prediction method according to the present invention includes an input unit that inputs sequence data that is a sequence of learning sample data and sample data for prediction, an arithmetic unit, and the input sequence data. And a storage unit that stores the prediction sample data and stores the calculation processing result by the calculation unit, and an output unit that outputs the prediction result by the calculation unit , the prediction method of the prediction device comprising: The learning sample data includes first observation data whose properties change according to time or region, second observation data whose properties change according to time or region in relation to the first observation data, and the first And the prediction sample data, the data having three elements: index information indicating discrete values of time or area information corresponding to the second observation data Of the three elements constituting the learning sample data, when the second observation data is data which is the first observation data and the index information is unknown known, the calculating means of the prediction device, In order to approximate the true distribution of the series data at a predetermined discrete value of the index information, a predetermined mixing ratio mixed with an empirical distribution at each discrete value of the index information, Estimating the posterior probabilities for the index information of the first and second observation data on the condition of the first and second observation data of predetermined learning sample data, and the estimated posterior probabilities The mixture ratio is set so as to maximize the likelihood for the sequence data at the predetermined discrete value of the index information. Determining a weight of the index information from the mixture ratio when the likelihood is maximized, writing a weight of the determined index information to the storage means, and from the storage means The weight of the determined index information is read, and the objective function based on the expected error using the weight and the series data are used to calculate the predetermined discrete value of the index information from the prediction sample data. A model construction step of constructing a model for predicting the second unknown observation data and writing it to the storage means; reading the constructed model and the prediction sample data from the storage means; and constructing the constructed model And the prediction sample data in the predetermined discrete value of the index information based on the prediction sample data And a prediction processing step for performing processing for predicting known second observation data.
In order to solve the above-described problem, the prediction method according to the present invention includes an input unit that inputs sequence data that is a sequence of learning sample data and sample data for prediction, an arithmetic unit, and the input sequence data. And a storage unit that stores the prediction sample data and stores the calculation processing result by the calculation unit, and an output unit that outputs the prediction result by the calculation unit, the prediction method of the prediction device comprising: The learning sample data includes first observation data whose properties change according to time or region, second observation data whose properties change according to time or region in relation to the first observation data, and the first And index information indicating discrete values of time or area information corresponding to the second observation data, and data having three elements, and the series data is the second view When the data y is given, the distribution of the first observation data x is a series of learning sample data similar in discrete values of the different index information, and the prediction sample data is the learning sample data. When the second observation data is unknown and the first observation data and the index information are known among the three elements constituting the calculation device, the calculation unit of the prediction device determines the index information in advance. In order to approximate the true distribution of the series data at the discrete values, a predetermined mixing ratio mixed with the empirical distribution at each discrete value of the index information, and a predetermined learning sample data in the series data Estimating the posterior probability of the second observation data with respect to the index information, on the condition of the second observation data, and the estimated Using each posterior probability, the mixture ratio is estimated so as to maximize the likelihood for the sequence data at the predetermined discrete value of the index information, and the likelihood is maximized. Determining the weight of the index information from the mixing ratio, writing the determined weight of the index information to the storage means, and reading the determined weight of the index information from the storage means The unknown second observation data at the predetermined discrete value of the index information is predicted from the prediction sample data using the objective function based on the expected error using the weight and the series data. A model construction step of constructing a model and writing it in the storage means; reading the constructed model and the prediction sample data from the storage means; Prediction processing step for performing a process of predicting the unknown second observation data of the prediction sample data at the predetermined discrete value of the index information based on the built model and the prediction sample data And executing.

According to the prediction method of the procedure, the prediction device, as to minimize the expected error approximating the expected error obtained from series data related to the prediction target, to construct a model in predetermined discrete values of indicators Information Based on the constructed model and the prediction sample data, a process of predicting unknown second observation data at a predetermined discrete value of the index information is performed. For example, when the latest time is set as a discrete value determined in advance using time-series data with reference to time, the prediction device estimates the weight of time information as the weight of index information . By using the weight of the time information, useful information for learning a model that matches the data at the latest time can be acquired from past data other than the data at the latest time. In addition, with the weight of this time information, the experience error at the latest time can be used as an accurate approximation to the expected error at the latest time. Therefore, the prediction device predicts the unknown data at the latest time of the sample from the model constructed using the weights estimated in this way and the sample whose output (second observation data) is unknown with high accuracy. It becomes possible.

  A prediction program according to the present invention causes a computer to execute the above-described prediction method. By being configured in this way, a computer in which this program is installed can realize each function based on this program.

  A computer-readable recording medium according to the present invention is characterized in that the prediction program described above is recorded. By being configured in this way, a computer equipped with this recording medium can realize each function based on a program recorded on this recording medium.

  According to the present invention, information useful for learning a model that matches the predetermined discrete value data of the index information by using the weight of the index information indicating time information or the like, the predetermined discrete value data By taking in data other than the above, it is possible to use an experience error at a predetermined discrete value as an accurate approximation to an expected error. Therefore, it is possible to predict data at the latest time with high accuracy in time-series data whose properties change with time.

  Hereinafter, the best mode (hereinafter referred to as “embodiment”) for carrying out the prediction apparatus and the prediction method of the present invention will be described in detail. For convenience of explanation, the outline of the present invention will be described, and then embodiments will be described with reference to the drawings.

[Outline of the present invention]
The present invention provides an input sample (first observation data on the reference side), an output sample (second observation data on the prediction side), and a predetermined value for discretely perceiving a change in properties indicated by the output sample. The input sample and the index information are known by learning the sequence data indicating the sequence related to the index information about the input / output sample (learning sample data) related to the prediction target having the index information indicating the discrete index based on the reference. This relates to a prediction problem in which an unknown output sample is predicted from a sample whose output sample is unknown (prediction sample data). In particular, according to the present invention, it is preferable that data (time-series data) that changes according to time information be a prediction target. Examples of data types whose characteristics change with time include purchase data, news article data, paper data, Web surfing data, and the like. In particular, when the data to be predicted is purchase data, the purchased object (product) is not limited to an entity as a substance but may be a service such as data. Such service data is, for example, image data, video data, music data, and the like.

  In the present embodiment, in order to simplify the description of the present invention, it will be described with the purchase data that changes according to the time information as a prediction target. In this case, it can be said that the present invention is a prediction method for learning a model having high prediction accuracy for the latest data, rather than learning a model having high prediction accuracy for all learning data. In order to perform such learning, it is necessary to accurately estimate the expected error that should be minimized at the latest time. Therefore, in the present invention, a process for estimating a weight for weighting the time (time information) at which a sample is generated (weight estimation process) and a process for building a prediction model that stably incorporates the estimated weight ( Model building process). Hereinafter, the principles of the weight estimation process and the model construction process will be sequentially described.

<Principle of weight estimation processing>
The expected error E _T for the data (sample) at the latest time is expressed by equation (1). Here, the unit of time is arbitrary, and the latest time can indicate the latest hour, minute, second, the latest date, the latest month, the latest year, and the like. When learning a model with high prediction accuracy for such latest data, a model that minimizes the expected error E _T shown in Expression (1) is desirable. In Expression (1), T represents the latest time, and J (x, y; M) represents an error function of a prediction model (hereinafter simply referred to as a model) M when a sample (x, y) is given. . P (x, y; T) indicates a probability distribution (hereinafter simply referred to as distribution) of the sample (x, y) at the latest time. As the model M, for example, an arbitrary model such as a primary Markov model or a secondary Markov model can be used. Here, the input sample (x) is the first observation data and is also simply referred to as input. The output sample (y) is the second observation data, and is also simply called output.

  As the error function J (x, y; M), for example, a negative log likelihood shown in Expression (2), a 0-1 loss function shown in Expression (3), and the like can be considered. In the following, it is assumed that the logarithm is a natural logarithm, that is, the base of the logarithm log is “e”.

When the number of samples at the latest time is an infinite limit, the experience error at the latest time coincides with the expected error E _T at the latest time. However, when learning is performed using only the latest data, the number of samples is smaller than in the case where learning is performed using all data, and therefore, an appropriate model may not be obtained. Therefore, it is not preferable to learn using only the latest time data. In addition, it is considered that past data before the latest time data includes some information useful for learning a model that matches the latest time data. Therefore, in the weight estimation process, the error function J (x, y; M) of the model M when the sample (x, y) is given is weighted by the weight at the time when the sample (x, y) is generated. The model M is learned so as to minimize the error E (M). The error E (M) at this time is expressed by equation (4). In equation (4), w (t _n ) is a weight at time t _n , and how much reference is made to the data at time t _n in order to learn the model at the latest time T (model to be constructed later based on the weight). Represents

The validity of using the formula (4) instead of the above-described formula (1) will be described below.
First, the number of samples whose time is t is N (t). The number of samples with time t, input x, and output y is N (t, x, y). Further, the distribution (true distribution) P (x, y | T) of the sample (x, y) at the latest time T is approximated by a mixture of empirical distributions at each time, as shown in Equation (5). To do. A mixing ratio satisfying the formula (5) is represented by the formula (6). Hereinafter, each mixing ratio is referred to as a mixture distribution P (t). In other words, Equation (5) approximates the true distribution P (x, y | T) at the latest time T with the result of adding the product of the empirical distribution and the mixed distribution P (t) at each time for the time t. Will show that. Further, the experience distribution at each time has a relationship of Expression (7) between the number of samples N (t) and N (t, x, y). In addition, the symbol “^ (hat)” added to P in Expression (7) indicates that P is an empirical distribution.

  As shown in Expression (8), the weight is defined as a value obtained by dividing the mixture distribution P (t) by the number of samples N (t) whose time is t. In addition, when the relationship of Formula (7) is applied to Formula (8), a weight is shown by Formula (9).

At this time, the error E (M) shown in Equation (4) is rewritten as Equation (10). In Expression (10), the second line on the right side shows a modification result based on the definition of the number of samples N (t, x, y), and the third line on the right side shows a modification result according to Expression (9). The fourth line on the right side shows a modification result using approximation of the above-described equation (5), and the fifth line on the right side shows a modification result by the above-described equation (1). As a result, it is understood that the error E (M) shown in the equation (4) is an approximation of the expected error E _T at the latest time T.

  Next, the weight w (t) at time t is estimated. Estimating the weight w (t) from the relationship of the above equation (8) is synonymous with estimating the mixture distribution P (t). That is, in order to estimate the weight w (t), the mixture ratio may be determined. The mixing ratio shown in Equation (6) can be estimated by maximizing the likelihood L (P) for the sample set at the latest time T shown in Equation (11). Note that the leave-one-out method is used to avoid overlearning. The second line on the right side of Expression (11) shows a rewritten expression using the relational expression shown in Expression (12). Expression (12) is a definition expression of the estimated value of the distribution at the latest time T estimated from Expression (5) described above when the n-th sample is removed.

An empirical distribution at time t when the n-th sample on the right side of Expression (12) is removed is expressed by Expression (13). In Equation (13), α is introduced to avoid the zero probability problem and indicates a parameter of the Dirichlet prior distribution. Also, δ _{t, T} represents the Kronecker delta.

Here, since the logarithmic function is an upward convex function, the function form of L (P) shown in the above equation (11) is also upward convex, and by maximizing L (P), P The global optimal solution (denoted P ^* ) can be obtained. An optimization method for maximizing L (P) is arbitrary, and for example, an EM algorithm can be used. In the present embodiment, description will be made using an EM algorithm. The log likelihood of each complete data is expressed by equation (14). Here, Expression (14) is shown in the second row on the right side of L (P) shown in Expression (11). In addition, the log likelihood to be maximized is the log likelihood of the complete data of the number n that satisfies t _n = T.

  Therefore, the conditional expected value of the log likelihood of the complete data to be maximized is expressed by equation (15). Here, τ indicates the number of times (τ = 0, 1, 2,...) That the two steps of the E step (Expectation step) and the M step (Maximization step) are repeated. In addition, when τ = 0, a predetermined initial value of the estimated value is shown.

In the EM algorithm, first, in step E, when the conditional expected value Q (P | P ^(τ) ) shown in equation (15) is calculated, P ^(τ) (t estimated using equation (17) is used. ) To calculate equation (16). In the calculation of Expression (16), the posterior probabilities for all times t of all samples (x, y) are estimated. Next, in the M step, when the conditional expected value Q (P | P ^(τ) ) shown in Expression (15) is maximized, Expression (17) is obtained using the result obtained using Expression (16). To obtain a new estimated value P ^{(τ + 1)} (t) for the mixture distribution. That is, the mixture ratio is estimated.

Then, by repeating the two steps of the E step and the M step until the convergence condition is satisfied, each mixture distribution P (t), that is, the mixture ratio shown in the above equation (6) is obtained. Here, when the conditional expected value Q (P | P ^(τ) ) shown in the equation (15) is maximized, the likelihood L (P) shown in the equation (11) is maximized. Become. That is, the likelihood L (P) shown in the above equation (11) converges. At this time, the convergence condition is satisfied. Therefore, in the present embodiment, in step E, calculation of equation (16) is performed using P ^(τ) (t) estimated using equation (17), and in step M, equation (16) is calculated. ) Is used to calculate the equation (17), and when the combination of the E step and the M step is completed, the likelihood L (P) shown in the equation (11) converges. It is determined whether or not. If the result of determination is that it has not converged, the E step and M step are alternately repeated until convergence. Then, the weight distribution w at the time t at which the sample (x, y) is generated is obtained by using the mixture distribution P (t) when the convergence condition is satisfied in the weight definition formula shown in the above formula (8). (T) is estimated.

<Principle of model building process>
In the weight estimation process, by estimating the weight w (t) at the time t when the sample (x, y) is generated, the expected error E _{T with} respect to the data (sample) at the latest time T shown in the above equation (1). Is approximated by the error E (M) shown in Equation (4). In order to stabilize the estimated weight w (t), in the model construction process, the hyperparameter λ is introduced, and the objective function to minimize the error E (M, λ) shown in Expression (18). It was decided that. That is, the model M is learned so as to minimize the error E (M, λ).

If the hyperparameter λ is learned from the data learned from the model M, overlearning may occur. Therefore, the hyperparameter λ is learned using a K-fold cross validation method. Note that the phenomenon in which the generalization error for data not used for learning becomes large is called overlearning. Specifically, in the K-fold cross-validation method, first, the entire learning data D shown in Expression (19) is divided into K subsets D _k as shown in Expression (20). In Expression (20), k represents a subset index in the learning data D. Of the divided subsets, the subset D _j is used for learning. Here, the subset D _j indicates (K−1) subsets excluding a predetermined subset, as shown in Expression (21). In Equation (21), the index of the predetermined subset to be excluded is k, and the index of the other subset is j. Then, using the subset D _j for learning, a model that minimizes the objective function (error E (M, λ)) shown in the equation (18) is constructed as a model at the latest time T. As a model to be constructed, a model depending on a predetermined subset k is shown in Expression (22). In the K-fold cross-validation method, since all types (K types) of sets divided as subsets D _{k to} be excluded are considered, the model shown in Expression (22) is applied to each index k of a predetermined subset to be excluded. Learn and build K models. In the equation (22), the symbol “^ (hat)” added to M indicates that M minimizes the argument of the argmin function.

For the hyperparameter λ, learning is performed so that the error function J other than the subset D _j , that is, for each subset D _k not used for learning, is minimized with respect to the data at the latest time T of the subset D _k . Furthermore, all types (K types) are considered as the subset D _{k to} be excluded. Specifically, the hyperparameter λ that minimizes the error function J is obtained by the calculation shown in Expression (23). In the equation (23), the symbol “^ (hat)” added to λ indicates that λ minimizes the argument of the argmin function.

  Here, when the objective function (error E (M, λ)) shown in the equation (18) is minimized, the objective function (error E (M, λ)) converges. At this time, the convergence condition is satisfied. Therefore, in the present embodiment, the calculation of the equation (22) is performed using the hyperparameter estimated using the equation (23), and the equation (22) is calculated using the model obtained using the equation (22). 23) When the calculation of 23) is completed and the combination of the estimation of the model and the estimation of the hyperparameter is completed, it is determined whether or not the objective function (error E (M, λ)) shown in the above equation (18) has converged. Determine. As a result of discrimination, if it has not converged, model estimation and hyperparameter estimation are alternately repeated until convergence. Then, when the convergence condition is satisfied, the model at that time is estimated as the model at the latest time T.

[Entire configuration of prediction device]
Next, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration of a prediction device according to an embodiment of the present invention. The prediction device 1 predicts an unknown output sample from a sample whose output is unknown by learning input data (sequence data) indicating a sequence of input / output samples (learning sample data) related to a prediction target. It is. As shown in FIG. 1, the prediction device 1 includes a calculation unit 2, an input unit 3, a storage unit 4, and an output unit 5. Each means 2 to 5 is connected to the bus line 11.

  The computing means 2 is a main control device composed of, for example, a CPU (Central Processing Unit) and a RAM (Random Access Memory). As shown in FIG. 1, the computing unit 2 includes a weight estimation unit (weight estimation unit) 21, a model construction unit (model construction unit) 22, a prediction processing unit (prediction processing unit) 23, and a memory 24. Consists of including. In addition, description of each part 21-23 is mentioned later.

  The input unit 3 includes, for example, a keyboard, a mouse, a disk drive device, and the like. The input means 3 inputs, for example, input data (series data) and samples as data and stores them in the storage means 4. In this embodiment, the input data (series data) is assumed to be data (purchase data) in the case of the purchase forecast shown in Table 1 described above.

  The storage means 4 is composed of, for example, a general hard disk device or the like, and stores various programs and various data used by the calculation means 2. The storage unit 4 stores a weight estimation program 41, a model construction program 42, and a prediction processing program 43 as programs in the program storage unit 40a. Then, the calculation means 2 reads these programs 41 to 43 from the storage means 4, expands them in the memory 24, and executes them, so that each function of the weight estimation unit 21, model construction unit 22, and prediction processing unit 23 is performed. Is realized.

  The storage means 4 stores input data (series data) 44, weights 45, model parameters 46, and samples 47 in the data storage unit 40b. Here, the input data (series data) 44 is data input from the input means 3, for example, purchase data shown in Table 1 described above. The weight 45 is data indicating the calculation processing result of the weight estimation unit 21 of the calculation means 2. The model parameter 46 is data indicating the calculation processing result of the model construction unit 22 of the calculation means 2. The sample 47 is, for example, data corresponding to the purchase data shown in Table 1 above, and the input x and time t are known and the output y is unknown.

  The output means 5 is, for example, a graphic board (output interface) and a monitor connected thereto. The monitor is composed of, for example, a liquid crystal display or the like, and displays a calculation processing result (for example, a prediction result <predicted purchase product information>).

Next, the detail of the structure of each part of the calculating means 2 is demonstrated.
<Weight estimation unit>
The configuration of the weight estimation unit 21 will be described with reference to FIG. FIG. 2 is a functional block diagram showing the configuration of the weight estimation unit. The weight estimation unit (weight estimation means) 21 obtains input samples (first sample data) of input / output samples (learning sample data) at the latest time (predetermined discrete values of index information) obtained from the input data (series data) 44. The expected error for one observation data) and the output sample (second observation data) is approximated using the model error function given input / output samples and the weight of the time information for each input / output sample. The weight of the time information is estimated so as to minimize the expected error. As shown in FIG. 2, the weight estimation unit 21 includes an input data reading unit 211, a posterior probability estimation unit 212, a mixture ratio estimation unit 213, and a weight writing unit 214.

The input data reading unit 211 reads the input data 44 and outputs it to the posterior probability estimation unit 212.
A posteriori probability estimation unit (a posteriori probability estimation means) 212 uses a predetermined mixture ratio to be mixed with the empirical distribution at each time, and the input data 44 in order to approximate the true distribution of the input data 44 at the latest time. The posterior probabilities for the time information t of the input / output sample (x, y) are estimated on the condition of the input / output sample (x, y). In the present embodiment, the posterior probability estimation unit 212 calculates in advance the empirical distribution represented by the above equation (13). In addition, the posterior probability estimation unit 212 estimates the posterior probabilities for all the times of all samples by the above-described equation (16).

The mixture ratio estimation unit (mixing ratio estimation unit) 213 uses each posterior probability estimated by the posterior probability estimation unit 212 to set the mixture ratio so as to maximize the likelihood for the input data 44 at the latest time. The weight is estimated from the mixture ratio when the likelihood is maximized. In the present embodiment, the mixture ratio estimation unit 213 estimates the mixture ratio according to the above equation (17). Further, the mixture ratio estimation unit 213 determines whether or not the likelihood shown in the equation (11) converges. When the likelihood does not converge, the mixture ratio at that time is output to the posterior probability estimation unit 212. When converged, the weight w (t) shown in the above equation (8) is calculated from the mixture ratio at that time, and is output to the weight writing unit 214.
The weight writing unit 214 writes the weight w (t) estimated by the mixture ratio estimation unit 213 as the weight 45 in the storage unit 4 (see FIG. 1). The written weight 45 is used by the model construction unit 22.

<Model building department>
The configuration of the model construction unit 22 will be described with reference to FIG. FIG. 3 is a functional block diagram showing the configuration of the model construction unit. The model construction unit (model construction means) 22 uses the objective function based on the expected error using the weight of the time information (index information) estimated by the weight estimation unit 21 and the input data (series data) 44 to sample A model for predicting an unknown output sample (second observation data) at the latest time (predetermined discrete value of index information) from (prediction sample data) 47 is constructed. As shown in FIG. 3, the model construction unit 22 includes an input data reading unit 221, a weight reading unit 222, a model parameter estimation unit 223, a hyper parameter estimation unit 224, and a model parameter writing unit 225. Yes.

The input data reading unit 221 reads the input data 44 and outputs it to the model parameter estimation unit 223.
The weight reading unit 222 reads the weight 45 and outputs it to the model parameter estimation unit 223.
The model parameter estimation unit (model parameter estimation unit) 223 minimizes the objective function based on the weight 45 read by the weight reading unit 222, the hyperparameter of the weight 45, and the error function in the approximated expected error. The model at the latest time is estimated by learning the model based on the input data (series data) 44 and the weight 45. The model parameter estimation unit 223 uses the hyper parameter estimated by the hyper parameter estimation unit 224 for model estimation. In the present embodiment, the model parameter estimation unit 223 estimates model parameters using the above-described equation (22).

The hyper parameter estimation unit (hyper parameter estimation means) 224 estimates the hyper parameter so as to minimize the error function applied to the model estimated by the model parameter estimation unit 223. In the present embodiment, the hyper parameter estimation unit 224 estimates the hyper parameter using the above-described equation (23). Further, the hyper parameter estimation unit 224 determines whether or not the error shown in Expression (18) converges, and continues to output the hyper parameter at that time to the model parameter estimation unit 223 until the error converges. Thereby, the model parameter estimation unit 223 determines a model estimated using the hyperparameter when the error function is minimized as a model to be constructed. Further, when the hyper parameter estimation unit 224 determines that the error shown in Expression (18) has converged, the hyper parameter estimation unit 224 outputs the model estimated at that time to the model parameter writing unit 225.
The model parameter writing unit 225 writes the estimated model parameter as the model parameter 46 in the storage unit 4 (see FIG. 1). The written model parameter 46 is used by the prediction processing unit 23.

<Prediction processing unit>
The configuration of the prediction processing unit 23 will be described with reference to FIG. FIG. 4 is a functional block diagram illustrating the configuration of the prediction processing unit. The prediction processing unit (prediction processing means) 23 is based on the model (model parameter 46) constructed by the model construction unit 22 and the sample (prediction sample data) 47, and the latest time (index information is predetermined). A process of predicting an unknown output sample (second observation data) of the sample 47 in (discrete values) is performed. As shown in FIG. 4, the prediction processing unit 23 includes a sample reading unit 231, a model parameter reading unit 232, and a prediction result output unit 233.

The sample reading unit 231 reads a sample 47 whose output is unknown and outputs the sample 47 to the prediction result output unit 233.
The model parameter reading unit 232 reads the model parameter 46 and outputs it to the prediction result output unit 233.
The prediction result output unit 233 calculates the prediction result of the output sample (y) using the sample 47 and the model parameter 46, and outputs the calculation result (prediction result) to the output unit 5 (see FIG. 1). .

[Predictor operation]
<Process flow>
The operation of the prediction device 1 shown in FIG. 1 will be described with reference to FIG. 5 (refer to FIG. 1 as appropriate). FIG. 5 is an explanatory diagram showing the flow of processing of the prediction device. First, the prediction device 1 estimates the weight based on the input data 44 stored in advance in the storage unit 4 (see FIG. 1) by the weight estimation unit 21 (step S1: weight estimation step). The estimated weight is stored in the storage unit 4 as the weight 45. Next, the prediction device 1 constructs a model based on the input data 44 and the weight 45 stored in advance in the storage unit 4 (see FIG. 1) by the model construction unit 22 (step S2: model construction estimation step). The constructed model is stored in the storage unit 4 as the model parameter 46. Subsequently, the prediction device 1 predicts a corresponding output based on the model parameter 46 for the sample 47 whose output stored in the storage unit 4 (see FIG. 1) is unknown by the prediction processing unit 23. A prediction process is performed (step S3: prediction process step).

  Next, the weight estimation step in step S1 and the model construction estimation step in step S2 will be described with reference to FIGS. 6 and 7 (refer to FIGS. 1 to 5 as appropriate). FIG. 6 is a flowchart showing the weight estimation process, and FIG. 7 is a flowchart showing the model construction process.

<Weight estimation step>
In the weight estimation step of step S1 described above, as shown in FIG. 6, the weight estimation unit 21 reads the input data 44 from the storage unit 4 (see FIG. 1) by the input data reading unit 211 (step S11). Specifically, the input data reading unit 211 reads input data (series data) having the structure shown in Table 1 described above. Then, the weight estimation unit 21 calculates the experience distribution by using the posterior probability estimation unit 212 (step S12). Specifically, the empirical distribution represented by the above equation (13) is calculated. Then, the weight estimation unit 21 performs initialization by the posterior probability estimation unit 212 (step S13). Specifically, the posterior probability estimation unit 212 sets the number of repetitions τ of the two steps of the E step and the M step of the EM algorithm to 0 (τ = 0), and generates a predetermined random number, for example. A mixture distribution P (t) having a mixture ratio represented by Expression (6) is set at random.

Then, after the initialization is completed, the weight estimation unit 21 uses the posterior probability estimation unit 212 to execute the E step of the EM algorithm (step S14). Specifically, the posterior probability estimation unit 212 estimates the posterior probabilities for all the times t of all the samples (x, y) by the above-described equation (16). Subsequently, the weight estimation unit 21 executes the M step of the EM algorithm by the mixture ratio estimation unit 213 (step S15). Specifically, the mixture ratio estimation unit 213 estimates the mixture distribution P ^{(τ + 1)} (t), that is, the mixture ratio in the ^{(τ + 1) -th} step by the above-described equation (17). Next, the weight estimation unit 21 determines whether or not the convergence condition is satisfied by the mixture ratio estimation unit 213 (step S16). Specifically, the mixture ratio estimation unit 213 determines whether or not the likelihood L (P) shown in the above equation (11) has converged.

  When the convergence condition is satisfied, that is, when the likelihood L (P) shown in the above equation (11) has converged (step S16: Yes), the mixture ratio estimation unit 213 calculates the weight w (t) ( Step S17). Specifically, the mixture ratio estimation unit 213 uses the mixture distribution P (t) and the number of samples N (t) at the time of convergence to calculate the weight w (t) shown in the above equation (8) for all times t. Is calculated. Then, the weight estimation unit 21 writes the weight w (t) shown in the above equation (8) into the storage unit 4 (see FIG. 1) as the weight 45 by the weight writing unit 214 (step S18), and the processing Exit. The weight 45 indicates a weight estimated with respect to time t in order to construct a model at the latest time T.

  On the other hand, when the convergence condition is not satisfied in step S16, that is, when the likelihood L (P) shown in the above equation (11) is not converged (step S16: No), the weight estimation unit 21 determines that E “1” is added to the number of repetitions τ of the step and the M step (τ = τ + 1), the number of repetitions τ is updated (step S19), and the process returns to step S14.

<Model construction step>
In the model building step in step S2, the model building unit 22 first reads input data (series data) and weights as shown in FIG. 7 (step S21). Specifically, the model construction unit 22 reads the input data 44 from the storage unit 4 (see FIG. 1) by the input data reading unit 221. In addition, the model construction unit 22 reads the weight 45 estimated by the weight estimation process from the storage unit 4 (see FIG. 1) by the weight reading unit 222. Note that the execution order of the input data 44 reading and the weight reading is arbitrary, and the processes may be executed in parallel.

  And the model construction part 22 is initialized by the model parameter estimation part 223 (step S22). Specifically, the model parameter estimation unit 223 divides the learning data D into K subsets and sets the hyperparameter λ to 0 (λ = 0). And the model construction part 22 learns a model by the model parameter estimation part 223 (step S23). Specifically, the model parameter estimation unit 223 learns the model at the latest time T by the above-described equation (22) according to the K-fold cross validation method. Then, the model construction unit 22 learns the hyperparameter λ by the hyperparameter estimation unit 224 (step S24). Specifically, the hyper parameter estimation unit 224 learns the hyper parameter λ using the above-described equation (23). And the model construction part 22 discriminate | determines whether the convergence condition was satisfy | filled by the hyper parameter estimation part 224 (step S25). Specifically, the hyper parameter estimation unit 224 determines whether or not the error E (M, λ) shown in the equation (18) has converged.

  When the convergence condition is satisfied, that is, when the error E (M, λ) shown in the equation (18) has converged (step S25: Yes), the model construction unit 22 uses the model parameter writing unit 225 to The model at the time is written as the model parameter 46 in the storage means 4 (see FIG. 1) (step S26), and the process is terminated. The model parameter 46 indicates a model estimated as a model at the latest time T. On the other hand, when the convergence condition is not satisfied in step S25, that is, when the error E (M, λ) shown in the above equation (18) has not converged (step S25: No), the model construction unit 22 The process returns to step S23.

  Note that the prediction device 1 can also be realized by executing a prediction program that causes a general computer to execute the above-described steps. This program can be distributed via a communication line, or can be written on a recording medium such as a CD-ROM for distribution.

  According to the present embodiment, the prediction device 1 estimates useful information for learning a model suitable for the latest time data from past data other than the latest time data by estimating the weight of the time information. By taking in, the experience error at the latest time can be used as an accurate approximation to the expected error at the latest time. Therefore, the prediction apparatus 1 can predict the data at the latest time with high accuracy in the time-series data whose properties change with time. As a result, for example, the purchase tendency of the user at the current time (latest time) can be predicted from the user's historical purchase data, and the predicted product can be recommended to the user. Thereby, it is possible to bring about an effect of improving convenience so that the user can quickly access information on the product desired and an effect of increasing the profit of the product provider.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to this, It can implement in the range which does not change the meaning. For example, in the weight estimation process, the approximation by the above equation (5) is an example, and can be made variable according to the conditions. For example, the approximation by the equation (24) can be used. That is, the conditional distribution P (x | y, t) of the input / output samples at time t is equal to the conditional distribution P (x | y, t ') of the input / output samples at time t ′ different from time t. In a situation where only the output distribution differs depending on the time, the mixing ratio is determined so that the distribution P (y | T) of the output sample y at the latest time T is approximated by a mixture of the empirical distributions of the output samples y at each time. can do. At this time, the mixing ratio satisfying the relational expression of Expression (24) is an approximation of the mixing ratio satisfying the relational expression of Expression (5).

  When Expression (24) is used, the posterior probability estimation unit 212 of the weight estimation unit 21 of the prediction device 1 performs the condition of the output sample (y) on the condition of the mixture ratio and the output sample (y) in the input data 44. The posterior probabilities for the time information t are estimated. Specifically, the posterior probability estimation unit 212 calculates in advance an empirical distribution represented by the equation (13A) instead of the above equation (13). In Equation (13A), the number of samples whose time is t and output is y is N (t, y). Further, in this case, the posterior probability estimation unit 212 estimates the posterior probability for all the times of all samples by using the equation (16A) instead of the above equation (16). Furthermore, in this case, the mixture ratio estimation unit 213 estimates the mixture ratio using Expression (17A) instead of Expression (17) described above.

  Further, in the present embodiment, the error function J of the model when the sample is given is exemplified by the expressions (2) and (3) described above, but any error function can be used. . Furthermore, in the present embodiment, for example, T has been described as being the latest time in the above-described formula (1), but T is not limited to the latest time and can be any time.

  Furthermore, in this embodiment, time-series data whose properties change with time is handled. For example, t is time information in the above-described equation (4), but t is information other than time if it is a discrete variable. May be. In this case, t can be set as, for example, area information (for example, spatial coordinates) indicating a specific place or area. Table 2 shows an example of purchase data associated with the regional information t. As a result, unknown data related to the target area can be predicted with high accuracy by weighting and considering data related to another area different from that area.

  Moreover, the apparatus which comprises the prediction apparatus 1 is not limited to 1 unit | set, You may distribute and arrange | position a function to several apparatus. For example, the weight estimation unit 21, the model construction unit 22, the prediction processing unit 23, and the data storage unit 40b of the storage unit 4 may be configured as separate devices. As a result, the load on each device is distributed, and high-speed processing can be realized.

  In order to confirm the effect of the present invention, the prediction accuracy of the product when inputting the purchase history of the video distribution service and the purchase history of the comic distribution service for mobile phones are input to the prediction device 1 according to the present embodiment. The prediction accuracy of the product was determined by experiment.

<Setting>
The purchase history of the moving image distribution service (hereinafter referred to as moving image data) indicates moving image data in the moving image distribution service from January 1, 2007 to January 31, 2007. In this moving image data, the number of users was “40,282”, the number of products was “5,914”, and the number of purchases was “1,667,414”. The purchase history of the comic distribution service for mobile phones (hereinafter referred to as comic data) indicates comic data in the comic distribution service for mobile phones from April 1, 2005 to March 31, 2006. In this comic data, the number of users was “164,538”, the number of products was “175”, and the number of purchases was “1,018,741”. Here, one comic with multiple volumes was treated as the same product.

  It should be noted that, from the moving image data and the comic data, the products whose sales number is “1” or less are omitted, and the users whose purchase number is “1” or less are omitted. Further, when a certain product is purchased by the same user twice or more, the second and subsequent purchases regarding the product are omitted from the purchase history.

<Formulation>
A primary Markov model was used as the model M in this case. Further, as an error function of the first-order Markov model, negative log likelihood is used as shown in Expression (25). In equation (25), θ _ij is a model parameter that means the probability (0 ≦ θ _ij ≦ 1, Σ _j θ _ij = 1) of purchasing the product j after the product i. Also, product x _n denote the product with the user U _n purchases before that one when a purchase y _n.

  In this case, the error E shown in the equation (18) is rewritten as shown in the equation (26). That is, E shown in Expression (26) is an objective function to be minimized.

Here, the model parameter θ _ij that minimizes the error function J shown in the equation (25) is shown in the equation (27). In Expression (27), β is a smoothing parameter. In the following calculation, β = 10 ⁻² was used. V represents the number of products. In the equation (27), the symbol “」 (hat) ”attached to θ indicates that θ minimizes the error function.

(Example 1, Example 2)
In Example 1 (OurXY), the weight w (t) is set so as to approximate the simultaneous distribution (input / output distribution x, y) at the latest time T based on the above-described equation (5). In Example 2 (OurY), the weight w (t) is set so as to approximate the output distribution y at the latest time T based on the equation (24). In the first embodiment (OurXY) and the second embodiment (OurY), the value of the smoothing parameter α shown in the equation (13) is α = 10 ⁻⁸ , and the weight value at each time t is the maximum weight value. Hyperparameter λ was estimated. Here, the hyperparameter λ is estimated so that the maximum value of the weight is “1”. In addition, λ was estimated by the golden section method so that the generalization error was minimized by the 10-fold cross validation. At this time, the possible section of the hyper parameter λ is [0, 10].

(Comparative Example 1 and Comparative Example 2)
Example 1 (OurXY) and Example 2 (OurY) were compared with the following two models (Comparative Example 1 and Comparative Example 2).
Comparative Example 1 (NoWeight): No change in weight due to date (w (t) = 1)
Comparative Example 2 (Present): Only the latest date has a weight (w (T) = 1, t ≠ T and w (t) = 0)

<Experiment method>
In the experiment, the unit time of time information was 1 day. That is, the latest time means the latest day, that is, the last day. In the experiment, at the first stage, that is, on the first day, the final date T is set as the distribution start date. In the next stage, that is, the second day, the final date T is shifted by one day and set to the next day of the distribution start date. Similarly, the last day T was shifted by one day. At each stage, that is, on each date (each final date T), data on the final date T was created using only the purchase history before the final date T. And in each day T, the performance was evaluated by perplexity L (T) shown in Formula (28).

In Expression (28), N _T is the number of samples on the date T (latest date T) of that date. The perplexity indicates that the prediction performance is high when the value is low. For each date T, first, ten sets of learning and test data were created for the moving image data by 10-fold cross validation, and the perplexity of the date T was determined. Then, as shown in Expression (29), perplexity (average perplexity) L averaged over all days D was obtained. D shown in Expression (29) is D = 31 in the moving image data. Similarly, the average perplexity L was obtained for the comic data. Here, in the comic data, D = 365.

<Experimental result>
Table 2 shows the experimental results for the examples and comparative examples.

  As shown in Table 3, Example 1 (OurXY) and Example 2 (OurY) have a lower average perplexity L than Comparative Example 1 (NoWeight) and Comparative Example 2 (Present). In other words, Example 1 (OurXY) and Example 2 (OurY) have higher prediction performance than Comparative Example 1 (NoWeight) and Comparative Example 2 (Present). That is, in the present invention, it can be said that a model capable of predicting the data of the latest date more accurately can be learned by weighting the time information that is a feature of the present invention. Note that Comparative Example 1 (NoWeight) has low prediction accuracy because data of a past time having a different property from the latest date is also used for learning. Further, Comparative Example 2 (Present) has low prediction accuracy because the number of data used for learning is small.

It is a block diagram which shows the structure of the prediction apparatus which concerns on embodiment of this invention. It is a functional block diagram which shows the structure of a weight estimation part. It is a functional block diagram which shows the structure of a model construction part. It is a functional block diagram which shows the structure of a prediction process part. It is explanatory drawing which shows the flow of a process of a prediction apparatus. It is a flowchart which shows a weight estimation process. It is a flowchart which shows a model construction process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Prediction apparatus 2 Calculation means 3 Input means 4 Storage means 5 Output means 11 Bus line 21 Weight estimation part (weight estimation means)
22 Model building department (model building means)
23 Prediction processing unit (prediction processing means)
24 memory 40a program storage unit 41 weight estimation program 42 model building program 43 prediction processing program 40b data storage unit 44 input data (series data)
45 Weight 46 Model parameter 47 samples (prediction sample data)
211 Input data reading unit 212 A posteriori probability estimation unit (a posteriori probability estimation means)
213 Mixing ratio estimation unit (mixing ratio estimation means)
214 Weight writing unit 221 Input data reading unit 222 Weight reading unit 223 Model parameter estimation unit (model parameter estimation means)
224 Hyper parameter estimation unit (hyper parameter estimation means)
225 Model parameter writing unit 231 Sample reading unit 232 Model parameter reading unit 233 Prediction result output unit

Claims (7)

Input means for inputting series data that is a series of training sample data and sample data for prediction;
Computing means;
Storage means for storing the input series data and the prediction sample data and storing a calculation processing result by the calculation means;
Output means for outputting a prediction result by the calculation means,
The learning sample data includes: first observation data whose properties change according to time or region; second observation data whose properties change according to time or region in relation to the first observation data; Data having three elements, index information indicating discrete values of time or region information corresponding to the first and second observation data,
When the prediction sample data is data in which the second observation data is unknown and the first observation data and the index information are known among the three elements constituting the learning sample data,
The computing means is
In order to approximate the true distribution of the series data at a predetermined discrete value of the index information, a predetermined mixing ratio mixed with an empirical distribution at each discrete value of the index information, Posterior probability estimating means for estimating posterior probabilities for the index information of the first and second observation data, respectively, on the condition of the first and second observation data of predetermined learning sample data;
Using the estimated posterior probabilities, the mixture ratio is estimated so as to maximize the likelihood for the sequence data at the predetermined discrete value of the index information, and the likelihood is maximized. Mixing ratio estimating means for determining the weight of the index information from the mixing ratio when
Weight writing means for writing the weight of the determined index information into the storage means;
The weight of the determined index information is read from the storage means, the objective function based on the expected error using the weight and the series data are used, and the predetermined index information is determined from the prediction sample data. Model construction means for constructing a model for predicting the unknown second observation data at the discrete values obtained and writing the model to the storage means ;
Reading the constructed model and the prediction sample data from the storage means , and based on the constructed model and the prediction sample data, the index information in the predetermined discrete value A prediction apparatus comprising: a prediction processing unit that performs a process of predicting the unknown second observation data of the prediction sample data. Input means for inputting series data that is a series of training sample data and sample data for prediction;
Computing means;
Storage means for storing the input series data and the prediction sample data and storing a calculation processing result by the calculation means;
Output means for outputting a prediction result by the calculation means,
The learning sample data includes: first observation data whose properties change according to time or region; second observation data whose properties change according to time or region in relation to the first observation data; Data having three elements, index information indicating discrete values of time or region information corresponding to the first and second observation data,
The series data is a series of learning sample data in which the distribution of the first observation data x when the second observation data y is given is similar in the discrete values of the different index information,
When the prediction sample data is data in which the second observation data is unknown and the first observation data and the index information are known among the three elements constituting the learning sample data,
The computing means is
To approximate the true distribution of the sequence data in the predetermined discrete values of the index information, and a predetermined mixing ratio to be mixed with the empirical distribution for the discrete values of the index information, in the sequence data Posterior probability estimating means for estimating posterior probabilities for the index information of the second observation data on the condition of the second observation data of predetermined learning sample data;
Using the estimated posterior probabilities, the mixture ratio is estimated so as to maximize the likelihood for the sequence data at the predetermined discrete value of the index information, and the likelihood is maximized. mixing ratio estimating means for determining the weight of the index information from the mixing ratio when reduction,
Weight writing means for writing the weight of the determined index information into the storage means;
The weight of the determined index information is read from the storage means, the objective function based on the expected error using the weight and the series data are used, and the predetermined index information is determined from the prediction sample data. Model construction means for constructing a model for predicting the unknown second observation data at the discrete values obtained and writing the model to the storage means;
Reading the constructed model and the prediction sample data from the storage means, and based on the constructed model and the prediction sample data, the index information in the predetermined discrete value prediction apparatus comprising: a prediction processing means for performing processing to predict the unknown second observation data of the prediction for the sample data. The model construction means includes
Based on the sequence data and the estimated weight so as to minimize an objective function based on the weight of the estimated index information, the hyperparameter of the weight, and the error function in the approximated expected error Model parameter estimation means for estimating a model at the predetermined discrete value by learning a model;
Hyperparameter estimation means for estimating the hyperparameter so as to minimize the error function applied to the estimated model,
The model parameter estimation means includes:
Predicting apparatus according to claim 1 or claim 2, characterized in that determining a model is to be constructed estimated model using the hyper parameters when the error function is minimized. Input means for inputting series data that is a series of training sample data and sample data for prediction;
Computing means;
Storage means for storing the input series data and the prediction sample data and storing a calculation processing result by the calculation means;
Output means for outputting a prediction result by the computing means, and a prediction method of a prediction device comprising:
The learning sample data includes: first observation data whose properties change according to time or region; second observation data whose properties change according to time or region in relation to the first observation data; Data having three elements, index information indicating discrete values of time or region information corresponding to the first and second observation data,
When the prediction sample data is data in which the second observation data is unknown and the first observation data and the index information are known among the three elements constituting the learning sample data,
The calculation means of the prediction device includes:
In order to approximate the true distribution of the series data at a predetermined discrete value of the index information, a predetermined mixing ratio mixed with an empirical distribution at each discrete value of the index information, Estimating the posterior probabilities for the index information of the first and second observation data, respectively, on the condition of the first and second observation data of predetermined learning sample data;
Using the estimated posterior probabilities, the mixture ratio is estimated so as to maximize the likelihood for the sequence data at the predetermined discrete value of the index information, and the likelihood is maximized. Determining the weight of the indicator information from the mixture ratio when
Writing the weight of the determined index information into the storage means;
The weight of the determined index information is read from the storage means, the objective function based on the expected error using the weight and the series data are used, and the predetermined index information is determined from the prediction sample data. A model construction step of constructing a model for predicting the unknown second observation data at a given discrete value and writing the model to the storage means ;
Reading the constructed model and the prediction sample data from the storage means , and based on the constructed model and the prediction sample data, the index information in the predetermined discrete value And a prediction processing step of performing a process of predicting the unknown second observation data of the sample data for prediction. Input means for inputting series data that is a series of training sample data and sample data for prediction;
Computing means;
Storage means for storing the input series data and the prediction sample data and storing a calculation processing result by the calculation means;
Output means for outputting a prediction result by the computing means, and a prediction method of a prediction device comprising:
The learning sample data includes: first observation data whose properties change according to time or region; second observation data whose properties change according to time or region in relation to the first observation data; Data having three elements, index information indicating discrete values of time or region information corresponding to the first and second observation data,
  The series data is a series of learning sample data in which the distribution of the first observation data x when the second observation data y is given is similar in the discrete values of the different index information,
When the prediction sample data is data in which the second observation data is unknown and the first observation data and the index information are known among the three elements constituting the learning sample data,
The calculation means of the prediction device includes:
In order to approximate the true distribution of the series data at a predetermined discrete value of the index information, a predetermined mixing ratio mixed with an empirical distribution at each discrete value of the index information, Estimating each of the posterior probabilities for the index information of the second observation data on the condition of the second observation data of predetermined learning sample data;
Using the estimated posterior probabilities, the mixture ratio is estimated so as to maximize the likelihood for the sequence data at the predetermined discrete value of the index information, and the likelihood is maximized. Determining the weight of the indicator information from the mixture ratio when
Writing the weight of the determined index information into the storage means;
The weight of the determined index information is read from the storage means, the objective function based on the expected error using the weight and the series data are used, and the predetermined index information is determined from the prediction sample data. A model construction step of constructing a model for predicting the unknown second observation data at a given discrete value and writing the model to the storage means;
Reading the constructed model and the prediction sample data from the storage means, and based on the constructed model and the prediction sample data, the index information in the predetermined discrete value And a prediction processing step of performing a process of predicting the unknown second observation data of the sample data for prediction. A prediction program for causing a computer to execute the prediction method according to claim 4 or 5.   A computer-readable recording medium on which the prediction program according to claim 6 is recorded.

Images