JP2022162456A - Learning method, prediction method, learning device, prediction device, and program - Google Patents

Abstract

To provide a learning method that predicts an event sequence in a plurality of domains while considering an indirect relationship through a common latent factor.SOLUTION: A learning method according to one embodiment causes a computer to execute an acquisition step for acquiring event history information representing a history of events in a plurality of domains, and a learning step for learning, using the event history information acquired in the acquisition procedure, a parameter of an intensity function represented by a first composite function, which is obtained by combining an integral of a black-box function representing a temporal change in a latent state common among the plurality of domains and a trigger function, the black box function, and a background rate.SELECTED DRAWING: Figure 2

Description

The present invention relates to a learning method, a prediction method, a learning device, a prediction device, and a program.

A probabilistic model called a point process is widely used for modeling event sequences. A point process is a model for describing the number of occurrences of events in a minute interval, and describes the probability of occurrence of an event at an arbitrary time using an "intensity function".

Among event sequences, it is known that event sequences such as crimes, financial transactions, occurrence histories of demonstrations and strikes, etc. in particular have "transmissibility". Propagation is the property that an event causes another event, which in turn causes another event, and so on. For example, when a demonstration occurs in a certain area, another demonstration occurs in response to it, and the demonstration spreads to another area. The Hawkes process is widely used for modeling such a propagating event series. A Hawkes process is a kind of point process, and by modeling the mechanism of propagation with a function called a kernel function, it is possible to describe an event sequence with propagating properties.

Here, Hawkes processes are usually focused on application to a single domain and cannot be directly applied to modeling event sequences in multiple domains. On the other hand, a model called "multitask Hawkes process" that enables modeling of event sequences in multiple domains has been proposed (Non-Patent Document 1).

Luo, Dixin, et al. Multi-task multi-dimensional hawkes processes for modeling event sequences. (2015).

However, the model proposed in Non-Patent Document 1 above makes a strong assumption that when an event occurs in one domain, the event occurrence probability in another domain increases accordingly. Therefore, it is not possible to describe indirect relationships between multiple domains through common latent factors. On the other hand, in reality, the probability of occurrence of events for each domain is thought to be indirectly related via common latent factors. For example, two events in different domains, crime and financial transactions, may be related through a common latent factor of economic change.

An embodiment of the present invention has been made in view of the above points, and aims to predict event sequences in a plurality of domains, taking into consideration indirect relationships via common latent factors.

To achieve the above object, a learning method according to one embodiment includes an acquisition procedure for acquiring event history information representing a history of events in a plurality of domains; Learning the parameters of the intensity function represented by the first composite function of the integral of the black box function representing the time change of the latent state common between domains and the trigger function, the black box function, and the background rate. and a learning procedure to be performed by the computer.

Indirect relationships via common latent factors can also be considered to predict event sequences in multiple domains.

It is a figure which shows an example of event history information. It is a figure showing an example of whole composition of an event prediction device concerning this embodiment. 6 is a flowchart showing an example of parameter learning processing according to the embodiment; It is a flow chart which shows an example of prediction processing concerning this embodiment. It is a figure which shows an example of the hardware constitutions of the event prediction apparatus which concerns on this embodiment.

An embodiment of the present invention will be described below. In this embodiment, an event prediction device 10 that builds a point process model that takes into account indirect relationships between multiple domains via common latent factors and predicts the occurrence of events between multiple domains based on this point process model will be described. do.

Here, the point process model is generally described by a function called "intensity function" (or "strength function"), and this intensity function represents the probability of occurrence of an event. In this embodiment, by introducing a function representing changes in the latent state and using it to describe the intensity function of the Hawkes process, it is possible to consider indirect relationships between multiple domains through common latent factors. to This enables accurate modeling of event sequences in multiple domains and enables highly accurate prediction of future event occurrences.

<Theoretical configuration>
First, the theoretical configuration of this embodiment will be described. In this embodiment, it is assumed that event history information representing _a history of events from time _t1 to time tN is given. The event history information is data represented by time series, and is data representing, for example, the history of crimes, the history of financial transactions, the history of demonstrations and strikes, and the like. Below, the event history information to be analyzed by the event prediction device 10 is

と表す。ここで、ｔ_ｉは時刻（イベント発生時刻）、ｍ_ｉは時刻ｔ_ｉに発生したイベントのドメイン、Ｎはデータ数（イベント数）である。また、ドメインの数をＭとする。

is represented as Here, t _i is time (event occurrence time), m _i is the domain of the event that occurred at time t _i , and N is the number of data (number of events). Also, let M be the number of domains.

A specific example of event history information is shown in FIG. The example shown in FIG. 1 is event history information composed of an event sequence of the domain "crime", an event sequence of the domain "financial transaction", and an event sequence of the domain "strike". That is, the event history information _{ (t _i , m _i )} shown in FIG. 1 is for each _i =1, . event has occurred.

At this time, in this embodiment, given event history information is used to learn the parameters of the point process model.

In accordance with the procedure of general point process model, we first design the intensity function. The intensity function is a function representing the probability of an event occurring (or generating) per unit time.

In this embodiment, the intensity function λ _m (·) of the domain m is represented by the following equation (1).

ここで、μ_ｍは「バックグラウンドレート」と呼ばれ、過去のイベントに依らないイベントの発生確率を表す。本実施形態では、簡単のため、時間変化しない定数μ_ｍを用いるが、以下の説明はμ_ｍが時間に依存して変化する場合にも容易に一般化することが可能である。また、ｇ_ｍ（・）は既存のＨａｗｋｅｓ過程モデルで用いられるトリガー関数であり、例えば、指数減衰関数、ワイブル分布、ガンマ分布等が広く用いられている。本実施形態では、ドメインｍごとに異なるバックグラウンドレートとトリガー関数を仮定する。ｆ（ｔ）は潜在状態の時間変化を表す任意のブラックボックス関数であり、例えば、景気の変化等を表す。上記の式（１）に示す定式化は、犯罪や金融取引等の複数ドメインにまたがるイベントの発生が共通の潜在状態ｆ（ｔ）によって時間変化するという仮定に基づいている。

Here, _μm is called a “background rate” and represents the occurrence probability of an event that does not depend on past events. In this embodiment, for the sake of simplicity, a constant _μm that does not change with time is used, but the following description can be easily generalized to the case where _μm changes depending on time. Also, g _m (·) is a trigger function used in the existing Hawkes process model, and for example, an exponential decay function, Weibull distribution, gamma distribution, etc. are widely used. In this embodiment, we assume different background rates and trigger functions for each domain m. f(t) is an arbitrary black-box function that expresses the temporal change of the latent state, and represents, for example, changes in economic conditions. The formulation shown in Equation (1) above is based on the assumption that the occurrence of events across multiple domains, such as crime and financial transactions, changes over time according to a common latent state f(t).

は現在時刻ｔと過去のイベントの発生時刻ｔ_ｊに依存する関数であり、以下の式（２）でモデル化される。

is a function that depends on the current time t and the past event occurrence time _tj , and is modeled by the following equation (2).

ここで、Ｆ（ｔ）は潜在状態の時間変化を表す関数ｆ（ｔ）の積分である。本実施形態では、Ｆ（ｔ）をニューラルネットワークによりモデル化する。この定式化により、尤度関数をニューラルネットワーク関数Ｆ（ｔ）とその微分ｆ（ｔ）のみを用いて書き下すことが可能になる。

Here, F(t) is the integral of the function f(t) representing the time change of the latent state. In this embodiment, F(t) is modeled by a neural network. This formulation allows the likelihood function to be written down using only the neural network function F(t) and its derivative f(t).

Event sequence up to time T

が与えられたとき、本実施形態における点過程モデルの尤度Ｌは以下の式（３）のように書き下すことができる。

is given, the likelihood L of the point process model in this embodiment can be written as the following equation (3).

Ｈａｗｋｅｓ過程の学習においては、上記の式（３）のΛ_ｉに含まれる積分が問題になる。ニューラルネットワーク関数を含む関数の積分は一般に困難であるが、上記の式（１）及び（２）の定式化によりΛ_ｉの積分を解析的に解くことができる。すなわち、ｘ＝Ｆ（ｔ）－Ｆ（ｔ_ｊ）と変数変換を行うことで、ｆ（ｔ）ｄｔ＝ｄｘの置き換えが可能になる。これをΛ_ｉに代入することで、以下の式（４）が得られる。

In the learning of the Hawkes process, the integral included in Λ _i of the above equation (3) becomes a problem. Integrating functions, including neural network functions, is generally difficult, but the formulation of equations (1) and (2) above allows the integral of Λ _i to be solved analytically. That is, by performing the variable transformation x=F(t)-F(t _j ), f(t)dt=dx can be replaced. By substituting this for Λ _i , the following equation (4) is obtained.

ここで、Ｇ_ｍ（・）はトリガー関数ｇ_ｍ（・）の積分であり、指数減衰関数、ワイブル分布、ガンマ分布等の多くのトリガー関数ｇ_ｍ（・）について解析解が得られる。

Here, G _m (·) is the integral of the trigger function g _m (·), and analytical solutions can be obtained for many trigger functions g _m (·) such as exponential decay function, Weibull distribution, and gamma distribution.

Then, during learning, parameters of the neural network function f(·) that minimize the likelihood L shown in the above equation (3) are estimated. Any known optimization method may be used for optimizing the parameters. Since the likelihood L shown in the above equation (3) is differentiable with respect to all parameters, it can be optimized using, for example, the gradient method. Note that the differentiation of the likelihood L can be calculated by, for example, the error backpropagation method.

<Overall composition>
Next, the overall configuration of the event prediction device 10 according to this embodiment will be described with reference to FIG. FIG. 2 is a diagram showing an example of the overall configuration of the event prediction device 10 according to this embodiment.

As shown in FIG. 2, the event prediction device 10 according to the present embodiment includes an acquisition unit 101, a parameter learning unit 102, a designation reception unit 103, a prediction unit 104, an output unit 105, and a parameter storage unit 106. have

The acquisition unit 101 acquires event history information from the event history information storage device 20 connected to the event prediction device 10 via a communication network.

Here, the event history information storage device 20 is, for example, a web server or database server that stores event history information. The operation (registration, deletion, correction, etc.) of the event history information stored in the event history information storage device 20 can be performed, for example, by a terminal connected to the event history information storage device 20 via a communication network (this terminal can be It may be the event prediction device 10 or the event history information storage device 20 itself).

The parameter learning unit 102 uses the event history information acquired by the acquisition unit 101 to obtain the parameters of the intensity function λ _m (t) shown in the above equation (1) (that is, the intensity function λ _m (t) parameters) of the neural network function f(t) embedded in . At this time, the parameter learning unit 102 learns the parameter by minimizing the likelihood L shown in the above equation (3) by any known optimization method (eg, gradient method, etc.). Parameters learned by parameter learning section 102 (learned parameters) are stored in parameter storage section 106 .

The designation receiving unit 103 receives designation of a prediction target time when predicting the occurrence of an event in the domain m using the intensity function λ _m (t) set with the learned parameters. For example, depending on the type of event, designation of information other than time may be accepted (for example, when predicting the occurrence of a crime, designation of information indicating the region to be predicted may be accepted). .).

The prediction unit 104 predicts the occurrence of an event at the time received by the designation receiving unit 103 using the intensity function λ _m (t) set with the learned parameters. At this time, the prediction unit 104 predicts the occurrence of an event by, for example, calculating the event occurrence probability up to the prediction target time using the intensity function λ _m (t) and performing a point process simulation. There are various methods for performing point process simulation, and for example, a method called thinning can be used. For thinning, see, for example, the reference "OGATA, Yoshihiko. On Lewis' simulation method for point processes. IEEE Transactions on Information Theory, 1981, 27.1: 23-31."

The output unit 105 outputs the result of prediction by the prediction unit 104 . Note that the output unit 105 may output the output result to an arbitrary output destination. For example, the output unit 105 may display the prediction result on a display or the like, store it in a storage area such as an auxiliary storage device, print it from a printer or the like, or output it as sound from a speaker or the like. It may be output or may be transmitted to an external device via a communication network.

Note that the configuration of the event prediction device 10 shown in FIG. 2 is an example, and other configurations may be used. For example, instead of acquiring the event history information from the event history information storage device 20, the event prediction device 10 may hold the event history information. Further, for example, the event prediction device 10 may be configured by a device (learning device) that executes parameter learning processing, which will be described later, and a device (prediction device) that executes prediction processing, which will be described later.

<Parameter learning processing>
Next, the flow of processing for learning the parameters of the intensity function λ _m (t) shown in Equation (1) above (that is, the parameters of the neural network function f(t)) will be described with reference to FIG. . FIG. 3 is a flowchart showing an example of parameter learning processing according to this embodiment.

First, the acquisition unit 101 acquires event history information from the event history information storage device 20 (step S101). At this time, the user of the event prediction device 10 may specify, for example, a range (for example, a temporal range, a locational range, etc.) to be acquired as event history information.

Next, the parameter learning unit 102 learns the parameters of the intensity function λ _m (t) shown in Equation (1) above using the event history information acquired in step S101 (step S102). At this time, parameter learning section 102 minimizes the likelihood L shown in the above equation (3) by any known optimization method, thereby increasing the intensity function λ _m ( Learn the parameters of t).

Then, the parameter learning unit 102 stores the parameters (learned parameters) learned in step S102 in the parameter storage unit 106 (step S103). As a result, it becomes possible to consider indirect relationships between multiple domains through common latent factors, and the intensity function λ _m (t) set with learned parameters enables highly accurate event occurrence in domain m. Prediction becomes possible.

<Prediction processing>
Next, the flow of processing for predicting the occurrence of an event in domain m using the intensity function λ _m (t) set with learned parameters will be described with reference to FIG. FIG. 4 is a flowchart showing an example of prediction processing according to this embodiment.

First, the designation receiving unit 103 receives designation of a time to be predicted (step S201). The time to be predicted can be specified by the user on a UI (user interface) displayed on the display of the event prediction device 10, for example. Also, at this time, if only a specific domain m is to be predicted, the designation of that domain m may be accepted.

Next, the prediction unit 104 uses the intensity function λ _m (t) to which the learned parameters stored in the parameter storage unit 106 are set, to determine the occurrence of an event at the time accepted in step S201 above. is predicted (step S202).

Then, the output unit 105 outputs the prediction result in step S202 to a predetermined output destination (step S203).

<Hardware configuration>
Finally, the hardware configuration of the event prediction device 10 according to this embodiment will be described with reference to FIG. FIG. 5 is a diagram showing an example of the hardware configuration of the event prediction device 10 according to this embodiment.

As shown in FIG. 5, the event prediction device 10 according to this embodiment is realized by a general computer or computer system, and includes an input device 301, a display device 302, an external I/F 303, a communication I/F 304, It has a processor 305 and a memory device 306 . Each of these pieces of hardware is communicably connected via a bus 307 .

The input device 301 is, for example, a keyboard, mouse, touch panel, or the like. The display device 302 is, for example, a display. Note that the event prediction device 10 does not have to have at least one of the input device 301 and the display device 302 .

An external I/F 303 is an interface with an external device. The external device includes a recording medium 303a and the like. The event prediction device 10 can read from and write to the recording medium 303 a via the external I/F 303 . The recording medium 303a stores, for example, one or more programs that realize each function unit (acquisition unit 101, parameter learning unit 102, designation reception unit 103, prediction unit 104, output unit 105, etc.) of the event prediction device 10. may have been

Note that the recording medium 303a includes, for example, a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), a USB (Universal Serial Bus) memory card, and the like.

Communication I/F 304 is an interface for connecting event prediction device 10 to a communication network. The event prediction device 10 can acquire event history information from the event history information storage device 20 via the communication I/F 304 . Note that one or more programs that implement each functional unit of the event prediction device 10 may be acquired (downloaded) from a predetermined server device or the like via the communication I/F 304 .

The processor 305 is, for example, various arithmetic units such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). Each functional unit of the event prediction device 10 is implemented by processing that one or more programs stored in the memory device 306 or the like cause the processor 305 to execute.

The memory device 306 is, for example, various storage devices such as HDD (Hard Disk Drive), SSD (Solid State Drive), RAM (Random Access Memory), ROM (Read Only Memory), and flash memory. The parameter storage unit 106 included in the event prediction device 10 can be implemented using the memory device 306 .

The event prediction device 10 according to the present embodiment can implement the above-described parameter learning processing and prediction processing by having the hardware configuration shown in FIG. Note that the hardware configuration shown in FIG. 5 is an example, and the event prediction device 10 may have other hardware configurations. For example, the event prediction device 10 may have multiple processors 305 and may have multiple memory devices 306 .

The present invention is not limited to the specifically disclosed embodiments described above, and various variations, modifications, combinations with known techniques, etc. are possible without departing from the scope of the claims. be.

10 event prediction device 20 event history information storage device 101 acquisition unit 102 parameter learning unit 103 designation reception unit 104 prediction unit 105 output unit 106 parameter storage unit 301 input device 302 display device 303 external I/F
303a recording medium 304 communication I/F
305 processor 306 memory device 307 bus

Claims (8)

an acquisition procedure for acquiring event history information representing a history of events in multiple domains;
using the event history information acquired in the acquisition step, a first composite function of an integral of a black box function representing temporal changes in a latent state common among the plurality of domains and a trigger function, and the black box function; a learning procedure for learning the parameters of the intensity function represented by , the background rate and
a computer-implemented learning method. 2. The learning method of claim 1, wherein the trigger function and the background rate are different for each domain. 3. A learning method according to claim 1 or 2, wherein the integral of the black-box function is a neural network function. The learning procedure includes:
By analytically solving the integral of the product of the second composite function of the trigger function and the black box function and the black box function by variable transformation, the second composite function and the black box function 4. A learning method according to any one of claims 1 to 3, wherein the parameters of the intensity function are learned by minimizing the likelihood consisting of the integral of the product and the background rate. an acquisition procedure for acquiring event history information representing a history of events in multiple domains;
using the event history information acquired in the acquisition step, a first composite function of an integral of a black box function representing temporal changes in a latent state common among the plurality of domains and a trigger function, and the black box function; a learning procedure for learning the parameters of the intensity function represented by , the background rate and
A prediction procedure for predicting the occurrence of future events in the plurality of domains by an intensity function set with parameters learned in the learning procedure;
is a computer-implemented prediction method. an acquisition unit that acquires event history information representing the history of events in multiple domains;
using the event history information acquired by the acquisition unit, a first composite function of an integral of a black box function representing temporal changes in a latent state common to the plurality of domains and a trigger function, and the black box function; a learning unit for learning the parameters of the intensity function represented by , background rate and
A learning device having an acquisition unit that acquires event history information representing the history of events in multiple domains;
using the event history information acquired by the acquisition unit, a first composite function of an integral of a black box function representing temporal changes in a latent state common to the plurality of domains and a trigger function, and the black box function; a learning unit for learning the parameters of the intensity function represented by , background rate and
a prediction unit that predicts the occurrence of future events in the plurality of domains using an intensity function set with parameters learned by the learning unit;
A prediction device with A program that causes a computer to execute the learning method according to any one of claims 1 to 4 or the prediction method according to claim 5.

Images