JP4801566B2 - Data stream monitoring device, data stream monitoring method, program thereof, and recording medium - Google Patents

Description

  The present invention relates to a technique (stream mining technique) for analyzing data that continuously flows in a large amount in real time.

  In recent years, technologies related to computer devices and networks have been developed, and a large amount of data is often continuously distributed on the network. Data that continuously circulates in large quantities on a network is called a data stream. Examples of data streams include financial data such as stock prices on the Internet and sensors in sensor networks such as a LAN (Local Area Network). Data (such as temperature sensor and illuminance sensor measurement data).

  Stream mining technology does not analyze the large amount of data stored in the database, but must analyze a large amount of continuously received data in real time, thereby speeding up computation and saving memory. It is necessary to plan. In addition, necessary information can be quickly provided to the user.

  As an analysis of the data stream, for example, there is a method of finding out by partial sequence matching whether or not a sequence somewhat similar to a certain query sequence (predetermined data string) exists in the data stream. In this case, since the relative aging speed may differ between the data stream and the query sequence depending on the sampling rate of the data stream, it is desirable to perform matching taking into account the expansion and contraction of the data in the time axis direction.

Therefore, as a dynamic programming method, dynamic time warping, which is a technology that performs sequence matching while adjusting the time axis direction so as to minimize the distance between sequences, taking into account the expansion and contraction in the time axis direction between sequences. DTW (Dynamic Time Warping) is widely used.
A matrix created and stored based on this DTW is called a time warping matrix, and a DTW distance that is a distance between two sequences can be calculated by the time warping matrix. The smaller the DTW distance, the more similar the two sequences are.

For example, Non-Patent Document 1 and Non-Patent Document 2 describe techniques related to speeding up detection of similar partial sequences based on DTW.
Non-Patent Document 3 describes a technique related to StatStream that improves accuracy by considering a correlation coefficient between data in a data stream as a data stream real-time analysis technique.
Further, Non-Patent Document 4 describes a technique related to BRAID, which is an algorithm for detecting a delay correlation between data, which is used in a real-time analysis method of a data stream.
Emonn Keogh, Exact Indexing of Dynamic Time Warping, Proceedings of the Twenty-eighth International Conference on VLDB (Very Large Data Bases), China, August 2002, pp. 406-417 Yasushi Sakurai, Masatoshi Yoshikawa, Christos Faloutsos, FTW: Fast Similarity Search under the Time Warping Distance, Proceedings of Symposium on PODS (Principles of Database Systems), USA, June 2005, pp.326-337 Yunyue Zhu, Dennis Shasha, StatStream: Statistical Monitoring of Thousands of Data Streams in Real Time, Proceedingsof the Twenty-eighth International Conference on VLDB (Very Large Data Bases), China, August 2002, pp. 358-369 Yasushi Sakurai, Spiros Papadimitriou, Christos Faloutsos, BRAID: StreamMining through Group Lag Correlations, Proceedings of ACM SIGMOD (Association For Computing Machinery Special Interest Group On Management Of Data), USA, June 2005, pp.599-610

However, Non-Patent Document 1 and Non-Patent Document 2 are techniques applied to accumulated data sets, and are not techniques for real-time analysis of data streams.
Non-Patent Document 3 and Non-Patent Document 4 perform sequence matching without adjusting in the time axis direction, and are not related to data stream monitoring using DTW.

When data stream monitoring using DTW is attempted, the amount of calculation and the amount of memory used increase in proportion to the length of the data stream, which is not practical (details are for carrying out the invention). (See description of comparative example of best mode).
Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to reduce calculation costs (calculation amount and memory use amount) in data stream monitoring using DTW.

In order to solve the above problems, the inventions according to claim 1 and claim 4 perform dynamic time warping on a partial sequence similar to a query sequence which is a predetermined data string from a data stream which is continuously received data. In the data stream monitoring device that detects based on the distance, when the storage unit receives the data x _t at time t in the data stream, the sequence of length m = (y ₁ , y ₂ ,... y _m) data _{y i} in query sequence is (where, i = 1,2, ···, m ) information on the distance between the data _{x t-1} at time t-1 in the data stream d (t −1, i) and information s (t−1, i) related to the start time corresponding to the information d (t−1, i) related to the distance are simply included in the (t−1, i) element. One time warping matrix, and stores a predetermined threshold value ε to a dynamic time warping distance.
Then, the processing unit, when receiving the data x _t, using the following formula stored time warping matrix in the storage unit (11) to (14), the data y _i in query sequence _(where, i = 1, 2,..., M) and the information d (t, i) regarding the distance between the data x _t and the d (t, i) calculated by the following equation (15). information s (t, i) relating to the start time to calculate the, update the time warping matrix stored in the storage unit, information relating to the distance between the data y _m and the data x _t in query sequence d (t when m) is equal to or less than a predetermined threshold epsilon, the starting time s (t, m) a starting time _{t s} corresponding to the information d (t, m) relating to the distance, the time t and the end time _{t e} partial sequence _X in the data stream _{[t s:} t e] the class It is detected as a partial sequence that.

According to this invention, since the time warping matrix is single, the calculation cost can be reduced. Further, when a similar partial sequence is detected, the start time can be specified.

In the inventions according to claims 2 and 5 , when the processing unit detects a similar partial sequence X [t _s : t _e ] , it further calculates a value of each matrix element of the time warping matrix. , The similar partial sequence when it is determined that none of the partial sequences whose time zone overlaps with the similar partial sequence X [t _s : t _e ] has a smaller dynamic time warping distance X [t _s : t _e ] is detected as a more appropriate partial sequence.

  According to this invention, it is possible to determine that the similar partial sequence is a more appropriate partial sequence by further examining the partial sequence whose time zone overlaps with the similar partial sequence.

The invention according to claims 3 and 6, processing unit, similar to the partial sequence _X: when detecting _[t s t e], partial sequence X that similar _{[t s:} t e] last data If the reception time is the end time, in a column of the time warping matrix after the end time, among the matrix elements, the dynamics of the partial sequences whose start time is before the end time and whose values are similar when there is no less than the time warping distance, the similar parts sequence _X: and detecting as [t s t _e] of the more appropriate.

  According to this invention, it is possible to determine that the partial sequence is more appropriate by examining each matrix element in a certain column of the time warping matrix after the end time of the similar partial sequence as described above.

The invention according to claim 7 is a program for executing the data stream monitoring method according to the computer of claims 4 to any one of claims 6.

  According to this invention, the computer can execute the data stream monitoring method.

The invention according to an eighth aspect is a recording medium that records the program according to the seventh aspect.

  According to this invention, the program of the data stream monitoring method can be recorded.

  According to the present invention, calculation cost can be reduced in data stream monitoring using DTW.

Hereinafter, the best mode for carrying out a data stream monitoring apparatus according to the present invention (hereinafter referred to as an embodiment) will be described with reference to the drawings as appropriate. Note that drawings other than the reference diagram are also referred to as appropriate.
Before that, a comparative example (prior art) of data stream monitoring using DTW will be described with reference to FIGS. 8 and 9 for easy understanding.

FIG. 8 is a diagram showing an image of query processing in a comparative example of data stream monitoring using DTW.
In FIG. 8, a sequence Y is data of a fixed length m, and is data that is a source of sequence similarity determination.

The sequence X, which is a data stream, is a sequence that is constantly expanding (the amount of data is increasing), the length of which is n, and data that is the target of sequence similarity determination.
TWM (Time Warping Matrix) 1 to S are time warping matrices starting from time t = 1 to t = S, respectively. Further, the connection of the black cells shown in TWMS is a warping path (route to be followed for calculating the DTW distance described later).

  Here, the purpose is to detect a partial sequence similar to the sequence Y in the sequence X based on the DTW distance. Hereinafter, the outline will be described with reference to FIG. 8, and then a specific example will be described with reference to FIG. There are two similar partial sequence detection methods, which are referred to as a first method and a second method. First, the first method will be described.

(First method)
The DTW distance between the two sequences is a distance after the two sequences are adjusted, in whole or in part, in the time axis direction. The DTW distance can be calculated based on the time warping matrix.
DTW distance D () between sequence Y = (y ₁ , y ₂ ,..., Y _m ) of length _m and sequence X = (x ₁ , x ₂ ,..., X _n ) of length _n X, Y) can be obtained by the following equations (1) to (4). Note that t = (1, 2,..., N) and i = (1, 2,..., M).

D (X, Y) = f (n, m) (1)
f (t, i) = || x _t −y _i || + min {f (t, i−1), f (t−1, i),
f (t-1, i-1)} (2)
f (0,0) = 0 (3)
f (t, 0) = f (0, i) = ∞ (4)

Equation (1) is a definition of the DTW distance. Formula (2) is a specific calculation formula.
In Expression (2), || x _t −y _i || represents the distance between two numerical values (x _t and y _i ). As the distance between the two numerical values, for example, the Euclidean distance and the Manhattan distance can be considered. In the n-dimensional space, the coordinates of two points a and b are a (a ₁ , a ₂ ,..., a _n ), b (b ₁ , b ₂ ,..., b _n ), If (1 ≦ j ≦ n), the Euclidean distance is √ (Σ (a _j −b _j ) ² ), and the Manhattan distance is a distance represented by Σ | a _j −b _j |.
In the following specific example, in order to facilitate calculation, the square value of the Euclidean distance is calculated and used as || x _t −y _i ||.

In the expression (2), min {f (t, i−1), f (t−1, i), f (t−1, i−1)} is among the three values in {}. Means to adopt the smallest one. When there are two or three minimum ones, a value that makes the detected partial sequence longer (or shorter) is selected, or the detected partial sequence becomes the query sequence Y. The selection criteria may be set in advance, such as selecting a value that is close.
Equations (3) and (4) are boundary conditions in the time warping matrix used when calculating these three values.

  The time warping matrix holds the value of the DTW function, that is, the value of f (t, i) in the above equation (2). In the comparative example, when calculating the distance between the sequence X of length n and the sequence Y of length m, since n time warping matrices are used, O (nm) is calculated (proportional to the product of n and m). It will take a long time. In other words, in this method, if the value of n increases, the calculation time becomes too long and cannot be practically used.

  Note that the memory usage is O (m). This is because the time warping matrix information can be stored only in two columns of information: the current time column information (a set of matrix elements at that time in the time warping matrix) and the immediately preceding time column information. This is because the previous information can be erased sequentially.

In addition, _X: the _[t s t _e], a partial sequence from time _{t s} to time _{t e. That, X = (x 1, x} 2, ···, x s, ···, x e, ···, x n) _{in, X [t s: t e} ] = (x s, ··· , X _e ). The purpose here is, a fixed length m of the query sequence Y and high similarity to a _X: is to discover _[t s t _e] (Detection).

In the first method, previously set a predetermined threshold epsilon, X satisfying the following equation _(5): detecting the _[t s t _{e].
D (X [t s: t} e], Y) ≦ ε ··· (5)

A specific example of the first method will be described with reference to FIG. FIG. 9 is a diagram illustrating an example of a time warping matrix in the comparative example.
Here, an object is to detect a partial sequence having a high similarity to the sequence Y = (11, 6, 9, 4) in the sequence X that expands with time. 9 (1) to (7), the time warping matrix (TWM) 1 to 1 when the length of the sequence X is “7” (X = (5, 12, 6, 10, 6, 5, 1)). 7 (each inside a thick line). Note that the threshold ε = 20.

Further, when the above formulas (1) to (4) are transformed into formulas for actually calculating the values of the elements (each matrix element) of the TWMs 1 to 7, the following formulas (6) to (9) are obtained. It becomes like this.
That is, in the t-th time warping matrix (time warping matrix starting at time t), the DTW distance elements _{(k, i) f t (} k, i) and. Note that t = (1, 2,..., N), i = (1, 2,..., M), and k = (1, 2,..., N−t + 1). In addition, regarding the items overlapping with the formulas (1) to (4), the description will be omitted as appropriate.

_{D (X [t s: t} e], Y) = f ts (t e -t s + 1, m) = min (f t (k, m)) ··· (6)
f _t (k, i) = || x _{t + k−1} −y _i || + min {f _t (k, i−1), f _t (k−1, i), f _t (k−1, i−) 1)} ... (7)
f _t (0,0) = 0 (8)
f _t (k, 0) = _ft (0, i) = ∞ (9)

  Equation (6) is a definition of the DTW distance. Formula (7) is a specific calculation formula. Equations (8) and (9) are boundary conditions in the time warping matrix.

Next, specific calculations will be described with reference to FIG.
Each element of TWM 1 to 7 stores a value calculated according to the above formulas (6) to (9).
Taking TWM2 as an example, from the above formulas (6) to (9), first, at time t = 2, f ₂ (1,1) = (12-11) ² + min {f ₂ (1,0) = ∞, f ₂ (0, 1) = ∞, f ₂ (0, 0) = 0} = 1 + 0 = 1.
Also, f ₂ (2,1) = (6-11) ² + min {f ₂ (2,0) = ∞, f ₂ (1,1) = 1, f ₂ (1,0) = ∞} = 25 + 1 = 26.

Further, f ₂ (1,2) = (12−6) ² + min {f ₂ (1,1) = 1, f ₂ (0,2) = ∞, f ₂ (0,1) = ∞} = 36 + 1 = 37.
Also, f ₂ (2,2) = (6-6) ² + min {f ₂ (2,1) = 26, f ₂ (1,2) = 37, f ₂ (1,1) = 1} = 0 + 1 = 1.
In the same manner, the value of each element in each TWM 1 to 7 can be calculated in the same manner.

Then, at each time in each TWM1～7, the minimum value is DTW distance among the elements of the value when the previous y _4. Further, the start time of each DTW distance in each TWM 1 to 7 (the time of the first value among the values of the sequence X used for the calculation) is the number of each TWM 1 to 7 (“1” for TWM1 and “3” for TWM3). ).

Under such circumstances, a partial sequence having a high similarity to the sequence Y is detected in the sequence X, that is, a DTW distance of “20” or less is detected.
First, at time t = 1, only TWM1 exists in the time warping matrix and the DTW distance is “54”, which is not applicable.

Next, since the DTW distance of TWM1 is “54” and the DTW distance of TWM2 is “110” at time t = 2, this is not applicable.
At time t = 3, the DTW distance of TWM1 is “50”, the DTW distance of TWM2 is “14”, the DTW distance of TWM3 is “38”, and TWM2 is applicable (“DTW distance is“ 20 ”or less”). Therefore, the partial sequence X [2: 3] can be found.

Thus, according to the comparative example, when the length of the sequence X is n, the target similar partial sequence can be found by using n time warping matrices.
However, as can be seen from the above formulas (6) to (9) and the specific example shown in FIG. 9, each TWM 1 to 7 can store only the information for the two columns of the current time and the time immediately before in the memory. What is necessary is just to erase information before that.

  According to the first method, when the DTW distance is equal to or smaller than the threshold ε, the corresponding partial sequence is detected. However, depending on the convenience of the user, if there is a partial sequence with higher similarity immediately after that, the first partial sequence is not detected and only the subsequent partial sequence with higher similarity is detected. Often you want to detect.

(Second method)
Therefore, in the second method, after detecting a similar partial sequence first, among the partial sequences whose time zone overlaps with the similar partial sequence even a little, the one with higher similarity (small DTW distance) Find out if there is. In other words, after detecting a similar partial sequence, it is only determined that there is no more similar partial sequence that overlaps even a little in the time zone. It is called “a partial sequence”).

  In the example of FIG. 9, at time t = 3, the DTW distance of TWM2 is “14”, and first, the partial sequence X [2: 3] that clears the condition that the threshold value ε or less is found. Since this partial sequence has an end time t = 3, it is not necessary to consider TWMs 4 to 7 whose start time is later than that.

  Therefore, in TWM 1 to 3, it becomes a problem whether there is a possibility that the DTW distance may become smaller than “14” after time t = 3. First, in TWM1, since the value of each element at time t = 3 (from the bottom, “62”, “37”, “46”, “50”) is already larger than “14”, then, There is no possibility that the DTW distance is smaller than “14”. Similarly, for TWM3, there is no possibility that the DTW distance becomes smaller than “14” after time t = 3.

However, in TWM2, when time t = 3, f ₂ (2,2) = 1 and f ₂ (2,3) = 10, and thereafter, the DTW distance may be smaller than “14”. . Actually, at time t = 5, the DTW distance is “6”. At the time t = 5, the values of the elements other than “6” (from the bottom, “52”, “17”, “11”) are all larger than “6”. There is no possibility of being smaller than “6”.

However, since the DTW distance “6” is the partial sequence X [2: 5], TWM4 and TWM5 are also related this time. Therefore, when TWM4 is viewed, at time t = 5 and t = 6, there are elements of “6” or less, that is, values of f ₄ (2,2) = 1 and f ₄ (3,2) = 2, respectively. However, at time t = 7, there is no value less than “6”. Further, in TWM5, when time t = 5, there is no element equal to or less than “6”.

Therefore, it can be seen that the partial sequence X [2: 5] at the time t = 5 of the TWM 2 giving the DTW distance “6” is the optimum sequence at the time t = 7.
Also, in the TWM, the route “f ₂ (1,1) → f ₂ (2,2) → f ₂ (3,3) → f ₂ (4,4) that has been taken to calculate the DTW distance“ 6 ”. 4) "is an example of a warping path.

Thus, according to the first method and the second method of the comparative example, when the length of the sequence X is n, it is necessary to use n time warping matrices, and O (nm) of the calculation is performed. It takes time, and if the value of n increases, the calculation time becomes too long, which is not practical.
Accordingly, in the following, referring to FIGS. 1 to 7, the present embodiment is capable of realizing a high-speed calculation and memory saving with a single time warping matrix and a calculation of O (m). The data stream monitoring apparatus will be described.

  FIG. 1 is a configuration diagram of a data stream monitoring apparatus according to the present embodiment. The data stream monitoring device 1 is a computer device, and includes an input unit 2, a communication unit 3, a storage unit 4, an output unit 5, a memory 6, and a processing unit 7.

The input unit 2 is for inputting data, and is, for example, a keyboard or a mouse. A user of the data stream monitoring apparatus 1 can input an inquiry sequence using the input unit 2.
The communication unit 3 receives data from an external device (not shown) or a sensor (not shown) via the Internet or a LAN, and is a communication interface, for example. The communication unit 3 receives a data stream from an external device (not shown) or the like.

The storage unit 4 stores data and is, for example, a hard disk. The storage unit 4 stores an inquiry sequence 41 and time warping data 42. Although not shown, the storage unit 4 stores a program describing a data stream monitoring method.
The inquiry sequence 41 is an inquiry sequence input from the input unit 2.
The time warping data 42 is data necessary for time warping calculation such as a time warping matrix.

The output unit 5 outputs data, and is, for example, a display or a speaker. The output unit 5 outputs detection data such as a partial sequence that satisfies a predetermined condition.
The memory 6 is a work area of the processing unit 7 and is, for example, a RAM (Random Access Memory).

  The processing unit 7 performs various arithmetic processes, and is, for example, a CPU (Central Processing Unit). The processing unit 7 stores the query sequence received by the input unit 2 in the storage unit 4 as the query sequence 41 or uses the time warping data 42 of the storage unit 4 for the data stream received by the communication unit 3. The partial sequence having high similarity is detected, or the partial sequence is output from the output unit 5.

  Next, an image of inquiry processing by the data stream monitoring apparatus of this embodiment will be described with reference to FIG. FIG. 2 is a diagram showing an image of inquiry processing by the data stream monitoring apparatus of the present embodiment.

As shown in FIG. 2, in the query processing by the data stream monitoring apparatus 1, unlike the comparative example (see FIG. 8), a single TWM is used to change the query sequence Y ′ (query sequence Y ′) from the sequence X. A partial sequence similar to that described later can be detected.
In Figure 2, a corresponding portion _{sequence, X [t s: t e} ] (X 1) is first detected, then, _{X 2} is detected.

In the following description, y ₀ is added to the beginning of a sequence Y = (y ₁ , y ₂ ,..., Y _m ) of length m, and Y ′ = (y ₀ , y ₁ , y ₂ , , Y _m ). At this time, y ₀ can take a value of (−∞: ∞), and x _t is the data received at time t with respect to the sequence X and used for the calculation of the time warping matrix from now on. y ₀ = x _t .
Then, sequence matching is performed between the Y ′ and the sequence X = (x ₁ , x ₂ ,..., X _n ) of length n, and the sequence matching is expressed by the following formula (10) to It can be realized by (14). Note that t = (1, 2,..., N) and i = (1, 2,..., M). In addition, regarding items that overlap with the formulas (6) to (9), description will be omitted as appropriate.

_{D (X [t s: t} e], Y) = d (t e, m) = min (d (t, m)) ··· (10)
d (t, i) = || x _t −y _i || + d _best (11)
d _best = min {d (t, i-1), d (t-1, i), d (t-1, i-1)} (12)
d (t, 0) = d (0,0) = 0 (13)
d (0, i) = ∞ (14)

Of the above formulas (10) to (14), formula (13) is one of the points of this embodiment. This expression (13) provides the same effect as changing the sequence Y to the sequence Y ′.
That is, when the processing unit 7 of the data stream monitoring apparatus 1 receives one data in the data stream at a certain time, the processing unit 7 adds the same data as the one data to the head of the inquiry sequence Y to form the sequence Y ′. , Calculate a dynamic time warping distance using the time warping matrix for the sequence Y ′, and detect a partial sequence corresponding to the dynamic time warping distance as a similar partial sequence when the dynamic time warping distance is equal to or less than a threshold value. However, the same effect as changing the sequence Y to the sequence Y ′ can be obtained by the equation (13).

  In addition, each element of the time warping matrix of the present embodiment is not only the distance d (t, i), but also information s (t I) (“1” if the start time is t = 1, “3” if the start time is t = 3).

<If i ≧ 2>
s (t, i) = s (t, i−1) (when d _best = d (t, i−1))
s (t-1, i) (when d _best = d (t-1, i))
s (t-1, i-1) (when d _best = d (t-1, i-1))
<When i = 1>
s (t, i) = s (t, 1) = t (15)

_{Also, D (X [t s:} t e], Y) starting time _{t s} of is obtained by the following equation (16).
t _s = s (t _e , m) (16)
The optimum warping path can be obtained from the distance calculation by this time warping matrix, and the start time of the detected similar partial sequence can be specified by being taken over on the warping path.

  Next, a specific example of the time warping matrix of the present embodiment will be described with reference to FIG. FIG. 3 is a diagram showing a specific example of the time warping matrix of the present embodiment. A description of the operation of the data stream monitoring apparatus 1 that is the hardware shown in FIG. 1 will be given later with reference to FIGS.

  As shown in FIG. 3, the time warping matrix TWM is a partial sequence having a high similarity with sequence Y = (11, 6, 9, 4) in sequence X that expands with time, as in FIG. The sequence is X = (5, 12, 6, 10, 6, 5, 1). Note that the threshold ε = 20.

A specific calculation will be described. From the above formulas (10) to (14), d (1,1) = (5-11) ² + min {d (1,0) = 0, d (0,1) = ∞, d (0, 0) = 0} = 36 + 0 = 36. At this time, the start time is s (1, 1) = 1 from the equation (15), and “(1)” is described below “36” in the TWM of FIG.

Further, d (2,1) = (12-11) ² + min {d (2,0) = 0, d (1,1) = 36, d (1,0) = 0} = 1 + 0 = 1. . At this time, the start time is s (2,1) = 2.
Unlike the case of the TWM1 of the comparative example (see FIG. 9 (1)), d (2,1) = 1 is one of the points of the present embodiment in this TWM (TWM1 in FIG. 9). The corresponding part in "37"). Although details will be described later, in this way, each element in the bottom row of the TWM in FIG. 3 is independently calculated regardless of the value of the element immediately to the left, that is, the sequence Y = (y ₁ , y ₂ ,..., Y _m ), y ₀ is added to the head, and Y ′ = (y ₀ , y ₁ , y ₂ ,..., Y _m ) is obtained. ) Realizes a single time warping matrix.

Similarly, by using the equations (10) to (15), the value of each element of the TWM and the value of each start time can be calculated.
Hereinafter, similarly to the first method, when a DTW distance equal to or less than the threshold ε is obtained, a method of detecting it as a similar partial sequence is referred to as a third method. Similarly to the second method, after detecting a similar partial sequence first, there is no higher similarity among the subsequent partial sequences that overlap with the similar partial sequence even if the time zone is a little. A method of determining that the partial sequence is the optimum partial sequence after this is determined is referred to as a fourth method.

  In the third method, at time t = 3, X [2: 3] is detected as a similar partial sequence because the DTW distance is “14”. The start time t = 2 is known from “(2)” under d (3,4) = 14, and the end time t = 3 is known from the time at that time.

  This detection result is the same as the detection result of the comparative example (see FIG. 9 (2)). That is, according to the data stream monitoring apparatus 1 of the present embodiment, a detection result equivalent to that of the comparative example can be obtained by a single time warping matrix with a smaller calculation amount and a smaller memory usage than in the comparative example. it can.

Further, in the fourth method, after detecting a similar partial sequence X [2: 3], the similarity is further increased in the subsequent partial sequences whose time zone overlaps with the similar partial sequence as much as possible. It is determined whether there is a high one.
First, if the values of d (3,1), d (3,2), and d (3,3) in the third column from the left are all greater than “14”, the DTW distance is equal to or earlier than t = 3. Therefore, the partial sequence X [2: 3] can be determined as the optimum partial sequence.

However, since d (3,2) = 1 and d (3,3) = 10 here, the partial sequence X [2: 3] cannot be determined to be optimal at the time t = 3.
Next, at time t = 4, it is determined whether or not the condition “value is greater than“ 14 ”” or “start time is after t = 4” is satisfied for all the elements in the column. to decide. If this condition is satisfied, it can be determined that the partial sequence X [2: 3] is optimal.
Here, since d (4,3) = 2 (start time t = 2) does not satisfy this condition, it cannot be determined that the partial sequence X [2: 3] is optimal.

Subsequently, at time t = 5, since the DTW distance is “6”, the optimum partial sequence candidate is changed to X [2: 5].
Next, at time t = 6, it is determined whether or not the condition “value is greater than“ 6 ”” or “start time is after t = 6” is satisfied for all the elements in the column. to decide.
Here, since d (6,2) = 2 (start time t = 4) does not satisfy this condition, it cannot be determined that the partial sequence X [2: 5] is optimal.

Next, at time t = 7, it is determined whether or not the condition “value is greater than“ 6 ”” or “start time is after t = 6” is satisfied for all the elements in the column. to decide.
Here, since the condition is satisfied, it can be determined that the partial sequence X [2: 5] is optimal.

  This detection result is the same as the detection result of the comparative example (see FIG. 9 (2)). That is, according to the data stream monitoring apparatus 1 of the present embodiment, a detection result equivalent to that of the comparative example can be obtained by a single time warping matrix with a smaller calculation amount and a smaller memory usage than in the comparative example. it can.

(Third method)
Next, the operation of the data stream monitoring apparatus when the third method is executed will be described with reference to FIG. FIG. 4 is a flowchart showing the operation of the data stream monitoring apparatus when the third method is executed. In addition, d (t, i) (refer to Formula (11)) is expressed as d _i , and d (t−1, i) is expressed as d _i ′. In addition, s (t, i) (see Expression (15)) is expressed as s _i , and s (t−1, i) is expressed as s _i ′.

First, the user inputs an inquiry sequence from the input unit 2 in the data stream monitoring apparatus 1, and the input inquiry sequence is stored in the storage unit 4 as an inquiry sequence 41. The storage unit 4 stores a single time warping matrix (see FIG. 3) and time warping data 42 such as a predetermined threshold ε.
And the communication part 3 starts reception of a data stream from an external device (not shown). Hereinafter, the processing unit 7 develops and processes the data received from the communication unit 3 and the storage unit 4 in the memory 6, but the description of developing in the memory 6 is omitted.

Processing unit 7 first starts processing as a time t = 1 (step S401), then, at the time at that time t ( "1" in this case), receives the _{x t} (step S402).
Subsequently, the processing unit 7 calculates all d _i and s _i based on the equations (10) to (15) (step S403), and updates the time warping matrix. At this time, data after time “t−2” in the time warping matrix may be deleted (when t ≧ 3).

Then, the processing unit 7, _{d m,} i.e., DTW distance at time t is equal to or less than the threshold value epsilon (step S404).
If the DTW distance at time t is not less than or equal to the threshold ε (No in step S404), the processing unit 7 proceeds to step S408.

If DTW distance at time t is equal to or less than the threshold value epsilon (Yes in step S404), the processing unit 7 substitutes starting time _{s m} of the DTW distance variable _{t s} start time, the time t at that time It is assigned to the variable _{t e} of the end time (step S405).
Subsequently, the processing unit 7, DTW distance _{d m,} the starting time _{t s,} and outputs the end time _{t e} to the output unit 5 (step S406).

The user can know that there is a partial sequence similar to the query sequence in the data stream by looking at the data output from the output unit 5.
Next, processing unit 7 for further processing, initializing the values of all d _i, i.e., use a sufficiently large value as compared to like ε is when creating ∞ (program d _i The same applies to the following (step S407).
Subsequently, the processing unit 7, in order to move to the processing of the next time, the value of all the d _i and s _i, respectively, to d _{i 'and} s _i' is a variable of one previous value of the time Substitution is performed (step S408), "t = t + 1" is set (step S409), the process returns to step S402, and the process is repeated.

Thus, according to the data stream monitoring apparatus 1 of the present embodiment, by using a single time warping matrix, the calculation amount and the memory usage amount can be kept constant even when the data stream becomes long. However, the same result as the comparative example can be obtained.
That is, according to the data stream monitoring apparatus 1 of the present embodiment, for example, in the example of FIG. 3, at the time t = 3, the DTW distance “14”, its start time “2”, and its end time “ 3 ”can be output from the output unit 5.

(Fourth method)
Next, the operation of the data stream monitoring apparatus when the fourth method is executed will be described with reference to FIG. FIG. 5 is a flowchart showing the operation of the data stream monitoring apparatus when the fourth method is executed. In addition, the description which overlaps with the case of FIG. 4 is abbreviate | omitted suitably. As in the case of FIG. 4, in the data stream monitoring apparatus 1, the storage unit 4 stores an inquiry sequence 41, a single time warping matrix (see FIG. 3), and time warping data 42 such as a predetermined threshold ε. Is remembered. Further, d _min is used as a variable for inputting an optimum partial sequence candidate.
And the communication part 3 starts reception of a data stream from an external device (not shown).

Processing unit 7 first starts processing as a time t = 1 (step S501), then, at the time at that time t ( "1" in this case), receives the _{x t} (step S502).
Subsequently, the processing unit 7 calculates all d _i and s _i based on the equations (10) to (15) (step S503), and updates the time warping matrix. At this time, data after time “t−2” in the time warping matrix may be deleted (when t ≧ 3).

Next, the processing unit 7 determines whether or not d _min , that is, whether the DTW distance at time t is equal to or less than the threshold ε (step S504).
If DTW distance at time t is not less than the threshold value epsilon (No at step S504), the processing unit 7, in step S514, whether or not meet, _{"d min} ≦ epsilon" and _"d m _{<d min"} Although it is determined, since “d _min ≦ ε” is not satisfied in step S514 (No), the process proceeds to step S516.

When the DTW distance d _{min at} time t is equal to or smaller than the threshold ε (Yes in step S504), the processing unit 7 determines whether or not the partial sequence that provides the DTW distance d _min is optimal, that is, after the partial sequence. of all the elements at a certain time (the _{"d i", "s i"),} "the value _{d i} is greater than _{d min"} or "start time _{s i} is later than _{t e",} a condition that It is determined whether or not it is satisfied (step S505). This is because, if this condition is satisfied, it can be determined that the partial sequence that provides the DTW distance d _min is optimal.

That is, when the processing unit 7 detects a partial sequence whose DTW distance d _min is equal to or less than the threshold value ε, the processing unit 7 further calculates the value of each matrix element of the time warping matrix, and the partial sequence and the time zone are partially However, when it is determined that there is no overlapping partial sequence having a smaller DTW distance, the partial sequence can be detected as an optimal (more appropriate) partial sequence.

If the condition is not satisfied in step S505 (No), the processing unit 7, in step S514, it is determined whether meet, _{"d min} ≦ epsilon" and _"d m _{<d min".}
Here, satisfies the condition of step S514 if there is a candidate for further optimal subsequence satisfied (Yes), then the process unit 7, the DTW distance _{d m} in the variable _{d min,} the starting time _{s m} in the variable _{t s,} the end time t to the variable _{t e,} the values are (step S515).

Satisfies case of step S505 (Yes), since the subsequence gives the DTW distance _{d min} can be determined to be optimal, the processing unit 7, the variable _{t s} start time start time _{s m} of the DTW distance assignment and, after substituting the time t at that time in the variable _{t e} end time, the DTW distance _{d min,} the starting time _{t s,} and outputs the end time _{t e} to the output unit 5 (step S506 ).
The user can know the optimum partial sequence among the partial sequences similar to the query sequence existing in the data stream by looking at the data output from the output unit 5.

Next, for the subsequent processing, the processing unit 7 substitutes ∞ for d _min (step S507), substitutes “0” for i (step S508), and further performs the processing of steps S509 to S513. .
In the process of step S509～S513, processing unit 7 with respect the value of "i" from "1" to "m" (see step S510 and step S513), satisfies the _{"s i} ≦ _{t e"} (Step S511 Yes), “∞” is substituted into “d _i ” (step S512). Here, in order to eliminate the possibility that a partial sequence whose time zone overlaps with the optimum partial sequence output in step S506 even if it is a little later will be output in step S506, in step S506 among the elements at that time. is the value of the end time t _e previous ones of output the best part sequence to infinity.

Subsequently, the processing unit 7, in order to move to the processing of the next time, the value of all the d _i and s _i, respectively, to d _{i 'and} s _i' is a variable of one previous value of the time Substitution is performed (step S516), "t = t + 1" is set (step S517), the process returns to step S502, and the process is repeated.

Thus, according to the data stream monitoring apparatus 1 of the present embodiment, an optimal partial sequence can be output even if a single time warping matrix is used.
That is, according to the data stream monitoring apparatus 1 of the present embodiment, for example, in the example of FIG. 3, the DTW distance “6” at the time t = 5, not the DTW distance “14” at the time t = 3. (And its start time and end time) can be outputted from the output unit 5 as the DTW distance of the optimum partial sequence.
According to the fourth method, the optimum partial sequence can be detected without fail due to the characteristics of the algorithm, as in the method according to the comparative example.

  Further, the third method and the fourth method can be realized in a computer apparatus such as the data stream monitoring apparatus 1 by creating a program for executing the flowcharts of FIGS. 4 and 5. Furthermore, these programs can be stored in a recording medium such as a hard disk, a flash memory, a CD-ROM (Compact Disk Read Only Memory), or a DVD (Digital Versatile Disk).

(First embodiment)
Next, a first example relating to the data stream monitoring apparatus of this embodiment will be described with reference to FIG. 6A and 6B are diagrams for explaining a first example relating to the data stream monitoring apparatus of the present embodiment, where FIG. 6A shows an inquiry sequence, and FIG. 6B shows a partial sequence detection state.

In the first example, the data stream monitoring apparatus 1 according to the present embodiment is implemented on a Linux (registered trademark) machine equipped with a 1 GB memory and an Intel (registered trademark) Xeon 2.8 GHz CPU. Here, an experiment is performed in the case where the temperature is measured by the temperature sensor.
FIG. 6A shows the waveform of the inquiry sequence, where the vertical axis represents temperature (Celsius) and the horizontal axis represents the passage of time. This inquiry sequence is characterized in that there are two patterns in which the temperature varies greatly from about 20 degrees to about 31 degrees due to changes in the weather.

FIG. 6B shows a waveform of the data stream. As in FIG. 6A, the vertical axis represents temperature (Celsius) and the horizontal axis represents the passage of time. This data stream is obtained by receiving data measured by the temperature sensor about every minute by the machine.
By executing the fourth method of the present embodiment on the above machine, as shown in FIG. 6B, in the data stream, Sub1 and Sub1 as a partial sequence (Subseq) similar to the query sequence of (a) Two of Sub2 are detected without missing.

(Second embodiment)
Next, a second example relating to the data stream monitoring apparatus of this embodiment will be described with reference to FIG. FIG. 7 is a diagram illustrating the relationship between the sequence length of the data stream and the calculation time in the second example relating to the data stream monitoring apparatus of the present embodiment.

In the second embodiment, the inquiry sequence is artificial data and the length is “256”. The data stream is artificial data, and the length (sequence length) is variable from “10 to the third power” to “10 to the sixth power”.
In FIG. 7, the vertical axis represents the calculation time (ms), and the horizontal axis represents the length of the data stream (sequence length). In both the method according to the comparative example and the method according to the present embodiment, the total time of detection of the partial sequences similar to the update of the time warping matrix is averaged to obtain the calculation time.

  As shown in FIG. 7, in the method (L2) according to the comparative example, the calculation time increases as the data stream becomes longer. On the other hand, in the method (L1) according to the present embodiment, the calculation time is kept constant even if the data stream becomes long. This is because as the data stream becomes longer, the number of time warping matrices increases in the method according to the comparative example, whereas the time warping matrix is always single in the method according to the present embodiment.

  Thus, as can be seen from the first example and the second example, according to the data stream monitoring apparatus 1 of the present embodiment, the calculation cost is reduced, that is, the calculation speed is higher than in the comparative example. And memory saving.

This is the end of the description of the embodiments, but the aspects of the present invention are not limited to these.
For example, in the present embodiment, the case of one-dimensional data has been described with respect to the query sequence and the data stream.
In addition to the temperature sensor, the present invention is applicable to many fields such as video and music distribution, bioinformatics, and various robots.
In addition, the specific configuration can be changed as appropriate without departing from the spirit of the present invention.

It is a block diagram of the data stream monitoring apparatus of this embodiment. It is the figure which showed the image of the inquiry process by the data stream monitoring apparatus of this embodiment. It is the figure which showed the specific example of the time warping matrix of this embodiment. It is the flowchart which showed operation | movement of the data stream monitoring apparatus in the case of performing a 3rd method. It is the flowchart which showed operation | movement of the data stream monitoring apparatus in the case of performing a 4th method. It is a figure for demonstrating the 1st Example regarding the data stream monitoring apparatus of this embodiment, (a) represents the inquiry sequence, (b) represents the mode of the partial sequence detection, respectively. It is a relationship figure of the length of a sequence, and calculation time about the 2nd Example regarding the data stream monitoring apparatus of this embodiment. It is the figure which showed the image of the inquiry process in the comparative example of the data stream monitoring which uses DTW. It is the figure which showed the example of the time warping matrix in a comparative example.

Explanation of symbols

1 Data Stream Monitoring Device 4 Storage Unit 5 Output Unit 7 Processing Unit TWM Time Warping Matrix

Claims (8)

A data stream monitoring device that detects a partial sequence similar to an inquiry sequence that is a predetermined data string from a data stream that is continuously received data based on a dynamic time warping distance,
When data x _t at time t in the data stream is received , the data y _i in the query sequence where the sequence of length m = (y ₁ , y ₂ ,..., Y _m ) (however, , I = 1, 2,..., M) and the information d (t−1, i) regarding the distance between the data x _t−1 at the time t−1 in the data stream and the information d ( a single time warping matrix having information s (t-1, i) relating to the start time corresponding to t-1, i) and (t-1, i) elements , and a predetermined relating to the dynamic time warping distance. a storage section for storing the threshold epsilon,
Upon receiving the pre-Symbol data x _t, using said storage unit to the stored time warping matrix with the following equation (11) to (14), the data y _{i (except} in the query sequence, i = 1 , 2, ..., m) and information d (t, i) regarding the distance between the data x _t and the start corresponding to the d (t, i) calculated by the following equation (15) Information s (t, i) relating to time is calculated, the time warping matrix stored in the storage unit is updated, and information d relating to the distance between the data y _m and the data x _t in the inquiry sequence (t, m) when there is more than a predetermined threshold epsilon, the starting time s (t, m) a starting time _{t s} corresponding to the information d (t, m) regarding the distance, terminating the time t time t the partial sequence X [t in the data stream as _{e s:} a processing unit for detecting a t _e] as subsequence said similar,
A data stream monitoring apparatus comprising:
d (t, i) = || x _t −y _i || + d _best (11)
d _best = min {d (t, i-1), d (t-1, i), d (t-1, i-1)}
(12)
d (t, 0) = d (0,0) = 0 (13)
d (0, i) = ∞ (14)
(Where || x _t −y _i || represents a distance between two numerical values (x _t and y _i ), and min {d (t, i−1), d (t−1, i), d (t−1, i−1)} represents that the smallest one of the three values in {} is adopted.)
<If i ≧ 2>
s (t, i) = s (t, i−1) (when d _best = d (t, i−1))
s (t-1, i) (when d _best = d (t-1, i))
s (t-1, i-1) (when d _best = d (t-1, i-1))
<When i = 1>
s (t, i) = s (t, 1) = t (15) Wherein the processing unit, the similar parts sequence _X: when detecting [t s t _e], then further subjected to calculation of the value of each matrix element of the time warping matrix, partial sequences X [t that similar When it is determined that there is no partial sequence having a smaller dynamic time warping distance among partial sequences that overlap with _s : t _e ] even in part, the similar partial sequence X [t _s : t _e ] The data stream monitoring apparatus according to claim 1 , wherein: is detected as a more appropriate partial sequence. Wherein the processing unit, the similar parts sequence _X: when detecting [t s t _e], partial sequence X that _similar: When the end time of the reception time of the last data [t s t _e], the in the column with the time warping matrix after the end time, of each matrix element, a the start time is the end time earlier, and partial sequence X whose value the similarity [t _s: t _e] of The data stream monitoring apparatus according to claim 2 , wherein when there is nothing smaller than the dynamic time warping distance, the similar partial sequence is detected as more appropriate. A data stream monitoring method by a data stream monitoring device that detects a partial sequence similar to an inquiry sequence that is a predetermined data string from a data stream that is continuously received data based on a dynamic time warping distance,
The data stream monitoring device receives the data x _t at time t in the data stream, and the inquiry sequence is a sequence m of length m = (y ₁ , y ₂ ,..., Y _m ). Data d _i (where i = 1, 2,..., M) and information d (t−1, i) relating to the distance between the data x _t−1 at time t−1 in the data stream and A single time warping matrix having (t-1, i) elements as information s (t-1, i) related to the start time corresponding to the information d (t-1, i) related to the distance , and a storage unit for storing a predetermined threshold value ε about the dynamic time warping distance, and a processing unit,
Wherein the processing unit, when receiving the previous SL data x _t, using said storage unit to the stored time warping matrix with the following equation (11) to (14), the data y _i in the query sequence ( However, the information d (t, i) regarding the distance between i = 1, 2,..., M) and the data x _t and the d (t, i) calculated by the following equation (15). ) To calculate the information s (t, i) related to the start time corresponding to), update the time warping matrix stored in the storage unit, and between the data y _m and the data x _t in the query sequence distance information d (t, m) in the case is less than a predetermined threshold epsilon, the starting time s (t, m) a starting time _{t s} corresponding to the information d (t, m) regarding the distance, the time portion in the data stream to end time t _e the t Shi Sequence X [t _s: t _e] the data stream monitoring method characterized by detecting a subsequence the similar.
d (t, i) = || x _t −y _i || + d _best (11)
d _best = min {d (t, i-1), d (t-1, i), d (t-1, i-1)}
(12)
d (t, 0) = d (0,0) = 0 (13)
d (0, i) = ∞ (14)
(Where || x _t −y _i || represents a distance between two numerical values (x _t and y _i ), and min {d (t, i−1), d (t−1, i), d (t−1, i−1)} represents that the smallest one of the three values in {} is adopted.)
<If i ≧ 2>
s (t, i) = s (t, i−1) (when d _best = d (t, i−1))
s (t-1, i) (when d _best = d (t-1, i))
s (t-1, i-1) (when d _best = d (t-1, i-1))
<When i = 1>
s (t, i) = s (t, 1) = t (15) Wherein the processing unit, the similar parts sequence _X: when detecting [t s t _e], then further subjected to calculation of the value of each matrix element of the time warping matrix, partial sequences X [t that similar When it is determined that there is no partial sequence having a smaller dynamic time warping distance among partial sequences that overlap with _s : t _e ] even in part, the similar partial sequence X [t _s : t _e ] The data stream monitoring method according to claim 4 , wherein the data stream is detected as a more appropriate partial sequence. Wherein the processing unit, the similar parts sequence _X: when detecting [t s t _e], partial sequence X that _similar: When the end time of the reception time of the last data [t s t _e], the in the column with the time warping matrix after the end time, of each matrix element, a the start time is the end time earlier, and partial sequence X whose value the similarity [t _s: t _e] of 6. The data stream monitoring method according to claim 5 , wherein when there is nothing smaller than the dynamic time warping distance, the similar partial sequence is detected as more appropriate. Program for executing a data stream monitoring method according to the computer of claims 4 to any one of claims 6. A recording medium for recording the program according to claim 7 .

Images