JP5162512B2 - Multidimensional time series data analysis apparatus and multidimensional time series data analysis program - Google Patents

Description

  The present invention relates to a multidimensional time series data analysis apparatus and a multidimensional time series data analysis program.

Multidimensional time-series data is multidimensional time-series data obtained by observing and recording phenomena that occur in multiple dimensions at regular time intervals. Multi-dimensional time-series data includes motion data representing movements of people and objects, video data, audio data collected by multiple microphones, data representing changes in multiple stock prices, and temperature data observed in multiple regions. and so on. Multidimensional data at a certain time is called a frame.
Determining the boundary between patterns in order to extract a pattern from multidimensional time-series data is called pattern boundary determination. For example, in the pattern boundary determination of human skeleton type motion data, which is multidimensional time series data indicating human motion, each motion pattern is extracted from motion data including multiple motion patterns such as walking and running. A start time and an end time are determined.

As a method for determining a pattern boundary from human skeleton-type motion data, for example, methods disclosed in Non-Patent Documents 1 and 2 are known. Non-Patent Document 1 shows three methods for determining a pattern boundary. In Method 1, a pattern boundary is determined based on a change amount of an error that occurs when principal component analysis is performed on human body skeleton type motion data. In Method 2, principal component analysis is performed on the human skeleton type motion data to evaluate the similarity between the frames of the motion data, and determine the boundary of the motion pattern. In Method 3, when consecutive frames are not included in the same Gaussian Mixture Model, it is determined that the position is the boundary of the motion pattern. Non-Patent Document 2 discloses a method of determining a motion pattern boundary using a support vector machine (SVM).
Human body skeleton type motion data representing human motion usually has many degrees of freedom, but there is a strong relationship between each joint. Therefore, it is effective to perform principal component analysis to reduce the dimension of human skeleton type motion data. As a method of principal component analysis, for example, there are methods shown in Non-Patent Document 3 and Non-Patent Document 4. In these methods, the entire human body skeleton type motion data is subjected to principal component analysis, and some principal components having a high contribution rate to motion are extracted.

J. Barbic and 5 others, "Segmenting Motion Capture Data into Distinct Behaviors", Proceedings of Graphics Interface 2004, 2004, Vol. GI04, p.185-194 O. Arikan and two others, "Motion synthesis from annotations", ACM Transactions on Graphics, July 2003, vol.22 (3), p.402-408 L. M. Tanco and 1 other, "Realistic synthesis of novel human movements from a database of motion capture examples", IEEE Workshop on Human Motion, 200, p.137-142 P. Glardon and two others, "PCA-based Walking Engine Using Motion Capture Data", Computer Graphics International, 2004, p.292-298

However, in the method 1 and method 2 of Non-Patent Document 1 described above, the principal component analysis is performed every time the number of frames is increased by a certain number, and the repeated principal component is used to determine the pattern boundary based on the difference in the principal component analysis results of each time. It is necessary to perform analysis, and the calculation amount becomes enormous.
In Method 3, it is difficult to determine the parameter value of the Gaussian mixture model in advance. Furthermore, since prior training is required, there is a problem that it cannot be applied to online processing. In the method of Non-Patent Document 2, when the user creates training data and the support vector machine makes a mistake, it is necessary for the user to indicate the correct answer.

  The present invention has been made in view of such circumstances, and its purpose is not to determine parameter values of a Gaussian mixture model, pre-training, creation of training data by the user, or presentation of correct answers by the user. Another object of the present invention is to provide a multidimensional time series data analysis apparatus capable of determining a pattern boundary with a smaller calculation amount.

[1] The present invention has been made to solve the above-described problems, and a multidimensional time-series data analysis device according to an aspect of the present invention arranges multidimensional data at each of a plurality of times according to a time series. A multidimensional time series data storage unit that stores multidimensional time series data, a principal component matrix storage unit that stores a principal component matrix, and the multidimensional time series data read from the multidimensional time series data storage unit A principal component analysis unit that generates a principal component matrix by performing principal component analysis and writes the principal component matrix to the principal component matrix storage unit , and a principal component that selects a plurality of elements included in one row in the principal component matrix as principal components Read out the principal component selected by the selection unit and the principal component selection unit from the principal component matrix storage unit, determine the distribution center and distribution width of the principal component value, and determine the principal component value and the distribution center Shift is characterized by comprising a boundary determining unit that determines and outputs the pattern boundary of the multi-dimensional time-series data on the basis of the point where greater than the distribution width of.

  [2] A multidimensional time-series data analysis apparatus according to an aspect of the present invention is the multidimensional time-series data analysis apparatus described above, wherein the boundary determination unit is configured to determine the extreme points for each of the extreme points of the principal component. A forward average value that is an average value of the principal component values in a previous predetermined period and a backward average value that is an average value of the principal component values in the predetermined period after the extreme point are calculated, and the forward average value On the basis of this, the center of the distribution and the distribution width are determined for each of the extreme points, and the difference between the backward average value and the center of the distribution is defined as the deviation.

  [3] A multidimensional time series data analysis apparatus according to an aspect of the present invention is the multidimensional time series data analysis apparatus described above, wherein the principal component analysis unit is a processing target based on the multidimensional time series data. A variation data matrix is generated by subtracting a matrix of average values in the time direction of the processing target matrix from a matrix, and a principal component matrix is generated by performing singular value decomposition on the variation data matrix .

  [4] A multidimensional time-series data analysis apparatus according to an aspect of the present invention is the multidimensional time-series data analysis apparatus described above, wherein the principal component selection unit includes each of the first row of the principal component matrix. When an element is selected as a principal component and the boundary determination unit cannot determine the position of the pattern boundary using the principal component, each element in the second row of the principal component matrix is selected as a principal component, and Each time the boundary determination unit cannot determine the position of the pattern boundary, it is a submatrix from the top to the Kth row (K is an arbitrary integer not less than 1 and not more than the number of rows of the principal component matrix) in the principal component matrix. Each element in one row is selected as a main component in order from the top row in a certain determination target main component.

  [5] A multidimensional time series data analysis apparatus according to an aspect of the present invention is the above-described multidimensional time series data analysis apparatus, which stores multidimensional time series data in a multidimensional time series data storage unit, The principal component analysis unit is instructed to read out data for N times (N is an arbitrary positive integer) from the multi-dimensional time-series data storage unit in chronological order, and the boundary determination unit determines the position of the pattern boundary. Data acquisition instructing the principal component analysis unit to read from the multi-dimensional time-series data storage unit the data for N times in chronological order from the multi-dimensional time-series data after the position of the pattern boundary. And the frame in which the principal component analysis unit has performed principal component analysis when the boundary determination unit cannot determine the position of the pattern boundary for all the principal components included in the determination target principal component and the frame A data addition unit for instructing the principal component analysis unit to read out P frames (P is an arbitrary positive integer) from the new frame in the oldest order from the multidimensional time-series data storage unit; Features.

[6] In addition, the multidimensional time-series data analysis apparatus according to one aspect of the present invention stores multidimensional time-series data stored by storing multidimensional time-series data in which multidimensional data at each of a plurality of times is arranged according to the time series. parts and the main component matrix storage unit for storing the main component matrix, produced the principal component matrix by performing principal component analysis on the multi-dimensional time-series data read out from the multi-dimensional time series data storage unit A principal component analysis unit that writes to the principal component matrix storage unit , a principal component selection unit that selects a plurality of elements included in one row in the principal component matrix as principal components, and the principal component selected by the principal component selection unit The component is read from the principal component matrix storage unit, the distribution center and distribution width of the principal component value are determined, and the difference between the principal component value and the distribution center is larger than the distribution width. Exist Calculating an autocorrelation value of the main component in engagement, characterized by comprising a boundary determination unit for determining and outputting the position of the pattern boundary based on the time value of the autocorrelation is maximum.

[7] A multidimensional time series data analysis program according to an aspect of the present invention is a multidimensional time series data storage that stores multidimensional time series data in which multidimensional data at a plurality of times is arranged according to a time series. A principal component analysis step of generating a principal component matrix by performing principal component analysis on the multi-dimensional time series data read out from the component and writing the principal component matrix into the principal component matrix storage unit , and included in one row of the principal component matrix A principal component selection step that selects a plurality of elements as principal components, and the principal component selected by the principal component selection step is read from the principal component matrix storage unit, and the distribution center and distribution width of the principal component values are read out. And determining and outputting a pattern boundary of the multi-dimensional time series data based on a point where a deviation between the value of the principal component and the center of the distribution is larger than the distribution width. Characterized in that it is a multi-dimensional time-series data analysis program for executing the boundary determining step in the computer.

  According to the present invention, in the multidimensional time series data analysis apparatus, it is not necessary to determine the parameter value of the Gaussian mixture model, to prepare the training data by the user in advance or to provide the correct answer by the user, and more The pattern boundary can be determined with a small amount of calculation.

It is a schematic block diagram which shows the structure of the multidimensional time series data analysis apparatus 1 by one Embodiment of this invention. It is the schematic which shows the example of the skeleton model of the human body used for the definition of a human body skeleton type | mold motion data. 5 is a graph showing examples of values calculated from principal components when the multidimensional time-series data analysis apparatus 1 performs pattern boundary determination in the embodiment. It is a flowchart which shows the process sequence of the multidimensional time series data analysis apparatus 1 in the embodiment. FIG. 5 is a flowchart showing a processing procedure in step Sa4 of FIG. 4 performed by a boundary determination unit 41 in the same embodiment.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic block diagram showing a configuration of a multidimensional time series data analysis apparatus 1 according to an embodiment of the present invention. In the figure, a multidimensional time series data analysis apparatus 1 includes a data acquisition unit 11, a data memory (multidimensional time series data storage unit) 12, a principal component analysis unit 21, and a principal component matrix storage unit (not shown). ), A principal component selection unit 31, a boundary determination unit 41, and a data addition unit 51.

  In the present embodiment, human skeleton type motion data defined by a human skeleton model (skeleton type) is used as an example of multidimensional time-series data. The skeleton type motion data is data in which the motion of each joint of the skeleton type object, which is an object for which the skeleton type is defined, is recorded. Examples of the skeleton type object include a human body, an animal, and a robot.

  FIG. 2 is a schematic diagram showing an example of a human skeleton model used for defining human skeleton type motion data (hereinafter simply referred to as “motion data”). The human skeleton model uses bone and bone connection points (joints) based on the human skeleton, with one joint as the root, and bones that are sequentially connected from the root via the joint. Is configured as a tree structure. In FIG. 5, the joint 100 is a waist portion and a root. Joint 101 is the elbow portion of the left arm, Joint 102 is the wrist portion of the left arm, Joint 103 is the elbow portion of the right arm, Joint 104 is the wrist portion of the right arm, Joint 105 is the knee portion of the left foot, and Joint 106 is the left foot portion. The ankle portion, the joint 107 is the knee portion of the right foot, and the joint 108 is the ankle portion of the right foot. A more complicated skeleton model including a joint at the shoulder may be used.

The motion data can be expressed by position information, angle information, velocity information, acceleration information, or a combination of each joint. The multidimensional time-series data analysis apparatus according to the present embodiment can perform pattern boundary determination on motion data in various representation formats such as human skeleton-type angle information data and human skeleton-type acceleration information data.
Human body skeleton-type angle information data indicates a series of movements of a person by a series of multiple poses (poses, poses), and includes basic pose data indicating a person's basic poses (Neutral Pose) and actual human movements. Frame data for each pose indicating each pose. The basic pose data includes information such as the position of the root and the position of each joint in the basic pose, and the length of each bone. The basic pose is specified by the basic pose data. The frame data represents the amount of movement from the basic pose for each joint. Here, angle information is used as the movement amount. Each frame data identifies each pose in which each movement amount is added to the basic pose. Thereby, a series of movements of a person is specified as a continuation of each pose specified by each frame data. The human skeleton-type motion data can be generated by a motion capture process from video captured by a camera, or can be generated using a key frame animation technique.
The human body skeleton type acceleration information data represents the acceleration of each joint of a person by continuous frame data for each pose and a plurality of poses. The human skeleton-type acceleration information data can be recorded by an accelerometer, or calculated from video and motion data.

Returning to FIG. 1, the function of each part will be described.
The data memory 12 stores multidimensional time series data in which multidimensional data is arranged in time series at each of a plurality of times. In the following, a matrix of multidimensional time series data stored in the multidimensional data memory 12 is indicated by X.
The principal component matrix storage unit (not shown) stores the principal component matrix Y generated by the principal component analysis unit 21.
The data acquisition unit 11 acquires motion data at a plurality of times at regular intervals. In the following, the time when the data acquisition unit 11 acquires motion data is indicated by time 1, time 2, time 3,. The data acquisition unit 11 writes data corresponding to other degrees of freedom excluding the degree of freedom of the route in the acquired motion data in the data memory 12 in chronological order. That is, the data acquisition unit 11 removes the degree of freedom of the route from the input multidimensional time series data (hereinafter, the matrix of the multidimensional time series data input to the data acquisition unit 11 is indicated by X ′). The matrix X of the dimension time series data is written into the data memory 12. Hereinafter, multidimensional data for each time written in the data memory 12 is referred to as a frame. Each frame is distinguished by assigning a number i to the frame at the i-th oldest time. The data acquisition unit 11 may store the data as it is in the data memory 12 without removing the degree of freedom of the route from the acquired motion data.
In addition, the data acquisition unit 11 instructs the principal component analysis unit 21 to read out N frames of N frames. Here, N is a predetermined constant. When the multidimensional time-series data analysis device 1 starts determining the pattern boundary, the data acquisition unit 11 outputs the number “1” of the first frame and the number N of the Nth frame to the principal component analysis unit 21. Here, N is an arbitrary positive integer. On the other hand, when the boundary determination unit 41 determines the position of the pattern boundary, the data acquisition unit 11 indicates N frames in chronological order from the frames acquired after the Qth frame determined as the pattern boundary position. The first frame number Q and the last frame number Q + N−1 are output to the principal component analysis unit 21. That is, the data acquisition unit 11 first instructs the principal component analysis unit 21 to read data for N times from the multi-dimensional time-series data storage unit (data memory) 12 in order from the oldest, and the boundary determination unit 41 sets the pattern boundary. Are read out from the multi-dimensional time-series data storage unit (data memory) 12 in the chronological order from the matrix X of the multi-dimensional time-series data after the position of the pattern boundary. The principal component analysis unit 21 is instructed. Note that the value of N indicating the number of frames may be changeable. For example, it can be changed when the boundary determination unit 41 determines the position of the pattern boundary and ends the processing.

  The principal component analysis unit 21 reads out N frames from the data memory 12 according to the frame number input from the data acquisition unit 11, and a matrix X = (x (1), x (2),... x (N)) are generated. Here, M represents the number of data included in each frame, and x (i) represents the data vertical vector of the i-th frame. The principal component analysis unit 21 generates a principal component matrix Y from the matrix X of multidimensional time series data according to the following procedure. That is, the principal component analysis unit 21 performs principal component analysis on N frames, which are multidimensional time series data read from the data memory (multidimensional time series data storage unit) 12, by the following procedure. A component matrix Y is generated.

  Procedure 1: Using equation (1), an N-row and M-column variation data matrix D is generated by subtracting a matrix of average values of multidimensional time series data from a matrix X of multidimensional time series data.

  Procedure 2: A singular value decomposition (Singular Value Decomposition) is performed with respect to the fluctuation | variation data matrix D using Formula (2).

  However, U is a unitary matrix of N rows and N columns. Σ is a diagonal matrix having N rows and M columns of non-negative diagonal elements in descending order, and indicates the distribution of coordinates in the principal component space. V is a unitary matrix of M rows and M columns, and is a coefficient (Principal Component) for the principal component.

Procedure 3: Generate a principal component matrix Y of M rows and N columns in the principal component space using Equation (3). That is, the principal component analysis unit 21 generates the principal component matrix Y by performing singular value decomposition on the variation data matrix D.
The principal component analysis unit 21 writes the principal component matrix Y in a principal component matrix storage unit (not shown). The principal component analysis unit 21 outputs the first frame number and the last frame number input from the data acquisition unit 11 to the principal component selection unit 31.

The principal component selection unit 31 selects an element for one row from the principal component matrix Y as the principal component, and outputs the selected principal component number to the boundary determination unit 41. That is, the principal component selection unit 31 selects a plurality of elements included in one row in the principal component matrix Y as principal components. In addition, the principal component selection unit 31 outputs the first frame number and the last frame number input from the principal component analysis unit 21 to the boundary determination unit 41.
The principal component selection unit 31 selects a principal component according to the following three cases.
Case 1: When the principal component is first output to the boundary determination unit 41 after the principal component matrix Y is input from the principal component analysis unit 21, the principal component selection unit 31 selects each element in the first row of the principal component matrix Y. As the main component. Hereinafter, each element in the first row of the matrix Y is referred to as a first principal component. The first principal component has a large change in the data value for each frame, and it is expected that the boundary determination is appropriately performed. However, depending on the frame, the change in the data value may be larger in other rows.
Case 2: When the boundary determination unit 41 determines that there is no pattern boundary using the first principal component, that is, when the position of the pattern boundary cannot be determined, each element in the second row of the principal component matrix Y is mainly used. Select as an ingredient. Hereinafter, each element of the second row of the matrix Y is referred to as a second principal component.
Case 3: When the boundary determination unit 41 determines that there is no pattern boundary in the determination using the second principal component, each element in the third row of the principal component matrix Y is selected as the principal component. Hereinafter, each element in the third row of the matrix Y is referred to as a third principal component. The principal components selected by the principal component selection unit are not limited to the first principal component to the third principal component, but the K principal component (K is an arbitrary integer not less than 1 and not more than the number of rows of the principal component matrix). Sequential selection may be performed as described above. That is, every time the boundary determination unit 41 cannot determine the position information of the pattern boundary, the principal component selection unit 21 moves over the determination target principal component that is a partial matrix from the top of the principal component matrix Y to the Kth matrix. Each element in one row is selected as a principal component in order from the row.
In the following, the principal components selected by the principal component selection unit 31 are denoted by y (1), y (2),..., Y (N) according to the order in the matrix Y, that is, the time series order.

The boundary determination unit 41 reads out the principal component selected by the principal component selection unit 31 from the principal component matrix storage unit (not shown) according to the principal component number input from the principal component selection unit 31. The boundary determination unit 41 uses the principal component to determine whether or not the boundary determination unit 41 selected by the principal component selection unit 31 from the principal component matrix Y has a pattern boundary in the determination target N frame. . When it is determined that there is a pattern boundary, the boundary determination unit 41 determines the end time of the previous pattern and the start time of the subsequent pattern as the position of the pattern boundary. The boundary determination unit 41 uses the value of the principal component at the extreme point of the principal component (the point at which the value is maximized or minimized) to determine whether or not the principal component satisfies a predetermined condition represented by a series of formulas described later. By determining whether or not there is a pattern boundary. The boundary determination unit 41 performs determination according to the following procedure.
Procedure 1: Find the poles of the main component using equation (4). In the following, the time at which the principal component takes a pole (hereinafter referred to as the pole time or simply the time) is indicated by c (1), c (2), c (3),.

  Procedure 2: Perform the following initialization. For the times c (1) and c (2) of the first two extreme points, the forward average value fm (1) and the forward average value fm (2) are calculated using Equation (5).

Procedure 3: A forward average value fm (j) at time c (j) is calculated using Expression (5) as a feature quantity for determining the presence or absence of a pattern boundary.
Further, the backward average values bm (j + 1) and bm (j + 2) at the times c (j + 1) and c (j + 2) are calculated using the equation (6) as the feature amount for determining the presence / absence of the pattern boundary. .

  Procedure 4: The average value mm (j) of the forward average value at time c (j) is calculated using equation (7).

  The standard deviation stdm (j) of the forward average value at time c (j) is calculated using Equation (8).

  Using equation (9), the difference diff1 (j) between the rear average value bm (j + 1) at the time c (j + 1) and the average value mm (j) of the front average value, and the rear at the time c (j + 2) A difference diff2 (j) between the average bm (j + 2) and the average value mm (j) of the forward average value is calculated.

  When Expression (10) is established, it is determined that there is a pattern boundary, and when it is not established, it is determined that there is no pattern boundary.

The boundary determination unit 41 performs the following according to the success or failure of the expression (10).
Case 1: When the formula (10) is not established, that is, when it is determined that there is no pattern boundary, there is a pole at the time c (j + 3) three times in the time direction from the pole at the time c (j) to be determined. Judge whether to do. If it is determined that the pole at time c (j + 1) exists, the procedure 3 and subsequent steps are repeated after the pole at time c (j + 1).
When it is determined that there is no extreme point at time c (j + 3), it is determined how many principal components are used. In the case of the first principal component or the second principal component, the principal component number, the first frame number, and the last frame number are output to the principal component selection unit 31. In the case of the third principal component, the first frame number and the last frame number are output to the data adding unit 51.
Case 2: When Formula (10) is satisfied, that is, when it is determined that there is a pattern boundary, a specific boundary position is determined. The diff1 (j) and diff2 (j) of the equation (9) are calculated in order from the pole at the time c (j) that is the object of the determination, and the success or failure of the equation (11) is determined. The time of the first extreme point where equation (11) is established is determined as the end time of the previous pattern, the time of the next extreme point is determined as the start time of the next pattern, and these times are output as boundary data. That is, the boundary determination unit 41 uses, for each of the extreme points of the principal component, a forward average value that is an average value of the principal component values in a predetermined period before the extreme point and an average value of the principal component values in a predetermined period after the extreme point. A certain backward average value is calculated, and based on the forward average value, the distribution center and distribution width of the principal component value are determined for each principal component extreme point, and the difference between the backward average value and the distribution center is mainly determined. As the deviation between the component value and the distribution center, the pattern boundary of the multi-dimensional time-series data X ′ is determined and output based on the point where the deviation is larger than the distribution width. When calculating the deviation between the principal component value and the center of the distribution, the backward average value is used as the principal component value because the principal component value is locally large or small, that is, the extreme point due to noise is regarded as the pattern boundary. This is because it is not judged. If there is no extreme point where equation (11) holds, the determination is made that there is no pattern boundary.

  FIG. 3 is a graph showing an example of each value calculated from the principal components when the multidimensional time-series data analysis apparatus 1 performs pattern boundary determination. The boundary determination unit 41 determines the presence / absence of a pattern boundary in order from time c (3). Here, Equation (10) is transformed to Equation (12) for explanation.

  In the same figure, bm (5) ≦ mm (3) + TH1 · stdm (3) and bm (5) ≧ mm (3) −TH1 · stdm (3), and equation (12) does not hold, so equation (10) ) Also does not hold. Therefore, the boundary determination unit 41 determines in step 4 that there is no pattern boundary, and determines the next extreme point. Similarly, equation (10) does not hold at times c (4) to c (14). Therefore, the boundary determination unit 41 determines that there is no pattern boundary, and determines the next extreme point. Since the equation (12) is established at the time c (15), the equation (10) is established. The boundary determination unit 41 determines that there is a pattern boundary in step 4 and determines a specific boundary position. Since Expression (11) is established at time c (15), the boundary determination unit 41 determines time c (15) as the end time of the previous pattern and time c (16) as the start time of the next pattern.

In addition, when Expression (10) is established in step 4, that is, when the boundary determination unit 41 determines that there is a pattern boundary, the autocorrelation of the principal component value is calculated as follows, and based on the autocorrelation The position of the pattern boundary may be obtained.
First, the boundary determination unit 41 calculates the autocorrelation R (τ) of the principal component coordinates y (t) using Expression (13).

Next, the boundary determination unit 41 calculates a time τ ^* that takes the maximum value R (τ ^* ) except for the time τ = 0 among the points where the autocorrelation R (τ) becomes maximum, and immediately before the time τ ^* . The extreme point time c (j) and the immediately following extreme point time c (j + 1) are determined as the end time of the previous pattern and the start time of the next pattern, respectively.

Step 5: The boundary determination unit 41 that has performed up to step 4 ends the processing in accordance with the following cases.
Case 1: When it is determined in step 4 that a pattern boundary exists, the end time of the previous pattern and the start time of the next pattern are output as the position of the pattern boundary, and the start time of the next pattern is sent to the data acquisition unit 11 Output.
Case 2: When a determination is made using the third principal component and it is determined in step 4 that there is no pattern boundary, the data addition unit uses the first frame number and the last frame number input from the principal component selection unit 31. To 51.
Case 3: In other cases (when determination is made using the first principal component or the second principal component and it is determined that there is no pattern boundary in step 4), the principal component number input from the principal component selection unit 31 is Output to the principal component selector 31.
Note that, as described in the description of the principal component selection unit 31, the principal component used by the boundary determination unit is not limited to the first principal component to the third principal component, but the K principal component (K is 1 or more and the principal component). The determination may be made by sequentially using up to an arbitrary integer equal to or less than the number of rows of the matrix.

  The data addition unit 51 is a target on which the principal component analysis unit 21 performs the principal component analysis when the boundary determination unit 41 determines that there is no pattern boundary using the third principal component configured by each element of the third row. To P frames (P is an arbitrary positive integer). The number of frames added by the data adding unit 51 may be changeable.

FIG. 4 is a flowchart showing a processing procedure of the multidimensional time series data analysis apparatus 1. When the multidimensional time-series data analysis device 1 starts pattern boundary determination, first, the data acquisition unit 11 acquires motion data as frames at a plurality of predetermined times and stores them in the data memory 12. The data acquisition unit 11 outputs the first frame number “1” and the last frame number N as the N frame numbers to the principal component analysis unit 21 (step Sa1). The principal component analysis unit 21 reads the first frame to the Nth frame from the data memory 12 to perform principal component analysis, generates a matrix Y of the principal component space, and writes it to a principal component matrix storage unit (not shown). . The principal component analyzing unit 21 outputs the first frame number “1” and the last frame number N to the principal component selecting unit 31 (step Sa2). The principal component selection unit 31 outputs the number “1” indicating the first principal component, the number “1” of the first frame, and the number N of the last frame to the boundary determination unit 41 (step Sa3). In the following, the number indicating the number of the main component is referred to as the main component number.

The boundary determination unit 41 reads the first principal component from the principal component matrix storage unit (not shown), determines whether or not there is a pattern boundary in N frames using the first principal component, and determines the pattern. If it is determined that there is a boundary, the end time of the previous pattern and the start time of the next pattern are output as boundary data (step Sa4). If it is determined that there is a pattern boundary, the boundary determination unit 41 outputs the start time Q of the next pattern to the data acquisition unit 11 (step Sa4—with boundary). The data acquisition unit 11 checks the number of remaining frames after the start time of the next input pattern (step Sa21), and when there are not more than N, ends the processing of the multidimensional time series data analysis device 1 (step Sa21-). Less than N frames). On the other hand, when there are N or more (step Sa21−N frames or more), the data acquisition unit 11 outputs the number Q of the first frame and the number Q + N−1 of the last frame to the principal component analysis unit 21 ( Step Sa1). Thereafter, the multidimensional time series data analysis apparatus 1 repeats step Sa2 and the subsequent steps.
Note that the data acquisition unit 11 does not need to acquire all data at the start of processing of the multidimensional time-series data analysis device 1, and may acquire data in parallel with the processing of each unit. In this case, the determination of the number of frames is performed using the number obtained by adding the number of frames scheduled to be acquired to the number of frames already acquired. Alternatively, information on whether or not the data acquisition is completed may be acquired, and the process may be ended when the data acquisition is completed and the remaining number of data is insufficient.

If it is determined in step Sa4 that there is no pattern boundary, the boundary determination unit 41 outputs the principal component number “1” to the principal component selection unit 31 (step Sa4—during determination). The principal component selection unit 31 outputs the second principal component next to the first principal component (data in the second row of the matrix Y) and the principal component number “2” to the boundary determination unit 41 (step Sa3). . The boundary determination unit 41 repeats Step Sa4 using the second principal component (Step Sa4). If it is determined that there is no pattern boundary for the second principal component, the boundary determination unit 41 outputs the principal component number “2” to the principal component selection unit 31 (step Sa4—during determination). The principal component selection unit 31 outputs the third principal component (the data in the third row of the matrix Y) next to the second principal component and the principal component number “3” to the boundary determination unit 41 (step Sa3). . The boundary determination unit 41 repeats Step Sa4 using the third principal component (Step Sa4).
If it is determined that there is no pattern boundary for the third principal component, the boundary determination unit 41 outputs the first frame number “1” and the last frame number N to the data addition unit 51 (step Sa4—Boundary). None). The data adding unit 51 refers to the data memory 12 to check the number of all frames, and determines whether there are P or more remaining (step Sa31). If there are no more than P remaining, the multidimensional time-series data analysis device 1 ends the process (step Sa31—less than P). If there are P or more remaining (step Sa31-P or more), the data adding unit 51 adds P frames, that is, the first frame number “1” and the last frame number N + P are the main components. It outputs to the analysis part 21 (step Sa32). Thereafter, the multidimensional time series data analysis apparatus 1 repeats step Sa2 and the subsequent steps.

FIG. 5 is a flowchart showing the procedure of the process in step Sa4 of FIG.
When the principal component number, the first frame number, and the last frame number are input from the principal component selection unit 31, the boundary determination unit 41 obtains the extreme points of the principal component (step Sb1). Next, the boundary determination unit 41 calculates, as initialization, forward average values fm (1) and fm (2) for the times c (1) and c (2) of the first two extreme points (step) Sb2). Next, the boundary determination unit 41 uses the forward average value fm (3) at the time c (3) of the third extreme point and the times of the fourth and fifth extreme points as feature quantities for determining the presence or absence of the boundary. The rear average values bm (4) and bm (5) in c (4) and c (5) are calculated (step Sb3). The boundary determination unit 41 determines the presence / absence of a pattern boundary at time c (3) using the calculated feature amount (step Sb4). If it is determined that there is no pattern boundary (step Sb4—no boundary), whether there is a third extreme point in the time direction from the determination target extreme point (in this case, whether there is an extreme point at time c (6)). Determination is made (step Sb21). If it is determined that the extreme point exists (step Sb21-present), step Sb3 and subsequent steps are sequentially repeated after time c (4) of the next extreme point after time c (3) of the extreme point at which the presence / absence of the pattern boundary is determined. If it is determined in step Sb4 that no pole exists (step Sb21-None), it is determined what number of principal components is used (Sb22). In the case of the first principal component or the second principal component (Sb22—first and second principal components), the principal component number, the first frame number, and the last frame number are output to the principal component selector 31 ( Step Sb31), the process is terminated. In the case of the third principal component (Sb22—third principal component), the first frame number and the last frame number are output to the data adding unit 51 (step Sb41), and the process is terminated.
If it is determined in step Sb4 that there is a boundary (step Sb4-boundary), the end time of the previous pattern and the start time of the next pattern are determined as specific boundary positions (step Sb51). If these times can be determined (step Sb52—boundary position determination), the end time of the previous pattern and the start time of the next pattern are output as boundary data. In addition, the start time of the next pattern is output to the data acquisition unit 11 (step Sb61), and the process ends. If a specific boundary position cannot be determined (step Sb52—no boundary position determination), the determination is made that there is no pattern boundary, and step Sb22 and subsequent steps are performed.

In the present embodiment, the data acquisition unit 11 uses data of other degrees of freedom obtained by removing the degree of freedom of the route from the acquired motion data as data of each frame. The relative position of the joint may be calculated and used as data for each frame. In this case, the data acquisition unit 11 receives the basic pose data and the frame data as human body skeleton angle information data, and calculates the position of each joint. Here, the basic pose data is data for specifying the basic pose, such as the position of the root in each pose defined as the basic pose, the angle of each joint, and the length of each bone. The frame data indicates the amount of movement of each joint from the basic pose at each data measurement time. Here, angle information is used as the movement amount.
The data acquisition unit 11 calculates the relative position M ⁱ (t) of the i-th joint viewed from the (i−1) -th joint at the data measurement time t using Expression (14).

The data acquisition unit 11 calculates the position p ^k (t) of the k-th joint at the data measurement time t using Expression (15). p ^k (t) is indicated in three-dimensional coordinates. In the following description, time t takes values of 0, 1, 2,..., T−1, and time t is also handled as an index of frame data. Here, T is the number of frame data input to the data acquisition unit 11.

However, the 0th (i = 0) joint is the root. The matrix R _axis ^{i−1, i} (t) is a matrix indicating the positional relationship between the i-th joint and its parent joint (“i−1” -th joint) in the basic pose, and is included in the basic pose data. . A local coordinate system is set for each joint, and the matrix R _axis ^{i−1, i} (t) indicates the correspondence of the local coordinate system between joints in a parent-child relationship. R ⁱ (t) is a matrix indicating angle information of the i-th joint using the local coordinate system set for the i-th joint, and is included in the frame data. T ⁱ (t) is a matrix indicating the transition between the i-th joint and its parent joint, specifically, a matrix indicating the bone length between the i-th joint and its parent joint, and the basic pose. Included in the data.
The data acquisition unit 11 calculates the relative position (joint relative position) p ′ ^k (t) of the k-th joint with respect to the route at time t using Expression (16).

Here, p ^root (t) is the position (p ⁰ (t)) of the route (0th joint) at time t. The data acquisition unit 11 uses the expression (17) to convert the frame x (t) at time t into x (t) = (p ′ ¹ (t), p ′ ² (t),..., P ′ ^K ( t)). K is the number of joints excluding the root.

The data acquisition unit 11 may receive the input of human body skeleton type acceleration information data, calculate the joint relative velocity of each joint at each time, and use this as the data of each frame. In this case, the data acquisition unit 11 first receives an input of the initial speed of each joint. Next, the data acquisition unit 11 calculates the velocity v ^k (t) of the k-th joint at time t using Equation (17). In equation (17), the integrated value of acceleration is added to the initial speed.

Where v ₀ ^k is the initial speed of the k-th joint. a ^k (τ) is the acceleration of the k-th joint at time τ, and is included in the human skeleton-type acceleration information data input to the data acquisition unit 11.
Next, the data acquisition unit 11 calculates the position p ^k (t) of the k-th joint at time t using Expression (18).

Note that the data acquisition unit 11 may calculate the joint relative position p ′ ^k (t) at time t using the above equation (16).

  As described above, the multidimensional time series data analysis apparatus 1 of the present embodiment is different from the multidimensional time series data analysis apparatus that performs pattern boundary determination using a Gaussian mixture model or pattern learning technique. It is not necessary to predetermine the parameter values of these, to prepare in advance, to create training data by the user, or to present correct answers by the user. In addition, the multidimensional time-series data analysis apparatus 1 of the present embodiment performs determination of the pattern boundary in the N frame that is the determination target, using the principal component at each time obtained by one principal component analysis. Therefore, the principal component analysis is performed every time the frame is increased by a certain number, and the number of principal component analysis is less than that of the multi-dimensional time series data analysis device that determines the pattern boundary based on the difference in the result of each principal component analysis. Therefore, the pattern boundary can be determined with a smaller calculation amount.

  In addition, the program for realizing the functions of the data acquisition unit 11, the principal component analysis unit 21, the principal component selection unit 31, and the boundary determination unit 41 in FIG. 1 is recorded on a computer-readable recording medium, Processing of each unit may be performed by causing a computer system to read and execute a program recorded on the recording medium. Here, the “computer system” includes an OS and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design changes and the like without departing from the gist of the present invention.

  The present invention is suitable for use in a multidimensional time series data analysis apparatus. The present invention is not limited to the human motion data handled in the above-described embodiment, and can be applied to various multidimensional time series data in which elements are indicated by numerical values.

DESCRIPTION OF SYMBOLS 1 ... Multidimensional time series data analyzer 11 ... Data acquisition part 12 ... Data memory 21 ... Principal component analysis part 31 ... Principal component selection part 41 ... Boundary determination part 51 ... Data addition part

Claims (7)

A multidimensional time series data storage unit for storing multidimensional time series data in which multidimensional data at each of a plurality of times is arranged according to a time series;
A principal component matrix storage unit for storing the principal component matrix;
A principal component analysis unit that generates a principal component matrix by performing principal component analysis on the multidimensional time series data read from the multidimensional time series data storage unit and writes the principal component matrix to the principal component matrix storage unit ;
A principal component selection unit that selects a plurality of elements included in one row in the principal component matrix as principal components;
The principal component selected by the principal component selection unit is read from the principal component matrix storage unit, the distribution center and distribution width of the principal component value are determined, and the principal component value and the distribution center are determined. A multi-dimensional time-series data analysis apparatus, comprising: a boundary determination unit that determines and outputs a pattern boundary of the multi-dimensional time-series data based on a time when the deviation becomes larger than the distribution width.   For each of the extreme points of the principal component, the boundary determination unit is configured to calculate a forward average value that is an average value of the principal component values in a predetermined period before the extreme point and a value of the principal component in the predetermined period after the extreme point. A rear average value that is an average value is calculated, and based on the front average value, the distribution center and the distribution width are determined for each of the extreme points, and a difference between the rear average value and the distribution center is calculated. The multi-dimensional time-series data analysis apparatus according to claim 1, wherein the shift is the shift.   The principal component analysis unit generates a variation data matrix by subtracting a matrix of average values in the time direction of the multidimensional time series data from the matrix of the multidimensional time series data, and a singular value for the variation data matrix The multidimensional time series data analysis apparatus according to claim 1 or 2, wherein a principal component matrix is generated by performing decomposition.   The principal component selection unit selects each element of the first row of the principal component matrix as a principal component, and when the boundary determination unit cannot determine the position of the pattern boundary using the principal component, Each element of the second row of the component matrix is selected as a principal component. Hereinafter, every time the boundary determination unit cannot determine the position of the pattern boundary, the K-th row from the top of the principal component matrix (K is 1 or more and 4. Each element in one row is selected as a principal component in order from the upper row in the principal component to be determined which is a sub-matrix up to an arbitrary integer equal to or less than the number of rows of the principal component matrix). The multidimensional time-series data analysis device described. The main component stores multi-dimensional time series data in the multi-dimensional time series data storage unit, and reads data for N times (N is an arbitrary positive integer) in chronological order from the multi-dimensional time series data storage unit. The analysis unit is instructed, and when the boundary determination unit determines and outputs the position of the pattern boundary, the data for N times in the oldest order from the multidimensional time series data after the position of the pattern boundary is A data acquisition unit that instructs the principal component analysis unit to read from the series data storage unit;
When the boundary determination unit cannot determine the position of the pattern boundary for all the principal components included in the determination target principal component, the frame in which the principal component analysis unit has performed the principal component analysis and a frame newer than the frame A data addition unit that instructs the principal component analysis unit to read out P frames (P is an arbitrary positive integer) from the oldest to the multidimensional time-series data storage unit. The multidimensional time series data analysis apparatus according to claim 4. A multidimensional time series data storage unit for storing multidimensional time series data in which multidimensional data at each of a plurality of times is arranged according to a time series;
A principal component matrix storage unit for storing the principal component matrix;
A principal component analysis unit that generates a principal component matrix by performing principal component analysis on the multidimensional time series data read from the multidimensional time series data storage unit and writes the principal component matrix to the principal component matrix storage unit ;
A principal component selection unit that selects a plurality of elements included in one row in the principal component matrix as principal components;
The principal component selected by the principal component selection unit is read from the principal component matrix storage unit, the distribution center and distribution width of the principal component value are determined, and the deviation between the principal component value and the distribution center is determined. Is calculated by calculating the autocorrelation of the principal component value when there is a point where the distribution width is larger than the distribution width, and determining and outputting the position of the pattern boundary based on the time when the autocorrelation value becomes maximum A multidimensional time-series data analysis apparatus comprising: a determination unit. By performing principal component analysis on the multidimensional time series data read from the multidimensional time series data storage unit that stores multidimensional time series data in which multidimensional data at each of a plurality of times is arranged according to the time series A principal component analysis step of generating a principal component matrix and writing it to the principal component matrix storage unit ;
A principal component selection step of selecting a plurality of elements included in one row in the principal component matrix as principal components;
The principal component selected in the principal component selection step is read from the principal component matrix storage unit, the distribution center and distribution width of the principal component value are determined, and the deviation between the principal component value and the distribution center is determined. A multidimensional time series data analysis program for causing a computer to execute a boundary determination step of determining and outputting a pattern boundary of the multidimensional time series data based on a point where becomes larger than the distribution width.

Images