JP4257421B2 - Motion body motion data segmentation processing device. - Google Patents

Description

  The present invention is a motion body motion data segment that performs information processing for temporally segmenting data obtained by measuring time-series motions of motion bodies having a plurality of independent motion degrees, such as humans and animals. The present invention relates to a processing apparatus.

  Conventionally, there are the following processing methods for the time segmentation processing method of the time series data representing the motion of the moving body. Note that the moving body here is a human body, an animal body, or a non-living object. Furthermore, virtual objects that are visualized by computer graphics are also targeted. Further, there may be a single operating body or a plurality of operating bodies.

(1) Motion motion segmentation method at equal time intervals:
This method is a method of segmenting so that the length of the segment becomes a constant time interval regardless of the contents of the time-series motion body motion time-series data. Specifically, there are a continuous photography method of motion of a moving body by photographing a strobe at regular time intervals, and a moving image photography method using a surveillance camera with a low imaging time frequency. It is used when motion data is less than in continuous moving image capturing at a normal high imaging time frequency, which is advantageous for analysis, search, and storage of motion information.

(2) Motion motion segmentation method based on trigger conditions:
This method obtains time-segmented motion body motion time-series data by capturing or recording a motion body motion picture or video only when the situation of the photographing target satisfies a specific condition (trigger condition). Is the method. For example, as shown in Non-Patent Document 1, there is a method of segmenting dance motion body motion time-series data at the timing of the dance music.

(3) Motion motion segmentation method based on motion motion speed or kinetic energy:
This method measures the speed or kinetic energy of the whole body or a part of the moving body of the subject to be imaged, determines the time point when the kinetic energy is below a specific threshold as the motion segment, This is a method for segmenting series data. For example, as shown in Non-Patent Document 2, the speed of the specific position is calculated for data obtained by measuring the displacement of the specific position of the human arm and foot in time series, and the sum of absolute values of the speed is constant. There is a method of considering a segment of motion when the threshold value is smaller than.
Takaaki Shiratori, Atsushi Nakazawa, Katsushi Ikeuchi, “Analysis of Dance Movement Considering Rhythm of Rhythm”, Proceedings of the 21st Annual Conference of the Robotics Society of Japan, 2003 Nakazawa, Nakaoka, Ikeuchi, “Extraction of dance motion primitives from motion capture data”, Proceedings of the 19th Annual Conference of the Robotics Society of Japan, 2001

  By the way, the conventional motion body motion segmentation method still has the following problems to be solved. That is, there are the following problems in the conventional motion body motion segmentation method in temporal segmentation of motion body motion data.

(1) The motion body motion segmentation method at equal time intervals performs segmentation regardless of the state of motion. For this reason, if the time interval is too small, there is a risk of dividing a portion that should be interpreted as a series of operations. On the other hand, if the time interval is too long, even a portion in which the operation changes drastically is interpreted as a group of operation segments, and the possibility that the segmentation that should be performed is not performed increases.

(2) The motion body motion segmentation method based on the trigger condition has the same disadvantages as the motion body motion segmentation method at equal time intervals if the trigger condition is irrelevant to the content of the motion body motion. Although the method of setting the trigger condition according to the contents of the motion of the motion body can be considered, a widely adaptable and quantitative guideline is currently presented as a method of determining the trigger condition for appropriately segmenting the motion of the motion body. Not. For this reason, it is difficult to appropriately set the trigger condition.

(3) The motion body motion segmentation method according to the speed of motion body motion or the magnitude of kinetic energy causes a problem when the motion body motion is slow or not an overall motion. For example, in the case of segmentation of movement of one human body, generally, kinetic energy and movement speed are small for movement by only a part of the body such as manual work, and in case of whole body movement such as walking Then kinetic energy and speed are large. For this reason, this method has a drawback that data of whole body movement is not easily segmented, and data of manual movement is easily segmented.

  Even if the moving body is not stationary, it should be regarded as a segment of movement if the characteristics of the moving body motion change greatly. However, this method has a drawback that the movement is difficult to be segmented when the moving body is not stationary.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to perform the motion of a motion body having a plurality of independent degrees of freedom of motion, for example, a motion body such as a human being or an animal. An object of the present invention is to provide a motion body motion data segmentation processing apparatus that performs information processing for segmenting data measured in time series in terms of time.

  In order to achieve the above object, in the motion body motion data segmentation processing apparatus of the present invention, as a basic configuration, displacement, speed, acceleration, joint angle, joint angular velocity, joint angle related to motion of a plurality of parts of the motion body It is assumed that a configuration for temporally segmenting motion data is provided based on temporal changes in correlation between dynamic feature quantities such as acceleration. In other words, the motion body motion data segmentation processing device inputs time series data of positions, velocities, or accelerations of a plurality of parts of the motion body, calculates feature quantities related to the correlation of motion generated between the data, and calculates By clustering the distribution of the feature amount, the timing position at which the motion is segmented is determined, and the time-series data is segmented.

  In general, the motion of human and animal motions is controlled by distributing a small number of motion commands to the many muscles of the motion. Therefore, the switching of the operation control can be observed as a change in the waveform of the operation command and a change in the distribution method of the operation command.

  Many of the conventional motion body segmentation methods observe changes in the waveform of the motion command, but do not explicitly observe changes in the motion command distribution method.

  Therefore, in the motion body motion data segmentation processing device of the present invention, not only the motion command waveform change but also the motion command distribution method change is observed, and motion body motion control switching is detected accordingly. Thus, it is determined as a time node of the motion body movement.

Therefore, as a first aspect of the present invention, the motion body motion data segmentation processing device according to the present invention has a basic configuration in which motion body motion that temporally segments time-series data obtained by measuring motion of the motion body is provided. Data segmentation processing device that inputs motion body time series data consisting of time series data of displacement of at least two parts of the motion body, joint angles of joints, time differential values of these physical quantities, and time integral values. The motion body motion time series data input means, and at least one time point included in the data series of at least two components of the motion body motion time series data. The inter-part motion correlation index calculating means for calculating the inter-part motion correlation index of the index defined as the index representing the index, and the inter-part motion correlation index value at at least two time points Flip and that having a the operation segment point determination unit determines the operation segment time.

  In this case, in the motion body motion data segment processing device of the present invention, the motion body motion time series data input by the motion body motion time series data input means is a human or animal limb, head, tail, end, or The joint angle for the displacement of the joint point, time-series data composed of any one of these physical quantities, the time differential value, the higher-order time differential value, the time integration value, the higher-order time integration value, and the inter-part motion correlation index The inter-part motion correlation index calculated by the calculation means is a product-moment correlation coefficient, rank correlation coefficient, inner product, weighted sum of the data values, or the data of at least two time point data of the motion body motion time series data. A weighted product of values is calculated, and the calculation result is output as an inter-part motion correlation index.

  Further, in the motion body motion data segmentation processing device of the present invention, the data is temporally segmented into the data according to the input of motion body time series data obtained by measuring motion of the motion body in time series. Action body motion segmentation means for outputting motion body time series data with segment information to which information is added, and data contents of at least one time point in at least one motion segment of motion body time series data with segment information Motion body motion summarizing means for outputting motion body motion summary data obtained by extracting the motion body motion summary data.

Further, the present invention as a second embodiment, an act body movement data segment processing program for executing processing for temporally segmenting the time series data obtained by measuring the motion of the operation member by a computer, the basic process steps Then, a first step of inputting motion body motion time series data constituted by time series data of displacement of at least two parts of the motion body, joint angles of joints, time differential values of these physical quantities, and time integral values; Inter-part movement of an index defined as an index representing the strength of correlation between the constituent elements according to data at at least one time point included in the data series of at least two constituent elements of the motion body movement time series data A second step of calculating a correlation index, and a third step of determining a motion segment time point according to the value of the motion correlation index between parts at at least two time points. Ru is executed by a computer processing with.

  In this case, in the motion body motion data segment processing program, the motion body motion time series data input in the first step is the joint angle for the displacement of the extremities, head, tail, end, or joint point of a human or animal. , Time series data constituted by any one of these time-dependent values, higher-order time-differentiated values, time-integrated values, and higher-order time-integrated values, and the inter-part motion correlation index calculated by the second step is motion body motion Calculate product-moment correlation coefficient, rank correlation coefficient, inner product, weighted sum of the data values, or weighted product of the data values of at least two time points of time-series data, Output as motion correlation index.

  In the motion body motion data segmentation processing program according to the second aspect of the present invention, the data is further added to the data according to the input of motion body time series data obtained by measuring motion of the motion body in time series. A fourth step of outputting motion body motion time series data with segment information to which information segmented in time is added, and at least one time point of at least one motion segment of motion body motion time series data with segment information The computer is caused to execute the process of the fifth step of outputting the motion body motion summary data that is obtained by extracting the data contents of the above.

  According to the motion body motion data segment processing apparatus and motion body motion data segment processing program of the present invention, when performing processing for segmenting motion body motion time-series data, the setting of the time segment is set as the element of motion body. Since it performs based on the correlation state of the motion state which exists in between, the temporal unit and the segment of the motion can be determined without depending on the kinetic energy of the moving body. Furthermore, the calculation of the correlation state of the movement situation can be basically applied regardless of the mechanism of the moving body and the number of movement elements.

  Hereinafter, an embodiment for carrying out the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing a hardware configuration of a system configuration of a motion object movement data segmentation processing apparatus according to the present invention, and FIG. 2 shows a software configuration. In FIG. 1, 100 is an input unit that receives input operations from a keyboard and mouse, 101 is a keyboard for input operations such as numerical data, 102 is a pointing device mouse for selection input operations, and 103 is a microprocessor ( CPU) and an arithmetic unit composed of a system device including a memory and the like, a display unit composed of a display device, for example, and a database composed of a hard disk device.

  FIG. 2 is a diagram showing an outline of the software configuration of the motion object movement data segmentation processing apparatus according to the present invention as functional blocks. As shown in FIG. 2, the software configuration of the motion body motion data segment processing device includes a motion body motion time series data input processing unit 202, an inter-part motion correlation index calculation processing unit 203, and a motion segment time point determination processing unit 204. Is configured as the main part.

  The motion body motion time series data input processing unit 202 performs processing for inputting motion body motion time series data 201. The moving body motion time series data 201 is composed of time series data of displacement of at least two parts of the moving body, joint angles of joints, time differential values of these physical quantities, and time integration values, as will be described in detail later. It is series data.

  The time series data of the motion of the moving body here is created by acquiring the data in advance by using, for example, a motion capture system, and performing necessary preprocessing and storing it as data in the database 105. It shall be. Further, the motion body motion time series data input processing unit 202 may be configured to perform input processing including this preprocessing. In other words, as input data, from the time series data of the position data of at least two parts of the moving body detected by the sensor, the joint angle of the joint, the time differential value of these physical quantities, the time series data of the time integral value It may be configured to be generated.

The inter-part motion correlation index calculation processing unit 203 calculates the inter-part motion correlation index according to data at at least one time point included in the data series of at least two components of the motion body motion time series data. I do. Then, according to the calculated result, the motion segment time point determination processing unit 204 determines the motion segment time point according to the value of the inter-part motion correlation index at at least two time points. The determined result data is output as operation segment time point data 205. Data is processed and displayed on the display unit 104 as necessary.

  In order to facilitate understanding of the embodiment of the present invention, a case where the time series data describing the motion of the moving body is realized with a minimum system configuration for segmenting in time will be specifically described. First, the motion body motion time series data will be described.

  FIG. 3 is a diagram for explaining a portion to be observed in a human body that is an operation body to be analyzed. As the movement data of each position of these parts, motion time series data based on the movement of each part of the moving body is obtained by using a motion capture system. For example, data obtained by measuring 11 XYZ coordinates of A to K of the human body at a sampling rate of 60 Hz is used as the motion body motion time series data. In this embodiment, specifically, the number of times of data sampling of the motion body motion time series data is 72. This is N.

  FIG. 4 is a diagram in which the posture of the human body corresponding to the motion body motion time series data to be processed is extracted and drawn at equal time intervals. FIG. 4 shows the operation of the operating body as a strobe-like continuous still image at regular time intervals. That is, the positions of the observation target parts A to K shown in FIG. 3 are represented by black circles, and the motion body motion time-series data is data described by, for example, ASCII data or binary data. As a file in a computer file system.

  The motion body motion time series data 201 to be processed here is not limited to the motion time series data of each part of a single human body, and in the case of segmenting motions in other motion bodies, the corresponding Time-series data. FIG. 5 is an explanatory view showing a measurement part of a road traffic object to be measured. For example, as shown in FIG. 5, XYZ coordinate data of the gravity center positions of a person and a car on a traffic road can be used as the motion body motion time series data.

  In addition to the coordinate data, the moving body motion time series data 201 may be data such as a point moving speed or acceleration. When measuring a motion using a gyrometer or an accelerometer, the obtained data is often the speed and acceleration of coordinates and angles. Data in which physical quantities of different dimensions are mixed may be used. In that case, a process of converting the physical quantity is performed as necessary.

  Next, information processing of the motion body motion data segment processing device according to the present invention will be described. In this embodiment, all means are realized using a computer. The software configuration shown in FIG. 2 is a block diagram showing the flow of information processing. Details of each processing unit will be described in the order of processing.

  First, the motion body motion time series data input processing unit 202 will be described. The motion body motion time series data input processing unit 202 is a processing module for inputting a motion body motion time series data file as described above. For example, it is realized by a file system of a computer as an example.

本実施例では、動作体運動計測のサンプリング時間は１／６０［秒］である。動作体運動時系列データを時間的に微積分の操作を加えたデータを算出して分析に用いることもできる。離散時間サンプリングでの時系列データの時間的な微積分の計算は、データに下記に例示する演算を施すことで、近似的に実現できる。動作体の各部位の例えば点Ａについては、次に示す式（２）によって、点Ａの座標の時間積分値が求まる。

式（３）によって、点Ａの座標の時間微分値、すなわち点Ａの移動速度ベクトルが求まる。

式（４）によって、点Ａの速度の絶対値が求まる。

式（５）によって、点Ａの移動加速度ベクトルが求まる。

式（６）によって、点Ａの移動加速度が求まる。

観測した点の間の相対位置ベクトルは、式（７）に記述する位置ベクトルの減算を行うことにより求められる。式（７）では、点Ｂから点Ａに向かう相対位置ベクトルの計算例を示している。

観測した点の間の相対位置ベクトルについても、式（２）〜式（６）を用いて、時間積分および時間微分の計算や速度や加速度の計算を行うことができる。 Next, the inter-part motion correlation index calculation processing unit 203 will be described. The inter-part motion correlation index calculation processing unit 203 represents the correlation existing between the data elements of the input motion body motion time series data 201. A processing module for calculating an index. Since the motion body motion time series data is data sampled in discrete time, the form of the data is time series number data, for example, at the time nth of the XYZ data of the point A of each part of the motion body The data is in the form of a three-dimensional vector as described in equation (1).

In the present embodiment, the sampling time for the motion body motion measurement is 1/60 [second]. It is also possible to calculate data obtained by adding a calculus operation to the moving body motion time series data in time and use it for analysis. Calculation of temporal calculus of time-series data in discrete time sampling can be realized approximately by performing the operation exemplified below on the data. For example, for the point A of each part of the operating body, the time integral value of the coordinates of the point A is obtained by the following equation (2).

The time differential value of the coordinates of the point A, that is, the moving speed vector of the point A is obtained by the equation (3).

The absolute value of the velocity at the point A is obtained by the equation (4).

From equation (5), the moving acceleration vector of point A is obtained.

The moving acceleration of the point A is obtained by the equation (6).

The relative position vector between the observed points is obtained by performing subtraction of the position vector described in Expression (7). Equation (7) shows an example of calculating the relative position vector from point B to point A.

With respect to the relative position vector between the observed points, calculation of time integration and time differentiation, and calculation of speed and acceleration can be performed using Expressions (2) to (6).

観測した任意の３点によって構成される角度についても、前述したように、式（２）〜（６）のベクトルの計算を行う場合と同様に、時間積分および時間微分の計算や速度や加速度の計算を行うことができる。 The angle formed by the observed arbitrary three points is obtained by Expression (8). That is, the inner product value of the relative position vector is divided by the absolute value of the vector, and the inverse cosine function is obtained by calculation. Expression (8) exemplifies a mathematical formula for calculating an angle formed by the line segment AB and the line segment BC at the point B.

As described above, with respect to the angle formed by the arbitrary three observed points, as in the case of calculating the vectors of Equations (2) to (6), the calculation of time integration and time differentiation, the speed and acceleration Calculations can be made.

  By performing the respective calculation processes described above, vector values and absolute values such as coordinates, speeds, accelerations, and the like of the observed moving body parts are obtained. Similarly, the angle of the moving body is obtained by calculating the angle, angular velocity, angular acceleration, and the like.

  Since vector values and absolute values such as coordinates, speeds, accelerations, etc., of the parts of the moving body observed in this way are obtained, the inter-part movement correlation index is calculated. This will be specifically described.

  In the calculation process of the inter-part motion correlation index, the strength of the correlation between the two data series constituting the motion body motion time series data is calculated for each data time. The data value of the motion body motion time series data can be used for the calculation as it is. Further, it is also possible to use data converted into a time integral value, a time differential value, or an angle obtained by the above-described equations (2) to (8).

この第１種部位間動作相関指数は計算が簡便であり、また、データの値の大きさを反映した相関指数である。 Next, the calculation process will be described with reference to mathematical expressions, where α and β are data series to be calculated for the inter-part motion correlation index. The index representing the strength of correlation can be variously defined. For example, the following indicators can be used. From equation (9), the correlation index of data series α and β at data time n and correlation time delay d can be obtained. This will be referred to as a first type inter-part motion correlation index.

The first type inter-part motion correlation index is a correlation index that is easy to calculate and reflects the magnitude of the data value.

この第２種部位間動作相関指数は、第１種部位間動作相関指数を正規化したものであって、データの値の大きさに依らない相関指数となっている。 In addition, as an index representing another correlation strength, for example, a correlation index at the data time n and the correlation time delay d of the data series α and β can be used as shown in Expression (10). Here, this will be referred to as a second type inter-part motion correlation index.

The second type inter-part motion correlation index is obtained by normalizing the first type inter-part motion correlation index, and is a correlation index that does not depend on the magnitude of the data value.

但し、式（１１）における上線記号はデータ系列の平均を意味する。データ系列の平均を求めるためには、例えば、式（１２）に示される計算を行う。式（１２）はデータ系列αの平均値を求める場合の計算例である。

したがって、この第３種部位間動作相関指数は、データ系列の直流成分によらない相関係数となっている。 As an index representing another correlation strength, for example, as shown in the equation (11), another type of correlation index of the data series α and β at the data time n and the correlation time delay d can be used. . Here, this will be referred to as a third type inter-part motion correlation index.

However, the overline symbol in equation (11) means the average of the data series. In order to obtain the average of the data series, for example, the calculation shown in Expression (12) is performed. Expression (12) is a calculation example in the case of obtaining the average value of the data series α.

Therefore, the third type inter-part motion correlation index is a correlation coefficient that does not depend on the DC component of the data series.

この第４種部位間動作相関指数は、データの値の大きさおよびデータ系列の直流成分によらない相関係数である。 As another index representing the strength of the correlation, for example, as shown in the equation (13), another type of correlation index of the data series α and β at the data time n and the correlation time delay d can be used. Here, this is referred to as a fourth type inter-part motion correlation index.

The fourth type inter-part motion correlation index is a correlation coefficient that does not depend on the magnitude of the data value and the DC component of the data series.

Further, as another index indicating the strength of correlation, for example, as shown in the equation (14), another type of correlation index at the data time n and the correlation time delay d of the data series α and β is used. Can do. Here, this will be referred to as a fifth type inter-part motion correlation index.

Further, as another index representing the strength of the correlation, for example, as shown in the equation (15), another kind of correlation index of the data series α and β at the correlation time delay d from the data time n to n + s is expressed as follows. Can be used. Here, this is referred to as a sixth type inter-part motion correlation index. The sixth type inter-part motion correlation index is called covariance in mathematics.

As another index representing the strength of another correlation, for example, as shown in Expression (16), another kind of correlation index of the data series α and β at the correlation time delay d from the data time n to n + s is expressed as follows. Can be used. Here, this is referred to as a seventh type inter-part motion correlation index. The seventh type inter-part motion correlation index is called a product moment correlation coefficient in mathematics.

The inter-part motion correlation index can be calculated not only for a one-dimensional data series but also for a multi-dimensional data series. For example, the correlation index between the two-dimensional data series, that is, the vectors (α, β) and (γ, δ) constituted by the two data series is obtained by an arithmetic expression represented by the following formula (17). In this example, the correlation index with the correlation time delay d is obtained from the data time n to n + s. Here, this will be referred to as an eighth type inter-part motion correlation index.

第９種部位間動作相関指数は、数学的には信号（α，β）と（γ，δ）との内積と呼ばれる。この第９種部位間動作相関指数を部位間動作相関指数として用いることができる。 Similarly, for a higher-dimensional data series, the inter-part motion correlation index can be defined by using the sum of products of the inter-part motion correlation indices for each element of the data series. For example, according to Equation (18), another correlation index can be defined by the elements (α, β) and (γ, δ) of the data series. This is referred to as a ninth type inter-part motion correlation index.

The ninth type inter-part motion correlation index is mathematically called an inner product of signals (α, β) and (γ, δ). The ninth type inter-part motion correlation index can be used as the inter-part motion correlation index.

  In the motion body motion data segmentation processing apparatus of the present embodiment, any inter-part motion correlation index of the inter-part motion correlation indices as described above can be basically used. In addition, other correlation coefficients such as a rank correlation coefficient can be basically used. However, in practice, a correlation index suitable for analysis is selected depending on the motion characteristics of the moving body and taking into account the calculation cost.

  Further, in the motion body motion data segment processing device of the present embodiment, the human body motion data is converted into the joint angular velocity and the correlation analysis is performed. FIG. 6 is a diagram schematically illustrating a skeleton structure of a human body that is an operation object to be analyzed.

  For the purpose of grasping the entire movement of the human body, it is sufficient that time series data of each joint angle exists. In this case, the information processing efficiency can be improved by adopting the main joint angles of the human body mechanism as the object of the correlation analysis.

The inter-part motion correlation index calculation processing unit 203 refers to the motion body motion time-series data, and the major joint angles of human body mechanisms are ∠ABC, ∠ABD, ∠ABF, ∠BDE, ∠BFG, ∠BAH, ∠ The time series data of each angle of BAJ, ∠AHI, ∠AJK is calculated according to the above-described equation (8). Furthermore, the angular velocity is calculated by calculating the time difference between these angle data. Table 1 summarizes the variable names and meanings of these angular velocities and angles.

In the present embodiment, the inter-part motion correlation index calculation processing unit 203 calculates a first type inter-part motion correlation index that exists between these angular velocity data. Of course, other types of inter-part motion correlation indices can also be used. Specifically, C1 (d) [n], C2 (d) [n], C3 (d) [n], C4 (d) [n], C5 according to the following equations (19) to (23): (d) Calculate each value of [n].

C1 (d) [n] = N [n] + RS [n] + RE [n + d] + LS [n] + LE [n + d]
+ RT [n] + RK [n + d] + LT [n] + LK [n + d]… (19)
C2 (d) [n] = N [n] + RS [n + d] + RE [n + 2d] + LS [n + d] + LE [n + 2d]
+ RT [n + 2d] + RK [n + 3d] + LT [n + 2d] + LK [n + 3d]… (20)
C3 (d) [n] = N [n] + RS [n] + RE [n] + LS [n] + LE [n]
-RT [n + d] -RK [n + d] -LT [n + d] -LK [n + d]… (21)
C4 (d) [n] = RS [n] + RE [n] -LS [n + d] -LE [n + d]
+ RT [n] + RK [n] -LT [n + d] -LK [n + d]… (22)
C5 (d) [n] = RS [n] + RE [n] -LS [n + d] -LE [n + d]
-RT [n + d] -RK [n + d] + LT [n] -LK [n]… (23)

  These five values are values representing the gait and control form of the operating body. In this embodiment, the parameter d is calculated for each of -8, -4, -2, -1, 0, 1, 2, 4, and 8, respectively. The operation data can also be segmented by calculating only when d = 0.

Therefore, since nine types of values with different parameters d are calculated for five types of data, 45 inter-part motion correlation indices are calculated. Expression (24) is a vector representation of this inter-part motion correlation index CC.
CC [n] = {C1 (-8) [n], C1 (-4) [n], C1 (-2) [n], C1 (-1) [n],
C1 (0) [n], C1 (1) [n], C1 (2) [n], C1 (4) [n], C1 (8) [n],
C2 (-8) [n], C2 (-4) [n], C2 (-2) [n], C2 (-1) [n],
C2 (0) [n], C2 (1) [n], C2 (2) [n], C2 (4) [n], C2 (8) [n],
C3 (-8) [n], C3 (-4) [n], C3 (-2) [n], C3 (-1) [n],
C3 (0) [n], C3 (1) [n], C3 (2) [n], C3 (4) [n], C3 (8) [n],
C4 (-8) [n], C4 (-4) [n], C4 (-2) [n], C4 (-1) [n],
C4 (0) [n], C4 (1) [n], C4 (2) [n], C4 (4) [n], C4 (8) [n],
C5 (-8) [n], C5 (-4) [n], C5 (-2) [n], C5 (-1) [n],
C5 (0) [n], C5 (1) [n], C5 (2) [n], C5 (4) [n], C5 (8) [n]}
…(twenty four)

  FIG. 7 is a graph showing temporal transition of absolute values of all elements of the inter-part motion correlation index. FIG. 7 is a graph in which the absolute value of each element of CC [n] is displayed in the range of n = 1 to 72. The horizontal axis is time n. On the vertical axis, CC element rows are arranged, starting from the bottom, starting with C1 (-8) [n], C1 (-4) [n], and ending with the top C5 (8) [n] row . The density of drawing in each column is displayed in proportion to the absolute value of the element. The inter-part motion correlation index calculation processing unit 203 is realized as a program that operates on a computer. That is, it is configured as one processing module.

  Next, the motion segment time point determination processing unit 204 will be described. This is also configured as one processing module. The motion segment time point determination processing unit 204 refers to the data of the inter-part motion correlation index CC [n] as the calculation result calculated by the inter-part motion correlation index calculation processing unit 203. Here, the range of n to be referred to is from 1 to N. The motion segment time point determination processing unit 204 determines CC [n] according to a predetermined determination criterion in a section from n = 1 to N, and segments motion body motion time series data. Specifically, the time point (hereinafter, “segment start time point”) at the beginning of each segment of the time-series data is calculated and output.

  Various methods can be used to determine the motion segment time point for segmenting the motion body motion time-series data based on the motion correlation index between parts. For example, the inter-part motion correlation index is basically segmented based on the following interpretation. The period in which the movement correlation index between the parts is small is a period in which the movement mode is small, so it is regarded as a group of segments, and conversely, the period in which the movement correlation index between the parts is large is similar to the movement mode. It is considered to be a segment of movement.

  Such segmentation processing by determination processing is performed, for example, by one of the following two methods. As a first method, the absolute value of the time differential value of the inter-part motion correlation index is calculated, and a time point at which this is relatively large is determined as the segment head time point. The determination condition is that it is determined that the absolute value of the time differential value is greater than a specific threshold value and that the absolute value of the time differential value is in a higher specific position.

  As a second method, the inter-part motion correlation index CC [n] from n = 1 to N is classified using a clustering method, and the segment start time is determined based on the classification result. Next, a case where the second method is used will be described.

ただしＣＣｋ［ｎ］はベクトルＣＣ［ｎ］のｋ番目の要素である。 Clustering is to classify n and m into the same group if the vectors CC [n] and CC [m] are similar for different n and m. As a mathematical index representing the similarity between vectors, a norm distance, a vector inner product, a product-moment correlation coefficient, a rank correlation coefficient, or the like can be used. Here, for example, K-means clustering based on the primary norm distance is performed. The first-order norm distance of the vector is an amount defined by the equation (25). The primary norm distance is also called street distance.

However, CCk [n] is the kth element of the vector CC [n].

  In this embodiment, K-means clustering is used in a general manner. That is, first, the position of the cluster nucleus is appropriately determined. Here, for example, four nuclei are prepared. The designation of the number of nuclei may be given to the apparatus from the outside by inputting a command with an input apparatus such as a keyboard or a mouse. Also, the motion segment time point determination processing unit 204 can automatically determine the determination process by using the Sturges formula.

  The position of the nucleus is represented by a 35-dimensional vector. This is called Core1, Core2, Core3, Core4. The initial position of the nucleus may be any value as long as it does not overlap. Next, the motion segment time point determination processing unit 204 performs provisional classification processing.

  First, for n = 1 to N, the primary norm distance between CC [n] and each cluster nucleus is calculated. As a result, the number of the cluster nucleus having the smallest primary norm distance is set as the n temporary classification destination cluster number. Next, the motion segment time point determination means updates the nuclear position. An average value of CC [n] included in each provisional classification destination cluster is calculated and set as a new nucleus position. Thereafter, the motion segment time point determination means repeats the provisional classification and the update of the nuclear position several times. For example, repeat 10 times. The provisional classification at the completed stage is set as the classification result of CC [n].

  FIG. 8 is a graph showing the result of segmentation processing of motion body motion time series data. In the graph illustrated in FIG. 8, the Clustering Raw Result row indicates the classification result of CC [n]. For each n, a bar is drawn at a vertical position corresponding to the cluster number of the classification destination. Note that the cluster number is a nominal scale, and the size and order of the values are meaningless.

  In this embodiment, when the vector absolute value of CC [n] is small, the original operation is considered to be weak and is excluded from the clustering process. Therefore, there is an area where no bar is drawn. By such processing, the motion body motion time series data is segmented in time.

  Although this result may be output as it is, here, in order to obtain a more appropriate segmented result, the cluster result is corrected. Depending on the content of the motion body motion time series data, there is a section where the segmented result “Clustering Raw Result” changes rapidly. Such an interval is an interval in which the motion body motion time series data changes in an unstable manner.

  As shown in a part of the section of FIG. 8, there is a section in which the Clustering Raw Result changes drastically. A short segment of Clustering Raw Result has only one sample time, that is, a duration of about 1/60 second. Such a reply with an extremely short segment is caused by an excessive segment and is an inappropriate determination result. In order to cope with this, the motion segment time point determination processing unit 204 performs processing for removing a segment having a short duration. Specifically, here, it is assumed that an interval that lasts less than 6 sample times, that is, less than 0.1 seconds, is excluded. Therefore, the motion segment time point determination processing unit 204 outputs the start time of the segment having a duration of 6 sample times or more. That is, as shown in FIG. 8, it is 4, 17, 27, 35, 57, 64 in this case. The row of Clustering Fine Result in FIG. 7 illustrates this result.

  In this manner, the motion segment time point determination processing unit 204 finally uses 4, 17, 27, as the result data obtained by segmenting the motion body motion time series data 201, that is, as the motion segment time point data 205 here. 35, 57 and 64 are output. The output form is output as a text file.

  As described above, the motion body motion time series data 201 is segmented. However, the segmented motion body motion time series data can be transformed into a single data as a single data. . By performing the processing to be summarized, the size of the data becomes smaller than the size of the motion body motion time series data describing the original motion. That is, the motion body motion time series data is compressed. When the motion body motion time series data and the motion body motion time series data are created as an image of the body posture, a small amount of posture data having an effect of recalling the original motion is extracted.

  Next, a process for summarizing data that records the motion of the motion object, that is, motion motion time-series data will be described. In this processing, as the preprocessing, the above-described processing for segmenting the motion body motion time series data is used.

  FIG. 9 is a diagram showing an outline of the software configuration of the motion object movement data segmentation processing apparatus to which the summary processing unit of the modification according to the present invention is added as a functional block. As shown in FIG. 9, the software configuration includes a moving body motion segmentation processing unit 302 and a moving body motion summary processing unit 304 as main parts.

  The outline of the motion will be described. The motion body movement segmentation processing unit 302 to which the motion body motion time series data 301 is input corresponds to the input of motion body motion time series data obtained by measuring motion of the motion body in time series. The motion body motion time series data 303 with segment information in which information obtained by segmenting the data with respect to time is added to the data is output. The motion body motion time series data 303 with segment information is input to the motion body motion summary processing unit 304, and the motion body motion summary processing unit 304 includes at least one motion segment of the motion body motion time series data with segment information. Then, the motion body motion summary data 305, which is obtained by extracting the data content of at least one time point, is output.

  This will be specifically described. As shown in the flow of information processing in FIG. 9, the data described as the previous embodiment is used as the moving body motion time series data 301 here. The motion body motion segmentation processing unit 302 performs the segmentation processing described above, and uses motion information with time segment data 303 with segment information to which the segment information is added. In the segmentation processing, the signal processing as described above is performed, and motion body time-series data 303 with segment information is output as data obtained by adding motion segment time point data to motion body motion time-series data.

  Next, the motion body motion summary processing unit 304 will be described. The motion body motion summary processing unit 304 first performs motion according to motion segment time point data added thereto from the motion body motion time series data 303 with segment information. Isolate body movement time series data. For example, as described above, if the motion segment time point data is the result of 4, 17, 27, 35, 57, 64, in this case, the value of n ranges from 4 to 16, from 17 to 26. The motion body motion time-series data is divided into six sections, a section from 27 to 34, a section from 35 to 56, a section from 57 to 63, and a section from 64 to 72.

  A segment is a section in which motions with similar kinematic features continue. Therefore, by extracting the motion body motion time series data at one time point from the segment and presenting it to a human, data that can be expected to recall the kinematic characteristics of the segment is obtained. Specifically, the motion body motion time series data at one time point is data of the body posture at that time point.

  If you can select the appropriate time as the body motion time series data for one point in time and extract the body posture data as the motion body motion summary data, the body motion of each segment can be extracted like a picture-story show or a cartoon expression. Posture data represents the content of the motion of each segment, and is data that provides an effect that reminds humans of the original motion.

  Various methods can be used as a method for extracting such motion body motion summary data. For example, at the time center of the segment, when the kinetic energy of the moving body reaches the maximum value, or when the value of a specific data series of the moving body movement time series data reaches the maximum or minimum value, A method of extracting body motion time series data as motion body motion summary data can be used.

  In this embodiment, the motion body motion summary processing unit 304 outputs motion body motion time series data at the beginning of the segment as motion body motion summary data. Specifically, the motion body motion summary data includes the text of 4,17,27,35,57,64 as motion segment time point data, and the value of n in 4,17,27,35,57,64. It can be realized as text data listing the values of the configuration data of the motion body motion time series data. The size of the motion body motion summary data is smaller than the motion body time series data describing the original motion. That is, the motion body motion time series data is compressed.

  FIG. 10 illustrates a diagram created as a body posture image based on the content of the segment representative data. Referring to FIG. 10, it is possible to extract a small amount of posture data having an effect of clearly recalling the original motion from the motion body motion time series data. Such data, like a picture-story show or a cartoon expression, is data on the movement mode of each segment representing the content of the movement of each segment and reminiscent of the original movement.

  Since the present invention segments motion-series motion time-series data, each segmented data uses, for example, motion-body motion analysis in physical therapy, detection of a suspicious person by a surveillance camera, or user gestures. It can be used for automatically creating a digest video or still picture from a sports video. Further, the present invention can be applied not only to the moving body mechanism but also to movement data of crowds, moving body organs, and non-living objects.

  For example, as shown in FIG. 5, when time series data of each position is given for a crowd of people or a group of cars waiting for a signal at an intersection, a red signal can be obtained by using the present invention. And segmentation of the time reflecting the state transition of the green light. This is because humans and cars on the road with a red light are stopped, while people and cars move on a road with a green light. A strong correlation is observed in the position change of the car. For this reason, if the change of the sign and the absolute value of the correlation coefficient can be measured, it strongly reflects the signal presentation status, and the signal presentation status can be accurately estimated without directly detecting it.

It is a figure which shows the hardware constitutions of the system configuration | structure of the operating body exercise | movement data segmentation processing apparatus by this invention. It is a figure which shows the outline of the software structure of the moving body exercise | movement data segmentation processing apparatus by this invention as a functional block. It is a figure explaining the site | part of the observation object in the human body which is an action | operation body used as the analysis object. It is the figure which extracted and drawn the posture of the human body corresponding to the action body movement time series data used as processing object at equal time intervals. It is explanatory drawing which showed the measurement site | part of the road traffic object made into measurement object. It is a figure which illustrates roughly the skeleton structure of the human body which is a moving body of analysis object. It is the graph which showed the time transition of the absolute value of all the elements of the movement correlation index between parts. It is the graph which showed the result of the segmentation process of motion body movement time series data. It is a figure which shows the outline of a software structure of the moving body exercise | movement data segmentation processing apparatus which added the summary process part of the modification by this invention as a functional block. It is an illustration figure of the figure produced as an image of a body posture based on the content of segment representative data.

Explanation of symbols

100 ... input section,
101 ... Keyboard,
102 ... mouse,
103 ... arithmetic unit,
104 ... display part,
105 ... Database,
202... Movement body time series data input processing unit,
203 ... Inter-part motion correlation index calculation processing unit,
204 ... operation segment time point determination processing unit,
302 ... Motion body motion segmentation processing unit,
304 ... Motion body motion summary processing unit

Claims (6)

A motion body motion data segmentation processing device that temporally segments time-series data obtained by measuring motion of a motion body,
Motion body motion time-series data input that inputs time-series data of displacement of at least two parts of the motion body, joint angles of joints, time differential values of these physical quantities, and time integral data Means,
Between the parts of the index defined as an index representing the strength of correlation between the components according to the data at at least one time point included in the data series of at least two components of the motion body movement time series data An inter-part motion correlation index calculating means for calculating a motion correlation index;
In the motion body motion data segment processing device having motion segment time point determination means for determining a motion segment time point according to the value of the motion correlation index between parts at at least two time points ,
The inter-part motion correlation index calculated by the inter-part motion correlation index calculation means is a product-moment correlation coefficient, rank correlation coefficient, inner product, and weight of the data value of at least two time point data of motion body movement time series data. A motion body motion data segmentation processing apparatus that calculates a sum of weights or a weighted product of the data values and outputs the calculation result as a motion correlation index between parts . The motion body motion data segment processing device according to claim 1,
The motion body motion time series data input by the motion body motion time series data input means is a joint angle with respect to displacement of a human or animal limb, head, tail, terminal, or joint point, a time differential value of these physical quantities, A motion body motion data segmentation processing device characterized by being time-series data composed of any one of a higher order time differential value, a time integration value, and a higher order time integration value. The motion body motion data segment processing device according to claim 1, further comprising:
According to the input of the motion body time series data obtained by measuring the motion of the motion body in time series, the motion body motion time series data with segment information obtained by adding information obtained by segmenting the data in time to the data is output. A moving body motion segmentation means;
Motion body motion summarizing means for outputting motion body motion summary data obtained by extracting data contents of at least one time point in at least one motion segment of motion body motion time-series data with segment information. A motion body motion data segmentation processing device. A motion body motion data segmentation processing program for causing a computer to execute processing for temporally segmenting time-series data obtained by measuring motion of a motion body,
A first step of inputting motion body motion time series data constituted by time series data of displacement of at least two parts of the motion body, joint angles of joints, time differential values of these physical quantities, and time integral values;
Between the parts of the index defined as an index representing the strength of correlation between the components according to the data at at least one time point included in the data series of at least two components of the motion body movement time series data A second step of calculating a motion correlation index;
In a motion body motion data segment processing program for causing a computer to execute processing with the third step of determining motion segment time points according to values of motion correlation indices between parts at at least two time points ,
The inter-part motion correlation index calculated by the second step is a product-moment correlation coefficient, rank correlation coefficient, inner product, weighted sum of the data values of at least two time points of motion body motion time series data, or A motion body motion data segment processing program for calculating a weighted product of the data values and outputting the calculation result as a motion correlation index between parts . In the motion body motion data segment processing program according to claim 4,
The motion body motion time-series data input in the first step is a joint angle with respect to displacement of a human or animal limb, head, tail, terminal, or joint point, a time differential value of these physical quantities, a higher-order time differential value, A motion body motion data segment processing program characterized in that it is time series data composed of either time integral values or higher order time integral values. The motion body motion data segment processing program according to claim 4, further comprising:
According to the input of the motion body time series data obtained by measuring the motion of the motion body in time series, the motion body motion time series data with segment information obtained by adding information obtained by segmenting the data in time to the data is output. The fourth step;
The computer executes processing with the fifth step of outputting motion body motion summary data obtained by extracting data contents of at least one time point in at least one motion segment of motion body motion time series data with segment information. An action body motion data segment processing program characterized in that

Images