JP2013257656A - Motion similarity calculation device, motion similarity calculation method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve robustness when calculating motion similarity between motion data pieces.SOLUTION: A motion similarity calculation device comprises: a spatial feature amount calculation unit 12 that calculates a spatial feature amount representing spatial distribution of motions in a principal component space from a motion data piece; a temporal feature amount calculation unit 13 that calculates a temporal feature amount representing temporal distribution of motions in the principal component space from the motion data piece; and a similarity calculation unit 14 that calculates motion similarity using a distance between spatial feature amounts and a distance between temporal feature amounts calculated for two motion data pieces, respectively.

Description

  The present invention relates to a motion similarity calculation device, a motion similarity calculation method, and a computer program.

  In recent years, as a method of acquiring motion capture data, a method of acquiring motion capture data from an image of a person with a marker attached to a joint part or the like, or a depth sensor system combining an RGB camera and a depth sensor A method of acquiring motion capture data by measuring a distance from a depth sensor system to a person is known. In the former method, for example, human skeleton type motion data as illustrated in FIG. 6 is defined as the motion capture data. As the latter method, for example, a so-called “Kinect (registered trademark) sensor” is known.

  Here, the human skeleton type motion data illustrated in FIG. 6 will be described as data representing the motion of the posture (pose) of the human body. FIG. 6 is a schematic diagram of a definition example of human skeleton type motion data. Human body skeleton-type motion data is based on the human skeleton, using bone and bone connection points (joints), with one joint as the root (root), and the structure of bones sequentially connected from the root via the joint (tree) Tree) structure. FIG. 6 shows only a part of the definition of the human body skeleton type motion data. In FIG. 6, a joint 100 is a waist part and is defined as a root. Joint 101 is the elbow portion of the left arm, Joint 102 is the wrist portion of the left arm, Joint 103 is the elbow portion of the right arm, Joint 104 is the wrist portion of the right arm, Joint 105 is the knee portion of the left foot, and Joint 106 is the left foot portion. Ankle part, joint 107 is right leg knee part, joint 108 is right leg ankle part, joint 109 is clavicle part, joints 110 and 111 are shoulder parts, joint 112 is neck part, joints 113 and 114 are The hip joint part.

  The skeleton type motion data is data that records the movement of each joint of the skeleton type object, and a human body, an animal, a robot, or the like is applicable as the skeleton type object. As the skeleton type motion data, position information, angle information, speed information, acceleration information, and the like of each joint can be used. Here, human body skeleton angle data and acceleration information will be described as an example of human body skeleton type motion data.

  Human body skeleton-type angle information data represents a series of movements of a person by a series of multiple poses. Basic pose data that represents a person's basic pose and each pose in the actual movement of the person. Frame data for each pose. The basic pose data includes information such as the position of the root and the position of each joint in the basic pose, and the length of each bone. The basic pose is specified by the basic pose data. The frame data represents the amount of movement from the basic pose for each joint. Here, angle information is used as the movement amount. Each frame data identifies each pose in which each movement amount is added to the basic pose. Thereby, a series of movements of a person is specified by the continuation of each pose specified by each frame data. The human skeleton-type angle information data can be created by a motion capture process from an image obtained by photographing a person's movement with a camera, or can be created manually by key frame animation.

  The human body skeleton type acceleration information data represents the acceleration of each joint of a person by continuous frame data for each pose and a plurality of poses. The human skeleton-type acceleration information data can be recorded by an accelerometer, or calculated from video and motion data.

  Conventionally, the conventional technique for calculating the degree of similarity between two movements generally includes the following two steps 1 and 2.

Step 1: The distance between each frame of two motion capture data is calculated. For example, Non-Patent Document 1 describes the following two methods for calculating the distance between frames.
(Method 1 for calculating the distance between frames)
In the human skeleton type motion data, the angle distance is defined for each joint, and each joint is weighted averaged.
(Method 2 for calculating the distance between frames)
The human skeleton type motion data is changed to the latent space, and the Euclidean distance is calculated between the coordinates of the latent space.
In the prior art described in Non-Patent Document 2 and Patent Document 1, feature quantities of movements of the joints are calculated, distances between the feature quantities are defined, and each joint is weighted averaged.

  Step 2: Consider all frames when calculating the similarity between two movements. For example, as described in Non-Patent Document 3, the most common method is DTW (Dynamic Time Warping).

JP 2010-033163 A

B.J.H. van Basten, A. Egges, “Evaluating distance metrics for animation blending,” ICFDG 2009, pp. 199-206. Meinard Muller, Tido Roder, and Michael Clausen, “Efficient content-based retrieval of motion capture data,” ACM SIGGRAPH 2005, pp. 677-685. Armin Bruderlin and Lance Williams, “Motion signal preprocess,” ACM SIGGRAPH 1995, pp. 97-104. Yossi Rubner; Carlo Tomasi, Leonidas J. Guibas (1998). "A Metric for Distributions with Applications to Image Databases". Proceedings ICCV 1998: 59-66

However, the conventional motion similarity calculation technique described above has the following problems.
(1) It is difficult to calculate motion similarity in common for motion capture data acquired by different motion capture systems. For example, if the definition of the bone structure of the human skeleton type motion data is different, it cannot be applied by the conventional technique.
(2) If the shooting environment for acquiring motion capture data (such as a person who is a subject, shooting distance, shooting angle, etc.) is different, it is difficult to calculate the similarity of motion in common.
(3) When the types of input data (for example, human skeleton type motion data, motion capture data acquired by a depth sensor system, RGB image data, distance image data, etc.) are different, the similarity of motion is commonly calculated. Is difficult.
(4) If data acquired by a motion capture system or the like includes noise, it is difficult to calculate a robust similarity.

  The present invention has been made in view of such circumstances, and it is an object of the present invention to provide a motion similarity calculation device, a motion similarity calculation method, and a computer program with improved robustness.

  In order to solve the above problems, a motion similarity calculation device according to the present invention includes a spatial feature amount calculation unit that calculates a spatial feature amount representing a spatial distribution of motion in a principal component space from motion data, and , A temporal feature amount calculation unit that calculates a temporal feature amount representing temporal distribution of motion in the principal component space from the motion data, and a distance between the spatial feature amounts calculated for each of the two motion data And a similarity calculation unit that calculates the similarity of motion using the distance between the temporal feature amounts.

  In the motion similarity calculation apparatus according to the present invention, the spatial feature amount calculation unit calculates a histogram using a distance between frames with respect to a main component of motion of each frame in motion data. .

  In the motion similarity calculation apparatus according to the present invention, the temporal feature amount calculation unit calculates a distance between the reference frame in the motion data and each of the other frames, and calculates an extreme value from the calculated inter-frame distance. The inter-frame distance value is interpolated or thinned out so that the number of frames between adjacent extreme values is constant.

  In the motion similarity calculation apparatus according to the present invention, the temporal feature amount calculation unit calculates a distance between the reference frame in the motion data and each of the other frames, and calculates an extreme value from the calculated inter-frame distance. In addition, when the interval between extreme values is extremely short, the extreme value is deleted, and when the interval between extreme values is extremely long, an appropriate extreme value is inserted.

  In the motion similarity calculation apparatus according to the present invention, the similarity calculation unit calculates a weighted average value for a distance between a spatial feature and a distance between temporal features. .

  The motion similarity calculation method according to the present invention includes a step of calculating a spatial feature amount representing a spatial distribution of motion in the principal component space from the motion data, and a temporal motion of the motion in the principal component space from the motion data. Calculate the similarity of motion using the step of calculating the temporal feature value representing the distribution and the distance between the spatial feature values calculated for each of the two motion data and the distance between the temporal feature values. And a step of performing.

  The computer program according to the present invention calculates a spatial feature amount representing a spatial distribution of motion in the principal component space from the motion data, and represents a temporal distribution of motion in the principal component space from the motion data. Calculating a temporal feature, and calculating a motion similarity using a distance between the spatial feature calculated for each of the two motion data and a distance between the temporal features Is a computer program for causing a computer to execute.

  According to the present invention, it is possible to improve the robustness when calculating the similarity of motion between motion data.

It is a block diagram which shows the structure of the motion similarity calculation apparatus 1 which concerns on one Embodiment of this invention. It is a block diagram which shows the structure of the spatial feature-value calculation part 12 shown in FIG. It is an example of a histogram. It is a graph for demonstrating the temporal feature-value calculation process which concerns on one Embodiment of this invention. It is the schematic which shows the procedure of the principal component analysis method which concerns on one Embodiment of this invention. It is a definition example of human body skeleton type motion data.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a configuration of a motion similarity calculation apparatus 1 according to an embodiment of the present invention. In FIG. 1, the motion similarity calculation device 1 includes an input unit 11, a spatial feature value calculation unit 12, a temporal feature value calculation unit 13, a similarity calculation unit 14, and an output unit 15. The motion similarity calculation device 1 according to the present embodiment calculates the similarity of motion between motion data. The motion data is data representing the motion of a person. The motion data includes motion capture data, RGB image data, distance image data, and the like. In the following description, motion capture data is used as an example of motion data.

[Input section]
The input unit 11 inputs motion capture data as motion data. The types of motion capture data include, for example, human skeleton type motion data and motion capture data acquired by a depth sensor system.

[Spatial feature calculation unit]
FIG. 2 is a block diagram showing a configuration of the spatial feature quantity calculation unit 12 shown in FIG. In FIG. 2, the spatial feature quantity calculation unit 12 includes a physical quantity change unit 21, a principal component analysis unit 22, and a histogram calculation unit 23. The spatial feature amount calculation unit 12 calculates a spatial feature amount regarding the movement of the person represented by the motion capture data input from the input unit 11. The spatial feature amount represents a spatial distribution of motion. In the calculation of the spatial feature amount, a relatively large movement is emphasized, and a relatively fine movement is ignored. In addition, robustness is improved by principal component analysis and histogram calculation.

[Physical quantity change part]
The physical quantity changing unit 21 generates data to be input to the principal component analyzing unit 22 from the input motion capture data. Hereinafter, an embodiment in which data to be input to the principal component analysis unit 22 is generated using human body skeleton type motion data will be described.

  The physical quantity changing unit 21 calculates at which position each joint is moving with respect to the root during the motion period represented by the human skeleton-type motion data. Here, the joint relative position at time t is calculated using human skeleton type angle information data which is a kind of human skeleton type motion data. The joint relative position is a relative position of the joint with respect to the root.

First, the physical quantity changing unit 21 calculates the joint position using the basic pose data and the frame data in the human body skeleton type angle information data. The basic pose data includes information for specifying the basic pose, such as the position of the root and the position of each joint in the basic pose, and the length of each bone. The frame data has information on the amount of movement from the basic pose for each joint. Here, angle information is used as the movement amount. In this case, the position p ^k (t) of the k-th joint at time t is calculated by the equations (1) and (2). p ^k (t) is represented by three-dimensional coordinates. Note that time t is the time of the frame data. In the present embodiment, time t is simply handled as a “frame index”. Thereby, the time t takes values of 0, 1, 2,..., T−1. T is the number of frames included in the human skeleton-type angle information data.

However, the 0th (i = 0) joint is the root. R _axis ^{i-1, i} (t) is a coordinate rotation matrix between the i-th joint and its parent joint ("i-1" -th joint), and is included in the basic pose data. A local coordinate system is defined for each joint, and the coordinate rotation matrix represents the correspondence of the local coordinate system between joints in a parent-child relationship. R ⁱ (t) is a rotation matrix of the i-th joint in the local coordinate system of the i-th joint, and is angle information included in the frame data. T ⁱ (t) is a transition matrix between the i-th joint and its parent joint, and is included in the basic pose data. The transition matrix represents the bone length between the i-th joint and its parent joint.

Next, the physical quantity changing unit 21 calculates the relative position (joint relative position) p ′ ^k (t) of the k-th joint with respect to the root at time t by using Equation (3).

Here, p ^root (t) is the position (p ⁰ (t)) of the route (0th joint) at time t.

Thus, the frame “x (t)” at time t is expressed as “x (t) = {p ′ ¹ (t), p ′ ² (t),..., P ′ ^K (t)} ^T ”. expressed. K is the number of joints excluding the root. ^T represents a transposed matrix.

[Principal component analysis section]
The principal component analysis unit 22 performs principal component analysis processing on the joint relative position data generated by the physical quantity changing unit 21. Here, using the frame “x (t)” at time t, the joint relative position data “X” is expressed as “X = {x (t1), x (t2),..., X (tN)}”. Where N is the number of frames included in the joint relative position data, and X is a matrix of M rows and N columns (where M = 3 × K).

  In the principal component analysis processing, principal component analysis processing is performed on X, and X is converted into a principal component space.

Here, the principal component analysis method will be described.
First, the matrix D of N rows and M columns obtained by subtracting the average value from X is calculated by the equation (4).

  Next, singular value decomposition (Singular Value Decomposition) is performed on the matrix D of N rows and M columns according to the equation (5).

  However, U is a unitary matrix of N rows and N columns. Σ is a diagonal matrix having non-negative diagonal elements of N rows and M columns in descending order, and represents the variance of the coordinates of the principal component space. V is a unitary matrix of M rows and M columns, and is a coefficient (principal component) for the principal component.

  Next, the matrix D of N rows and M columns is converted into the principal component space by the equation (6). The matrix Y with M rows and N columns represents the coordinates of the principal component space.

  In the principal component analysis process, a matrix (principal component coordinate matrix) Y representing coordinates of the principal component space and a matrix of principal components (principal component coefficient matrix) V are stored in a memory.

  Note that the matrix X representing the coordinates of the original space and the principal component coordinate matrix Y can be converted into each other by the equations (6) and (7).

  Moreover, it can convert by (8) Formula by the upper r main components.

Here, V ^r is a matrix of M rows and r columns composed of upper r rows in the principal component coefficient matrix V. Y ^r is an r-row N-column matrix composed of the upper r columns in the principal component coordinate matrix Y. X ^~ is a matrix of reconstructed M rows and N columns.

  Note that it is also possible to perform principal component analysis processing on only some degrees of freedom of the original space. For example, the principal component analysis process is performed on the M ′ × N matrix X ′ generated only from the joint relative position data regarding the foot using the equations (4), (5), and (6). It is also possible to perform principal component analysis processing on only some frames in the joint relative position data. For example, the principal component analysis processing is performed on the M × N ′ matrix X ′ generated from only the first N ′ frames by the equations (4), (5), and (6).

[Histogram calculation unit]
The histogram calculation unit 23 calculates a histogram using the principal component coordinate matrix Y calculated by the principal component analysis unit 22. This histogram is a spatial feature amount according to the present embodiment. In the histogram calculation process, a histogram is calculated using a matrix Y ^r of r rows and N columns composed of the upper r columns in the principal component coordinate matrix Y. The value of r can be set arbitrarily.

The matrix Y ^r has N frames like the joint relative position data X, and is ^expressed as “Y ^r = {y ^r (t1), y ^r (t2),..., Y ^r (tN)}”. is. histogram calculation unit 23 that is, for all combinations of the frame of the matrix Y ^r, calculates the Euclidean distance by equation (9).

  Next, the histogram calculator 23 calculates data of a histogram (frequency distribution diagram) of a predetermined number of bins using the calculated Euclidean distance d (i, j). The histogram is a bar graph representing the frequency distribution. FIG. 3 is an example of a histogram. In the present embodiment, the horizontal axis of the histogram shows the Euclidean distance d (i, j), and the vertical axis shows the frequency of each Euclidean distance d (i, j). At this time, based on the maximum value and the minimum value of the Euclidean distance d (i, j), the widths of the bins represented on the horizontal axis are set at equal intervals. Thereby, for example, a histogram “H = {h (1), h (2),..., H (64)}” composed of 64 bins is calculated.

  The above is the description of the spatial feature quantity calculation unit 12.

[Temporal feature value calculation unit]
The temporal feature amount calculation unit 13 calculates a temporal feature amount regarding the movement of the person represented by the motion capture data input from the input unit 11. The temporal feature amount represents a temporal distribution of motion.

  First, the temporal feature amount calculator 13 calculates the distance between the reference frame in the motion capture data and each of the other frames. In the present embodiment, the first frame (first frame) in time is used as the reference frame. As a result, the distances “D = {d (t2), d (t3),..., D (tN)}” between the first frame and the other frames are calculated. As a calculation method, a method described in Non-Patent Document 1 can be used, or, as a principal component coordinate matrix Y calculated by the principal component analysis unit 22, the Euclidean distance is calculated by Equation (9). May be.

  Next, the temporal feature quantity calculation unit 13 obtains an extreme value from the calculated inter-frame distance D. At this time, as illustrated in FIG. 4, the first inter-frame distance 301 (the distance between the first frame and the second frame) and the last inter-frame distance 306 (the distance between the first frame and the Nth frame) are also obtained. Add as extreme value. Thereby, in the example of FIG. 4, six extreme values 301 to 306 are obtained. In addition, it is preferable to eliminate when there is a drastic change in the interval between extreme values. For this reason, when the interval between extreme values is extremely short, the extreme value is deleted. Conversely, if the interval between extreme values is extremely long, an appropriate extreme value is inserted.

  Next, the temporal feature quantity calculation unit 13 calculates the number of frames between adjacent extreme values (D = {d (t2), d (t3),. The inter-frame distance value is interpolated or thinned out so that the distance value number) is a fixed number (a value that can be arbitrarily set). f (1), f (2),..., f (N ′)} ”. According to this temporal feature amount F, there is an effect of not distinguishing the movements of the same shape by the speed of movement.

[Similarity calculation unit]
The similarity calculation unit 14 calculates the similarity between the motion of the person represented by one motion capture data and the motion of the person represented by the other motion capture data regarding the two motion capture data input by the input unit 11. . In the similarity calculation process, the spatial feature amount calculated by the spatial feature amount calculation unit 12 and the temporal feature amount calculated by the temporal feature amount calculation unit 13 are used for each motion capture data. Hereinafter, the similarity calculation method according to the present embodiment will be described.

  Here, with respect to the first motion capture data, a spatial feature amount (histogram “H1 = {h1 (1), h1 (2),..., H1 (64)})” and a temporal feature amount (interframe distance). “F1 = {f1 (1), f1 (2),..., F1 (N)}” For the second motion capture data, a spatial feature (histogram “H2 = {h2 (1), h2 (2),..., h2 (64)}) and temporal features (interframe distances “F2 = {f2 (1), f2 (2),..., f2 (M)}”). .

  First, the similarity calculation unit 14 calculates the distance between the histogram H1 and the histogram H2 using the spatial feature amounts (histograms H1 and H2) of the first and second motion capture data. In the calculation process of the distance between histograms, the distance between the histogram H1 and the histogram H2 can be calculated using the definition of distance called EMD (Earth Mover's Distance) described in Non-Patent Document 4, for example. The calculation result of the distance between histograms using this EMD is defined as EMD (H1, H2). The distance between histograms using EMD indicates that the smaller the value, the greater the similarity.

  Next, the similarity calculation unit 14 calculates the distance D1 (F1, F2) by the expression (10) using the temporal feature amounts (interframe distances F1, F2) of the first and second motion capture data. .

  Further, the similarity calculation unit 14 calculates the distance D2 (F1, F2) by the equation (11) using the temporal feature amounts (interframe distances F1, F2) of the first and second motion capture data. . This distance D2 (F1, F2) is K (arbitrarily settable) as a frame from which the distance calculation is started with respect to the longer one of the first and second motion capture data. It is calculated by shifting the value).

  The similarity calculation unit 14 selects a distance D1 (F1, F2) between temporal feature values of the distance D1 (F1, F2) and the distance D2 (F1, F2), whichever has a smaller value (a higher similarity). And

    Next, the similarity calculation unit 14 uses the distance between the spatial feature amounts (inter-histogram distance EMD (H1, H2)) and the distance D (F1, F2) between the temporal feature amounts according to the equation (12). The similarity is calculated.

  However, α is a weighting factor and is a value that can be arbitrarily set in the range from 0 to 1.

[Output section]
The output unit 15 outputs the similarity calculated by the similarity calculation unit 14. Thereby, the similarity of the motion regarding the two motion capture data input by the input unit 11 is output.

  In the above-described embodiment, an example in which principal component analysis processing is performed using motion capture data has been described. However, an example in which principal component analysis processing is performed using other types of motion data is described below with reference to FIG. Will be described with reference to FIG. FIG. 5 is a schematic diagram showing a procedure of a principal component analysis method according to an embodiment of the present invention.

(Step S1) The input unit 11 inputs RGB image data or distance image data as motion data. The RGB image data is moving image data captured by an RGB camera, and the pixel values in the frame are RGB pixel values. The distance image data is moving image data having depth information acquired by the depth sensor system, and the pixel value in the frame is a depth value.

(Step S2) The physical quantity changing unit 21 stores the motion data input by the input unit 11. This motion data is composed of N frames as shown in FIG. The number of pixels M which is the size M of the i-th frame is the product “M = height × width” of the number of vertical pixels “height” and the number of horizontal pixels “width”. However, “1 ≦ i ≦ N”. Thereby, it can be applied to motion data composed of images of an arbitrary size.

(Step S3) The physical quantity changing unit 21 generates a matrix B ′. In the column of the matrix B ′, pixel value vectors of N frames are arranged in the time-series order of the frames in the motion data. Therefore, the column vector of the i-th column of the matrix B ′ is the pixel value vector Vi of the i-th frame. In the pixel value vector Vi, the pixel values of M pixels constituting the i-th frame are arranged as vector elements in a predetermined order based on the positions of the pixels in the frame. Thereby, the matrix B ′ becomes a matrix of M rows and N columns.

(Step S4) The principal component analysis unit 22 calculates the eigenvalue λi (i = 1, 2,...) And the eigenvector vi (i = 1, 2,...) Of the variance-covariance matrix S of the matrix B ′. To do. The size of the variance-covariance matrix S is M rows and M columns.

(Step S5) The principal component analysis unit 22 uses the eigenvalues λi and eigenvectors vi of the variance-covariance matrix S of the matrix B ′ to convert the eigenvalues and eigenvectors into the principal component space values (principal component scores) for each eigenvalue. Generate converted time series data (time series of principal component scores). A time series of one principal component score is composed of N pieces of data that are the same as the number of frames in the motion data.

  The histogram calculation unit 23 uses the time series of the r highest principal component scores among the time series (i = 1, 2,...) Of the i-th principal component score created by the principal component analysis unit 22. Calculate a histogram.

  Next, an example of step S4 in FIG. 5 will be described. In general, the size (number of pixels) M of a moving image is about vertical × horizontal = (height × width = M) = 320 × 240 = 80,000 pixels, for example, when it is assumed that the camera attached to the mobile phone is simply captured. It can be considered that the number of frames (number of time series) is about N = 300. That is, it is assumed that “M> N”. In this case, from the viewpoint of reducing the amount of computation, instead of directly solving the eigenvalue problem of the M-column and M-column variance-covariance matrix S, the eigenvalue problem of the N-row N-column matrix C is solved as described later. From the result, it is preferable to calculate the eigenvalues and eigenvectors of the variance-covariance matrix S by a predetermined number from the larger eigenvalue. This embodiment will be described below.

  The matrix B ′ of M rows and N columns obtained in step S3 is expressed by equation (13).

  The row average (average with respect to the frame direction) of this matrix B ′ is expressed by equation (14) as a matrix mean of M rows and N columns.

  Then, the matrix B extracted from the matrix B ′ row by row, subtracting the average vector of the corresponding row of the matrix mean, and defining the matrix B coupled in the row direction is defined by equation (15).

  When this matrix B is used to express the variance-covariance matrix S, equation (16) is obtained.

  And the eigen equation becomes the equation (17).

  When this characteristic equation (17) is multiplied by the matrix B from the left on both sides and “Bv = u” is obtained, equation (18) is obtained.

  Equation (18) is an eigen equation of the matrix (N ^ -1) BB ^ T (denoted as matrix C). After obtaining the eigenvector u for the matrix C, the eigenvalue v of the variance-covariance matrix S is obtained using the eigenvector u. Considering the size of the matrix at this time, it is N rows and N columns, and generally the number of frames N is overwhelmingly smaller than the number of dimensions M of the pixel value vector. Therefore, it can be calculated with a much smaller amount of computation than when the eigenvalue is obtained.

  In order to obtain the eigenvector v of the variance-covariance matrix S from the eigenvector u, the matrix B ^ T is multiplied from the left in the equation (18). Thereby, the equation (19) is obtained.

  From this equation (19), it can be seen that B ^ Tu is the eigenvector v of the variance-covariance matrix S. However, since normalization is not performed, the eigenvector v of the variance-covariance matrix S is expressed by the equation (20).

  In this way, the eigenvector u of the matrix C of N rows and N columns can be obtained, and the eigenvector v of the variance-covariance matrix S of M rows and M columns can be obtained using this eigenvector u. However, of the M eigenvalues of the variance-covariance matrix S, the Nth and subsequent eigenvalues from the larger eigenvalue are considered to be zero. According to this method, the principal component analysis process can be actually performed even for a large-capacity moving image as generally assumed.

  Note that M may be reduced by reducing the resolution of the image, or N may be reduced by reducing the frame rate, or both. In addition, when the size of the moving image is “M <N”, the eigenvalue problem of the variance-covariance matrix S of M rows and M columns may be directly solved.

  According to the above-described embodiment, spatial feature amounts and temporal feature amounts are calculated for the two motion data, respectively, and the motion between the spatial feature amounts and the distance between the temporal feature amounts is used. The similarity is calculated. Thereby, the effect that the similarity of motion can be calculated in common for motion capture data acquired by different motion capture systems can be obtained. In addition, even when the shooting environment for acquiring the motion capture data (person who is the subject, shooting distance, shooting angle, etc.) is different, the similarity of motion can be calculated in common. can get. Even if the types of motion data to be input (for example, human skeleton motion data, motion capture data acquired by a depth sensor system, RGB image data, distance image data, etc.) are different, similar motions are commonly shared The effect that the degree can be calculated is obtained. As described above, according to the present embodiment, it is possible to improve the robustness when calculating the similarity of motion between motion data.

  Further, according to the present embodiment, the following effects can be obtained. Conventionally, a technique for moving a computer graphics (CG) character in accordance with a user's movement is known. Then, by adding the individual movement of the user to the database on which the movement of the CG character is based, it can be expected to add an effect that makes the movement of the CG character feel familiar. At this time, it is conceivable to recognize the movement of the user using the similarity between the movement of the user and the motion capture data in the database, and add the user's unique movement to the database. Here, according to the present embodiment, since the user's motion may be acquired by an arbitrary motion capture system, an effect that the user can use it very much is obtained.

  As mentioned above, although embodiment of this invention was explained in full detail with reference to drawings, the specific structure is not restricted to this embodiment, The design change etc. of the range which does not deviate from the summary of this invention are included.

Also, a program for realizing the function of the motion similarity calculation device 1 shown in FIG. 1 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed. Thus, the motion similarity calculation process may be performed. Here, the “computer system” may include an OS and hardware such as peripheral devices.
“Computer-readable recording medium” refers to a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a DVD (Digital Versatile Disk), and a built-in computer system. A storage device such as a hard disk.

Further, the “computer-readable recording medium” means a volatile memory (for example, DRAM (Dynamic DRAM) in a computer system that becomes a server or a client when a program is transmitted through a network such as the Internet or a communication line such as a telephone line. Random Access Memory)), etc., which hold programs for a certain period of time.
The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 1 ... Motion similarity calculation apparatus, 11 ... Input part, 12 ... Spatial feature-value calculation part, 13 ... Temporal feature-value calculation part, 14 ... Similarity calculation part, 15 ... Output part, 21 ... Physical-quantity change part, 22 ... Principal component analysis unit, 23 ... Histogram calculation unit

Claims (7)

A spatial feature amount calculating unit that calculates a spatial feature amount representing a spatial distribution of motion in the principal component space from the motion data;
A temporal feature amount calculating unit that calculates a temporal feature amount representing a temporal distribution of motion in the principal component space from the motion data;
A similarity calculation unit that calculates the similarity of motion using the distance between the spatial feature amount calculated for each of the two motion data and the distance between the temporal feature amounts;
A motion similarity calculation device comprising: The spatial feature amount calculation unit calculates a histogram using a distance between frames with respect to a main component of motion of each frame in motion data.
The motion similarity calculation apparatus according to claim 1, wherein: The temporal feature amount calculation unit calculates a distance between a reference frame in the motion data and each of the other frames, obtains an extreme value from the calculated inter-frame distance, and determines the number of frames between adjacent extreme values. Interpolate or decimate inter-frame distance values to be a constant,
The motion similarity calculation apparatus according to claim 1, wherein The temporal feature amount calculation unit calculates a distance between a reference frame in motion data and each of the other frames, obtains an extreme value from the calculated inter-frame distance, and an extreme value interval is extremely short Deletes the extreme value and inserts an appropriate extreme value if the interval between extreme values is extremely long.
The motion similarity calculation apparatus according to any one of claims 1 to 3, wherein The similarity calculation unit calculates a weighted average value for the distance between the spatial feature and the distance between the temporal features.
The motion similarity calculation apparatus according to claim 1, wherein: Calculating a spatial feature amount representing a spatial distribution of motion in the principal component space from the motion data;
Calculating a temporal feature amount representing a temporal distribution of motion in the principal component space from the motion data;
Calculating a similarity of motion using a distance between the spatial feature amount calculated for each of the two motion data and a distance between the temporal feature amounts;
A motion similarity calculation method comprising: Calculating a spatial feature amount representing a spatial distribution of motion in the principal component space from the motion data;
Calculating a temporal feature amount representing a temporal distribution of motion in the principal component space from the motion data;
Calculating a similarity of motion using a distance between the spatial feature amount calculated for each of the two motion data and a distance between the temporal feature amounts;
A computer program for causing a computer to execute.

Images