CN104156986A - Motion capture data key frame extracting method based on local linear imbedding - Google Patents

Abstract

The invention discloses a method for extracting key frames of motion capture data based on local linear embedding, which includes the following steps: S1: using a local linear embedding method to reduce the dimensionality of the motion capture data, and using smoothing filtering to remove noise to obtain a frame reflecting the original motion dimensional characteristic curve; S2: extract the local extremum points on the characteristic curve to obtain the initial key frame; S3: between the initial key frames, insert the corresponding frame number according to the amplitude difference of the characteristic curve and the set threshold, and obtain the final A collection of keyframes. The present invention uses the LLE algorithm to reduce the dimensionality of the motion capture data, and performs two key frame extractions according to the dimensionality-reduced one-dimensional characteristic curve, and focuses on solving a certain number of key motion postures that can be automatically extracted for a segment of motion sequence. Segment motion has a better visual generalization, while the reconstructed motion sequence can maintain a low error rate.

Description

Keyframe Extraction Method of Motion Capture Data Based on Local Linear Embedding

technical field

The invention belongs to the technical field of image processing, in particular to a method for extracting key frames based on locally linearly embedded motion capture data.

Background technique

In recent decades, with the rise and development of motion capture technology and the advancement of equipment technology, a large amount of 3D human motion capture data has been generated and widely used in computer animation, movie special effects, medical simulation and games. Accompanied by a problem, due to the large size of the captured data, the size of the motion capture database is also very large, how to make full use of these existing motion capture data, and how to obtain the motion required by the user from the motion library. Key frame technology is an effective solution. Select the most important and critical frame in the motion as the key frame to represent the entire motion sequence. It has a better visual generalization for this motion, and at the same time it can perform motion reconstruction and restoration. Raw movement, keeping a low error rate.

At present, the sampling methods are mainly divided into two categories: equal interval sampling and adaptive sampling. The problem of oversampling and undersampling may occur in equal interval sampling. Adaptive sampling can sample less in places with small changes and more samples in places with large changes, so it can solve the shortcomings of the former. The existing key frame extraction techniques of motion capture data are mainly divided into three categories: curve simplification-based, clustering and matrix decomposition-based techniques. In the method based on curve simplification technology, how to avoid the problems caused by the curse of dimensionality, Huaqiao University published in 2012 the key frame extraction of human motion sequences based on the center distance feature to extract the distance from the limbs to the center point to obtain a set of centers The scheme of distance feature, the distance feature extracted by this scheme can not represent the motion data very well. And Beijing University of Aeronautics and Astronautics published in 2011 the key frame extraction of human motion capture data based on hybrid genetic algorithm, etc., this program consumes a long time. For the feature curve after dimensionality reduction, the scheme of curve simplification is used to extract key frames. Such a scheme, such as the key frame extraction of motion capture data based on layered curve simplification published by Zhejiang University in 2006, and the "3D Human Motion Retrieval Based on Key" published in 2009 -Frames", etc. However, these methods still have a large error rate under the condition of ensuring the compression ratio.

Contents of the invention

According to the problems existing in the prior art, the present invention discloses a method for extracting key frames based on locally linearly embedded motion capture data, comprising the following steps:

S1: Use the local linear embedding method to reduce the dimensionality of the motion capture data, and use smoothing filtering to remove noise to obtain a one-dimensional characteristic curve reflecting the original motion;

S2: extract the local extreme point on the characteristic curve to obtain the initial key frame;

S3: Between the initial key frames, insert the corresponding frame number according to the magnitude difference of the characteristic curve and the set threshold, and obtain the final key frame set.

Further, the dimensionality reduction of motion capture data using the local linear embedding method in S1 specifically includes the following steps:

S11: Select K neighboring points in the high-dimensional space, for each sample point Xi ( _i =1,2,...,N) in the high-dimensional space, N is the total number of samples, and calculate the relationship between each sample point and others Euclidean distance between sample points;

S12: Calculate the local reconstruction weight matrix of each sample point from the neighboring points of each sample point;

S13: Calculate the output value of the sample point according to the local reconstruction weight matrix of the sample point and its neighbor points;

Further, S3 specifically includes the following steps:

S31: Calculate the amplitude difference value vary of the characteristic curve between adjacent initial key frames, where vary represents the amplitude difference value of two adjacent frames in the initial key frame set;

S32: Set a threshold φ ₁ , if vary<threshold φ ₁ , no keyframe is inserted; if vary>threshold φ ₁ , insert one or more frames between the corresponding initial keyframes;

S33: When the amplitude difference varies>threshold φ ₁ , set the corresponding initial key frames as f ₁ and f ₂ , first set f ₁ as the current frame, set f _new = f ₁ , f _new is a temporary variable, according to the frame number Extract the next frame and represent it as f _next , detect the amplitude difference between f _next and f _new according to the amplitude of the characteristic curve vary ₁ , set another threshold φ ₂ , if vary ₁ <φ ₂ , do not process; if vary ₁ >φ ₂ , f _next is then added to the set of key frames, so that f _next = f _new ;

S34: S32 and S33 are repeated until all frames are processed to obtain a final set of key frames.

Due to the adoption of the above technical solution, the key frame extraction method based on locally linearly embedded motion capture data provided by the present invention uses the LLE algorithm to reduce the dimensionality of the motion capture data, and performs two key frame extractions according to the dimensionality-reduced one-dimensional characteristic curve. Extraction focuses on solving a certain number of key motion postures automatically for a motion sequence, which has a better visual generalization of the motion, and at the same time, the reconstructed motion sequence can maintain a low error rate. The present invention has the following beneficial effects:

1. The LLE algorithm is used to reduce the dimensionality of the original motion capture data, which reveals the essential characteristics behind the movement and avoids the problems caused by the curse of dimensionality.

2. Using two key frame extraction methods, the first time is based on the local extreme point of the characteristic curve as the initial key frame, and the second time is based on the characteristic curve amplitude difference interpolation frame to obtain the final key frame, which can be set according to different motions Different thresholds automatically acquire a certain number of sampling frames, extract fewer key frames at gentle motions, and extract more key frames at severe motions. The extracted keyframes can not only well summarize the original motion, but also have a lower error rate and compression ratio.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments described in this application. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

Fig. 1 is the flowchart of method among the present invention;

Fig. 2 is the human skeleton model diagram shown in MTALAB;

Fig. 3 is the characteristic curve and initial key frame distribution figure of " kicking a ball " motion; (circle among the figure represents initial key frame);

Fig. 4 is the characteristic curve and final key frame distribution figure of " kicking a ball " motion; (circle among the figure represents initial key frame, and asterisk represents final key frame);

Fig. 5 " kicking a ball " the different key frame extraction method comparative result display figure of motion, wherein: Fig. 5 (a) is the result display figure of the method of the present invention; Fig. 5 (b) is the result display figure of uniform sampling method, (square Boxes represent oversampling and undersampling); Figure 5(c) is a graph showing the results of the curve simplification method, (boxes represent oversampling and undersampling); Figure 5(d) is a graph showing the results of the quaternion distance method;

Fig. 6 is the comparison chart of the compression rate of six different motion types (kicking, jumping, running-stop, walking, dancing, walking-jumping-walking) using the method of the present invention;

Fig. 7 is the reconstruction error comparison figure of six kinds of sampling motions of method disclosed by the present invention, quaternion distance method, curve simplification and uniform sampling method; (a) kicking error (extracting 33 key frames); (b) jumping (extract 24 keyframes); (c) run-stop error (extract 11 keyframes); (d) walk error (extract 16 keyframes); (e) dance error (extract 37 keyframes); keyframes); (f) walk-jump-walk error (50 keyframes extracted).

Detailed ways

In order to make the technical solutions and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention are clearly and completely described below in conjunction with the drawings in the embodiments of the present invention:

As shown in Figure 1 and Figure 2, the key frame extraction method based on locally linearly embedded motion capture data: specifically includes the following steps:

S1: Use the local linear embedding method to reduce the dimensionality of the motion capture data, and use smoothing filtering to remove noise to obtain a one-dimensional characteristic curve reflecting the original motion;

S2: extract the local extreme point on the characteristic curve to obtain the initial key frame;

S3: Between the initial key frames, insert the corresponding frame number according to the magnitude difference of the characteristic curve and the set threshold, and obtain the final key frame set.

Further, the dimensionality reduction of motion capture data using the local linear embedding method in S1 specifically includes the following steps:

S11: Select K neighboring points in the high-dimensional space, for each sample point Xi ( _i =1,2,...,N) in the high-dimensional space, N is the total number of samples, and calculate the relationship between each sample point and others Euclidean distance between sample points;

S12: Calculate the local reconstruction weight matrix of each sample point from the neighboring points of each sample point;

S13: Calculate the output value of the sample point according to the local reconstruction weight matrix of the sample point and its neighbor points;

Further, S3 specifically includes the following steps:

S31: Calculate the amplitude difference value vary of the characteristic curve between adjacent initial key frames, where vary represents the amplitude difference value of two adjacent frames in the initial key frame set;

S32: Set a threshold φ ₁ , if vary<threshold φ ₁ , no keyframe is inserted; if vary>threshold φ ₁ , insert one or more frames between the corresponding initial keyframes;

S33: When the amplitude difference varies>threshold φ ₁ , set the corresponding initial key frames as f ₁ and f ₂ , first set f ₁ as the current frame, set f _new = f ₁ , f _new is a temporary variable, according to the frame number Extract the next frame and represent it as f _next , detect the amplitude difference between f _next and f _new according to the amplitude of the characteristic curve vary ₁ , set another threshold φ ₂ , if vary ₁ <φ ₂ , do not process; if vary ₁ >φ ₂ , f _next is then added to the set of key frames, so that f _next = f _new ;

S34: S32 and S33 are repeated until all frames are processed to obtain a final set of key frames.

Example:

Step 1: Select a representative movement from the CMU database, the 'kicking football' movement (the total number of frames is 801 frames). Using LLE to reduce the dimensionality of the 'kicking ball' movement, using Lowess smoothing filter, setting the corresponding parameters to denoise the characteristic curve, obtaining a one-dimensional characteristic curve and then extracting key frames.

Step 2: In MATLAB, find the local extremum points on the curve according to the curvature change of the characteristic curve, and obtain the initial key frame set. As shown in Figure 3.

Step 3: The final key frame extracted by the splitting algorithm based on the magnitude of the characteristic curve is shown in Fig. 4 . The local extremum points we extracted are represented by circles as initial keyframes, while asterisks represent final keyframes. Through the research, it is found that when the motion posture changes greatly, the corresponding characteristic curve amplitude changes greatly, and additional key frames need to be inserted. When the motion posture changes small, the corresponding characteristic curve amplitude changes small, and no additional key frames need to be inserted. Through this method we can effectively obtain the final set of keyframes.

Step 4: Comparison of different key frame extraction algorithms. We have adopted four kinds of methods: the method of the present invention adopts evenly, and curve simplifies, only has the method for quaternion distance, extracts the same number of key frames (compression ratio is identical) from a ' kicking a ball ' motion. The comparison results of key frame extraction by different methods are shown in Fig. 5. We extracted 33 key frames, and the method of the present invention generalizes the motion well, avoiding the problem of over-sampling and under-sampling.

Step five: using the method of the present invention to test six different types of motion sequences, including kicking, jumping, running, walking, dancing, and walking-jumping-walking. As shown in Figure 6, it is found from Table 1 that the compression ratio obtained by the method of the present invention is within 8%.

Table 1. Comparison of compression ratios for the six motion types

Then, we calculate the absolute mean error with the following formula:

E＝[∑(F(i)-F'(i)) ² ]/N

Here, i=1,2,...,n represents the i-th frame, n is the total number of motion frames, F(n) is the original motion data, F'(n) is the corresponding reconstructed motion data, and N is the motion The total number of frames is multiplied by 96 degrees of freedom.

As can be seen from Table 2, we use six sampled motions to calculate the absolute average error after reconstruction, where the data in brackets represent the number of key frames extracted. Figure 7, shows the error comparison of the six sampled motions.

Table 2. Comparison of error rates of the four methods

We use four methods to obtain the same number of key frames, and then we reconstruct the motion sequence by linear interpolation to obtain the reconstructed error. We can find that the error rate obtained by using the method of the present invention is lower than other methods under the same compression ratio. And it can be found that the method in this paper has a good effect on both regular and irregular sports.

The above is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto, any person familiar with the technical field within the technical scope disclosed in the present invention, according to the technical solution of the present invention Any equivalent replacement or change of the inventive concepts thereof shall fall within the protection scope of the present invention.

Claims (3)

1. A key frame extraction method based on motion capture data of local linear embedding, characterized in that comprising the following steps: S1: Use the local linear embedding method to reduce the dimensionality of the motion capture data, and use smoothing filtering to remove noise to obtain a one-dimensional characteristic curve reflecting the original motion; S2: extract the local extreme point on the characteristic curve to obtain the initial key frame; S3: Between the initial key frames, insert the corresponding frame number according to the magnitude difference of the characteristic curve and the set threshold, and obtain the final key frame set. 2. the key frame extracting method based on the motion capture data of local linear embedding according to claim 1, it is also characterized in that: when adopting local linear embedding method to carry out dimensionality reduction to motion capture data in S1, specifically comprise the following steps: S11: Select K neighboring points in the high-dimensional space, for each sample point Xi ( _i =1,2,...,N) in the high-dimensional space, N is the total number of samples, and calculate the relationship between each sample point and others Euclidean distance between sample points; S12: Calculate the local reconstruction weight matrix of each sample point from the neighboring points of each sample point; S13: Calculate the output value of the sample point according to the local reconstruction weight matrix of the sample point and its neighbor points. 3. the key frame extracting method based on the motion capture data of local linear embedding according to claim 1, it is also characterized in that: S3 specifically comprises the following steps: S31: Calculate the amplitude difference value vary of the characteristic curve between adjacent initial key frames, where vary represents the amplitude difference value of two adjacent frames in the initial key frame set; S32: Set a threshold φ ₁ , if vary<threshold φ ₁ , no keyframe is inserted; if vary>threshold φ ₁ , insert one or more frames between the corresponding initial keyframes; S33: When the amplitude difference varies>threshold φ ₁ , set the corresponding initial key frames as f ₁ and f ₂ , first set f ₁ as the current frame, set f _new = f ₁ , f _new is a temporary variable, according to the frame number Extract the next frame and represent it as f _next , detect the amplitude difference between f _next and f _new according to the amplitude of the characteristic curve vary ₁ , set another threshold φ ₂ , if vary ₁ <φ ₂ , do not process; if vary ₁ >φ ₂ , f _next is then added to the set of key frames, so that f _next = f _new ; S34: S32 and S33 are repeated until all frames are processed to obtain a final set of key frames.