CN104331904A - Three-dimensional human motion key frame extracting method based on fuse of improved LLE and PCA - Google Patents

Abstract

The invention discloses a motion key frame extracting method of fusing the improved locally linear embedding (LLE for short) and principal component analysis (PCA for short). The extracting method comprises the steps as follows: 1, using the improved LLE for pre-processing the original motion capturing data and reducing dimension to obtain the data with fixed dimension; 2, using PCA method for processing the data with reduced dimension to extract the one-dimensional main component, using the flat filtering method for removing the noise to obtain the one-dimensional characteristic curve; 3, extracting the extreme point of the characteristic curve to obtain the initial key frame;, 4, inserting the corresponding frame number between the initial key frames according to the characteristic curve amplitude difference and set threshold value, merging the overstocked key frame to obtain the final key frame set.

Description

Based on the 3 d human motion extraction method of key frame merging LLE and PCA improved

Technical field

The present invention relates to the key-frame extraction relating to human motion, specifically relating to a kind of 3 d human motion extraction method of key frame based on merging LLE and PCA improved.

Background technology

The appearance of key-frame extraction technique makes motion capture database scale can become less, key-frame extraction is exactly select the frame of most important most critical in motion as key frame for one section of motion sequence, represent whole motion sequence, a good vision generality is had to this section of motion, restructure from motion can be carried out again simultaneously, reduction original motion, keeps the error rate that lower.

Current slave sampling side formula is mainly divided into two large classes: equal interval sampling and adaptively sampled.Likely there is the problem of over-sampling and lack sampling in equal interval sampling, adaptively sampled can change little place sample less and change large place sample, so the former deficiency can be solved more.

The factor affecting classical LLE Algorithm robustness mainly contains distribution (2) Neighbor Points number K (3) the reconstruction weight matrix W of three aspect (1) noise sources.For the problem of classical LLE algorithm, " the L that the Southern Yangtze University as domestic delivers for 2011 ₁model local linear embeds ", " the LLE innovatory algorithm based on sparse constraint " that University Of Ningbo delivers for 2013 etc., and " Robust locally linear embedding using penalty functions " that external University of Waterloo (CA) Waterloo, Ontario, N2L3GI Canada delivers for 2011 is although improve the robustness of LLE algorithm all to a certain extent, stable result can not be obtained and do not reduce choosing of Neighbor Points number." local linear of Neighbourhood parameter dynamic change embeds " that and for example domestic South China Science & Engineering University delivers for 2008, although well control the selection of Neighbor Points number, result often has certain randomness.

Summary of the invention

Technical matters to be solved by this invention is the stability of classical LLE algorithm and reduces Neighbor Points number and rebuild the structure of weight matrix to the impact of algorithm.

In order to solve the problem, the technical solution adopted in the present invention is the 3 d human motion extraction method of key frame based on merging LLE and PCA improved, its Local Liner Prediction LLE adopting fusion to improve and Principal Component Analysis Algorithm PCA carries out dimensionality reduction to movement capturing data, and select Lowess smothing filtering, carry out key-frame extraction according to the one-dimensional characteristic curve after dimensionality reduction; Its detailed process is as follows:

The LLE that S1, employing improve carries out pre-service dimensionality reduction to movement capturing data;

The LLE method improved mainly is divided into following three steps:

Step one: select K Neighbor Points.For each frame X in higher dimensional space _i(i=1,2 ..., N), N is the totalframes of motion, calculates the distance d between it and other every frame _ij, range formula is:

$d_{\mathrm{ij}}=\left|\right|X_i-X_j\left|\right|/\sqrt{T\left(i\right)T\left(j\right)}$

Wherein, || X _i-X _j|| represent X _iand X _j(j=1,2 ..., N) between Euclidean distance, X _iand X _jrepresent the frame that in motion sequence two are different, T (i) and T (j) represents X respectively _ito mean value and the X of the distance between its K neighbour _jto the mean value of the distance between its K neighbour;

Step 2: the partial reconstruction weight matrix W being calculated this sample point by the Neighbor Points of each sample point;

Step 3: the output valve being calculated this sample point by partial reconstruction weight matrix and its Neighbor Points of this sample point.

S2, to data, reprocessing is carried out for the pretreated linear dimension reduction method of data acquisition PCA, obtain one-dimensional characteristic curve, and elimination noise;

S3, based on MATLAB platform, according to the change detection Local Extremum of characteristic curve, by extract Local Extremum obtain initial key frame;

S4, between initial key frame, the threshold value according to its characteristic curve Magnitude Difference and setting inserts corresponding frame number, merging overstocked key frame, obtains final key frame set.

The present invention compared with prior art has the following advantages:

1, the LLE algorithm and the PCA algorithm that have employed fusion improvement carry out dimensionality reduction to original motion capture data, well disclose motion essential characteristic behind, and avoid the problem that dimension disaster brings, namely reduce choosing of Neighbor Points number and also make the stability of algorithm be protected.

2, the method for twice key-frame extraction is adopted, the Local Extremum of foundation characteristic curve is as initial key frame for the first time, second time obtains final key frame according to characteristic curve Magnitude Difference interleave, the sampling frame number that different threshold values obtains some automatically can be set according to different motions, extract less key frame in mild motion place, extract more crucial frame number in violent motion place.The key frame extracted well can summarize original motion, has again a lower error rate and ratio of compression simultaneously.

Accompanying drawing explanation

Below by way of drawings and the specific embodiments, the present invention is described in detail.

Fig. 1 process flow diagram of the present invention;

Fig. 2 ' plays football ' motion characteristics curve and final key frame distribution plan, circle represents initial key frame here, and asterisk is expressed as final key frame;

Fig. 3 ' walks ' motion characteristics curve and final key frame distribution plan, circle represents initial key frame here, and asterisk is expressed as final key frame;

Fig. 4 context of methods extracts the key frame that ' jumping ' moves;

The classical LLE of Fig. 5 extracts the key frame that ' jumping ' moves;

Fig. 6 curve simplifies the key frame extracting ' jumping ' and move;

Fig. 7 ' plays football ' contrast of the error rate that moves; Wherein (left side) is context of methods, (in) be classical LLE, (right side) is curve short-cut method;

Fig. 8 ' jumps ' contrast of the error rate that moves; Wherein (left side) is context of methods, (in) be classical LLE, (right side) is curve short-cut method;

Fig. 9 ' runs ' contrast of the error rate that moves; Wherein (left side) is context of methods, (in) be classical LLE, (right side) is curve short-cut method;

Figure 10 ' walks ' contrast of the error rate that moves; Wherein (left side) is context of methods, (in) be classical LLE, (right side) is curve short-cut method;

Figure 11 ' walks-jumps-walk ' the error rate contrast of moving; Wherein (left side) is context of methods, (in) be classical LLE, (right side) is curve short-cut method.

Embodiment

In order to understand this further based on the 3 d human motion extraction method of key frame merging LLE and PCA improved, be described as follows below in conjunction with accompanying drawing.

It adopts the LLE method of improvement to carry out pre-service dimensionality reduction to movement capturing data, then adopt PCA to carry out last dimension-reduction treatment and adopt smothing filtering to remove noise, obtain one-dimensional characteristic curve, initial key frame is obtained by the Local Extremum extracted on characteristic curve, between initial adjacent key frame, calculate Magnitude Difference between initial adjacent key frame according to characteristic curve and, according to threshold interpolation, obtain final key frame set.Attachedly Figure 1 shows that algorithm flow chart of the present invention, it specifically comprises following sport technique segment:

S1 adopts the LLE improved to carry out pre-service dimensionality reduction to original motion capture data;

The LLE method improved mainly is divided into following three steps:

Step one: select K Neighbor Points.For each frame X in higher dimensional space _i(i=1,2 ..., N), N is the totalframes of motion, calculates the distance d between it and other every frame _ij, range formula is:

$d_{\mathrm{ij}}=\left|\right|X_i-X_j\left|\right|/\sqrt{T\left(i\right)T\left(j\right)}$

Wherein, || X _i-X _j|| represent X _iand X _j(j=1,2 ..., N) between Euclidean distance, X _iand X _jrepresent the frame that in motion sequence two are different, T (i) and T (j) represents X respectively _ito mean value and the X of the distance between its K neighbour _jto the mean value of the distance between its K neighbour;

Step 2: the partial reconstruction weight matrix W being calculated this sample point by the Neighbor Points of each sample point;

Step 3: the output valve being calculated this sample point by partial reconstruction weight matrix and its Neighbor Points of this sample point.

S2 adopts principal component analysis (PCA) to carry out last dimension-reduction treatment, then adopts Lowess smothing filtering, arranges corresponding parameter to characteristic curve denoising, obtain one-dimensional characteristic curve.

S3, based on MATLAB platform, for one section of motion segments, extracts the Local Extremum on characteristic curve, thus obtains initial key frame set.

These initial key frame of S4 can not well reflect archeokinetic feature, need insertion one frame or multiframe to reach the object better can summarizing original motion sequence in the place that athletic posture changes greatly.

The present invention is in implementation procedure, and the value of design parameter needs to determine according to actual conditions, without specifically limiting, and according to circumstances value.

Below, with specific implementation of the present invention specific embodiment.Embodiments of the invention are implemented under premised on technical solution of the present invention, give detailed embodiment and concrete operating process, but protection scope of the present invention are not limited to following embodiment.Example sample chooses in CMU database (i.e. the motion capture database of Carnegie Mellon University of U.S. graphics establishment of laboratory).

Concrete implementation step is:

Step one: select the motion that two are representative from CMU database, ' playing football ' motion and ' walking ' motion.Adopt LLE and PCA merging improvement to do exercises to ' playing football ' and ' walking ' and carry out dimensionality reduction, adopt Lowess smothing filtering, corresponding parameter is set to characteristic curve denoising, obtains one-dimensional characteristic curve and then carry out key-frame extraction.

Step 2: in MATLAB, finds the Local Extremum on curve, obtains initial key frame set according to characteristic curve Curvature varying situation.

Step 3: the final key frame that the splitting-up method of feature based profile amplitude extracts is shown in accompanying drawing 2 and accompanying drawing 3.The Local Extremum that we extract has circle to represent as initial key frame, and asterisk represents final key frame simultaneously.Found by research, when motion intense attitudes vibration is larger, characteristic of correspondence profile amplitude changes greatly, and needs to insert extra key frame, and when athletic posture change is less, the change of characteristic of correspondence profile amplitude is less, does not need to insert additional key frame.By this method, we effectively can obtain final key frame set.

Step 4: the comparison of different Key-frame Extraction Algorithm.We have employed three kinds of methods: the inventive method, and classical LLE and curve simplify three kinds of methods, extract the key frame (ratio of compression is identical) of identical number from one ' jumping ' motion.The comparative result that distinct methods extracts key frame is shown in accompanying drawing 4,5 and 6.We are extracted 24 key frames, and the inventive method well summarises this motion, avoid over-sampling and lack sampling problem to a certain extent compared with other two kinds of methods.

Step 5: the impact chosen result of different Neighbor Points, we adopt the inventive method and classical LLE method to contrast, from one ' playing football ' motion, extract key frame, when Neighbor Points number K goes different value to make the Comparative result of two kinds of methods as table 1 and table 2:

Table 1: in classical LLE, the selection of different Neighbor Points K is on the impact of error rate

K	20	25	30	35	40
						Error rate	14.30	12.35	12.21	13.52	15.37
Crucial frame number	31	31	31	31	31

Table 2: in context of methods, the selection of different Neighbor Points K is on the impact of error rate

K	7	8	11	12	13
						Error rate	11.16	4.83	7.26	7.67	5.68
Crucial frame number	31	31	31	31	31

As can be seen from Table 1 and Table 2, when identical ratio of compression, the inventive method not only reduces error rate and decreases the selection of Neighbor Points number, illustrate that the inventive method is feasible, and it is different for the value of different motion K, when K gets 8 time, error rate is minimum as can be seen from Table 2, and the value of K to move across by the present invention the optimal value that a large amount of experiments selects for playing football.

Step 5: employing the inventive method, classical LLE and curve short-cut method test five kinds of dissimilar motion sequences, comprising playing football, jumping, running, walking, walk-jump-walk.Error rate to such as table 3, accompanying drawing 7-11. tri-kinds of methods we all use following formulae discovery absolute average error E:

E＝[Σ(F(i)-F’(i)) ²]/M

Here, i=1,2 ..., n represents the i-th frame, and n is motion totalframes, and F (i) is original motion sequence, and F ' (i) is the motion sequence after corresponding reconstruction, and M is multiplied by 96 degree of freedom for this motion totalframes.

The contrast of table 3. context of methods and classical LLE method time and error rate

Kind	Play football (31)	Jump (24)	Run (13)	Walk (16)	Walk-jump-walk (51)
						The inventive method	4.83	1.91	4.24	2.80	4.47
Classical LLE	8.08	2.67	5.35	4.46	11.89
						Curve simplifies	15.52	2.48	4.86	5.68	7.12

By testing five kinds of different motion types, in guarantee ratio of compression situation, be better than other two kinds of methods at the context of methods of relatively going up of error rate.

The above; be only the present invention's preferably embodiment; but protection scope of the present invention is not limited thereto; anyly be familiar with those skilled in the art in the technical scope that the present invention discloses; be equal to according to technical scheme of the present invention and inventive concept thereof and replace or change, all should be encompassed within protection scope of the present invention.

Claims (2)

1., based on the 3 d human motion extraction method of key frame merging LLE and PCA improved, it is characterized in that, it comprises the steps:

The LLE algorithm that S1, employing improve carries out pre-service dimensionality reduction to movement capturing data;

The LLE algorithm improved improves on classical LLE algorithm basis, and it is divided into three steps:

1), K Neighbor Points is selected; For each frame X in higher dimensional space _i(i=1,2 ..., N), N is the totalframes of motion, calculates the distance d between it and other every frame _ij, range formula is:

$d_{\mathrm{ij}}=\left|\right|X_i-X_j\left|\right|/\sqrt{T\left(i\right)T\left(j\right)}$

Wherein, || X _i-X _j|| represent X _iand X _j(j=1,2 ..., N) between Euclidean distance, X _iand X _jrepresent the frame that in motion sequence two are different, T (i) and T (j) represents X respectively _ito mean value and the X of the distance between its K neighbour _jto the mean value of the distance between its K neighbour;

2), by the Neighbor Points of each sample point calculate the partial reconstruction weight matrix W of this sample point, be initialized as unit matrix;

3) output valve of this sample point, is calculated by the partial reconstruction weight matrix of this sample point and its Neighbor Points.

S2, to data, reprocessing is carried out for the linear dimension reduction method of data acquisition PCA of pre-service dimensionality reduction, obtain one-dimensional characteristic curve, and elimination noise;

S3, the Local Extremum passed through on extraction characteristic curve obtain initial key frame;

S4, between initial key frame, the threshold value according to its characteristic curve Magnitude Difference and setting inserts corresponding frame number, merges overstocked key frame, obtains final key frame set.

2. according to claim 1 based on the extraction method of key frame of the movement capturing data of LLE and PCA of fusion improvement, it is characterized in that: it exports intrinsic dimensionality by the LLE algorithm of the automatic computed improved of program, and intrinsic dimensionality d by computing formula is:

$\mathrm{\ϵ}=\underset{j=1}{\overset{d}{\mathrm{\Σ}}}{\mathrm{\λ}}_j/\underset{j=1}{\overset{n}{\mathrm{\Σ}}}{\mathrm{\λ}}_j$

Wherein, ε is the contribution rate of rebuilding weight matrix, and choose different numerical value for different motions, d is intrinsic dimensionality, λ _j(j=1,2 ..., n) for rebuilding n eigenwert of weight matrix.