WO2015165260A1

WO2015165260A1 - Triaxial feature fusion method for human body movement identification

Info

Publication number: WO2015165260A1
Application number: PCT/CN2014/092630
Authority: WO
Inventors: 薛洋; 胡耀全; 金连文
Original assignee: 华南理工大学
Priority date: 2014-04-29
Filing date: 2014-12-01
Publication date: 2015-11-05
Also published as: CN103984921A; CN103984921B

Abstract

Provided is a triaxial feature fusion method for a human body movement identification, comprising the following steps: (1) based on the triaxial feature representation of a feature base, representing the triaxial features as a linear combination of the features base, and determining the coefficient of each axial feature base; (2) fusion weight: calculating the fusion weight of each axial feature based on a variance contribution ratio by utilizing the coefficient of each axial feature base; (3) triaxial feature fusion: fusing the triaxial features by utilizing the amount of contribution of each axial feature for different movements identification to improve the rate of movement identification. The present invention has a high accuracy rate of movement identification.

Description

Three-axis feature fusion method for human motion recognition

Technical field

The invention relates to a pattern recognition and artificial intelligence technology, in particular to a three-axis feature fusion method for human body motion recognition.

Background technique

In recent years, with the development of personal electronic devices, smart phones have more and more built-in sensing devices and enhanced embedded computing capabilities. When the smart phone is placed in a trouser pocket or a backpack, as the frequency of the human body changes, the mobile phone acceleration sensor can detect the state of the human body motion, which greatly improves the convenience of recognizing the human body's motion behavior, and the mobile phone acceleration sensor gradually becomes the human body. The ideal platform for classification of sports patterns. However, the classification of human motion patterns based on smartphone accelerometers still has great limitations and difficulties. One of them is the poor fusion ability of the acceleration signal features. The convergence rate of many features is reduced after the fusion, and the successful fusion method is also based on many considerations. The fusion effect is not very good. For this reason, we propose a more common feature fusion algorithm with high recognition rate.

Summary of the invention

The object of the present invention is to overcome the shortcomings and deficiencies of the prior art, and to provide a three-axis feature fusion method for human body motion recognition, which is an extraction method suitable for a three-axis acceleration signal fusion feature of a smart phone.

The object of the present invention is achieved by the following technical solution: a three-axis feature fusion method for human motion recognition, comprising the following steps:

(1) Based on the triaxial feature representation of the feature base, the triaxial feature is represented as a linear combination of feature bases, and the coefficients of the feature base of each axis are determined;

(2) Fusion weights, using the coefficients of the feature base of each axis, and calculating the fusion weight of each axis feature based on the variance contribution rate;

(3) Three-axis feature fusion, using the contribution of each axis feature to different motion recognition to fuse three axes Features to improve the recognition rate of motion recognition.

In the step (1), the triaxial feature representation based on the feature base is a linear combination in which the triaxial feature is represented as a feature base, thereby determining the coefficient of the feature base of each axis, and the specific method is as follows:

The time-frequency domain features of each axis are extracted from a large number of three-axis acceleration signal samples composed of multiple types of motions to form a three-axis feature vector space, which is denoted as F=[F _x , F _y , F _z ], F _x , F _y , F _z denotes the eigenvectors of the x-axis, the y-axis and the z-axis, respectively, and the eigenvector dimension of each axis is denoted as m, that is, the triaxial feature vector space F is a matrix of 3×m, then the triaxial feature vector can Expressed as a linear combination of the characteristic bases [X ₁ , X ₂ ,..., X _m ], namely:

F _x =A _x1 X ₁ +A _x2 X ₂ +...+A _xm X _m +ε _x

F _y =A _y1 X ₁ +A _y2 X ₂ +...+A _ym X _m +ε _y , (1)

F _z =A _z1 X ₁ +A _z2 X ₂ +...+A _zm X _m +ε _z

Wherein, the characteristic bases [X ₁ , X ₂ , . . . , X _m ] can uniquely represent the eigenvectors of each axis, and A _ij is a 3×m matrix representing the coefficients of the feature base, i∈{x, y, z}, j=[1,2,...,m]. ε _i represents the error balance for each axis, i ∈ {x, y, z};

The linear combination of the characteristic base X = [X ₁ , X ₂ , ..., X _m ] reconstructs the cost function of the triaxial eigenvector space F by sparse coding, expressed in matrix form as follows:

The base vector X is sparsely penalized by the L ₁ norm, and the coefficient matrix A of the feature base is constrained by the L ₂ norm to prevent the coefficient matrix A of the feature base from being too large; but the L ₁ norm at the base vector X The number is not divisible at 0, so the gradient function can't be used to optimize the above cost function, so to be derivable at 0, the formula (2) becomes:

By performing the following algorithm on the formula (3), the feature base X and the coefficient matrix A of the feature base which minimize J(A, x) can be determined, and the algorithm includes the following steps:

1) Randomly initialize A;

2) According to the given A of step 1), solve the X which can minimize J(A, x);

3) According to the X obtained in step 2), solve the A which can minimize J(A, x);

4) Repeat steps 2) and 3) until AX converges to F.

The step (2) fuses the weights, and the fusion weight coefficients of the feature vectors of each axis are extracted as follows:

The variance contribution rate can be calculated using the coefficients of the characteristic base per axis:

among them,

Indicates the mean of the characteristic basis coefficients of each axis;

Because the coefficients of the characteristic bases of the three axes of each type of action should be stabilized near the mean of the triaxial coefficients, respectively, so that the feature space of each type of action is stable, so that the feature spaces of different actions will be less overlapped. The recognition effect is better. Therefore, the variance contribution rate of the characteristic basis coefficient is recalculated as follows:

By compressing the variance contribution rate of the three-axis characteristic base coefficient, the fusion weight matrix of the triaxial feature can be obtained, which is denoted as W=[W _x , W _y , W _z ], W _x , W _y , W _z denotes the feature fusion weights of the x-axis, the y-axis and the z-axis, respectively. The fusion weight matrix W of the tri-axis feature is also a 3×m matrix, and the feature fusion weight after amplitude compression is expressed as:

Where [W _x , W _y , W _z ] represents the feature fusion weight after amplitude compression.

The triaxial feature fusion in the step (3) can obtain the fused feature vector by using the fusion weight of the triaxial feature:

EFF=[F _x ,F _y ,F _z ][W _x ,W _y ,W _z ] ^T , (7)

Where EFF represents the fused feature vector.

The invention is based on the fusion method of three-axis feature representation of feature base, fusion weight determination and triaxial feature fusion, and the specific method can also be described as follows:

1. A three-axis feature representation based on a feature base;

The time-frequency domain features of each axis are extracted from a large number of three-axis acceleration signal samples composed of multiple types of motions to form a three-axis feature vector space, which is denoted as F=[F _x , F _y , F _z ], F _x , F _y , F _z denotes the eigenvectors of the x-axis, the y-axis, and the z-axis, respectively, and the eigenvector dimension of each axis is denoted by m, that is, the triaxial feature vector space F is a matrix of 3×m. Then the triaxial feature vector can be expressed as a linear combination of the characteristic bases [X ₁ , X ₂ , . . . , X _m ], ie

F _x =A _x1 X ₁ +A _x2 X ₂ +...+A _xm X _m +ε _x

F _y =A _y1 X ₁ +A _y2 X ₂ +...+A _ym X _m +ε _y , (1)

F _z =A _z1 X ₁ +A _z2 X ₂ +...+A _zm X _m +ε _z

Among them, the feature base [X ₁ , X ₂ , ..., X _m ] can uniquely represent the feature vector of each axis. A _ij is a 3 × m matrix representing the coefficients of the feature base, i ∈ {x, y, z}, j = [1, 2, ..., m]. ε _i represents the error balance for each axis, i ∈ {x, y, z}.

Here, the sparsity penalty is applied to the base vector X by the L ₁ norm, and at the same time, to prevent the coefficient matrix A of the feature base from being too large, it is constrained by the L ₂ norm. However, the L ₁ norm at the base vector X is not divisible at 0, so the above cost function cannot be optimized by the gradient descent method, so that in order to be derivable at 0, the formula (2) is changed to:

By performing the following algorithm on the formula (3), it is possible to determine the feature base X which minimizes J(A, x) and the coefficient matrix A of the feature base. The algorithm is as follows:

1) Randomly initialize A;

2) According to the given A of step 1), solve the X which can minimize J(A, x);

4) Repeat steps 2) and 3) until AX converges to F.

2. Determination of the fusion weight;

The fusion weight of each axis feature is calculated by using the variance contribution rate of the characteristic base coefficients of each axis. details as follows:

among them,

Indicates the mean of the characteristic base coefficients for each axis.

Because the coefficients of the characteristic bases of the three axes of each type of action should be stabilized near the mean of the triaxial coefficients, respectively, so that the feature space of each type of action is stable, so that the feature spaces of different actions will be less overlapped. The recognition effect is better. Therefore, based on formula (4), the variance contribution rate of the characteristic base coefficient of each axis is recalculated as follows:

By compressing the variance contribution rate of the three-axis characteristic base coefficient, the fusion weight matrix of the triaxial feature can be obtained, which is denoted as W=[W _x , W _y , W _z ], W _x , W _y , W _z represents the feature fusion weight of the x-axis, the y-axis, and the z-axis, respectively, and the fusion weight W of the tri-axis feature is also a 3×m matrix. The feature fusion weight after amplitude compression is expressed as:

3. Three-axis feature fusion;

The fused feature vector can be obtained by using the fusion weight of the triaxial feature obtained by formula (6):

EFF=[F _x , F _y , F _z ][W _x , W _y , W _z ] ^T , (7) where EFF denotes the fused feature vector.

The present invention has the following advantages and effects over the prior art:

The motion recognition accuracy is high; the present invention utilizes the contribution of each axis feature to the different motion recognition to fuse the triaxial features, and the three-axis feature extracted from the three-axis acceleration signal obtained from the smartphone acceleration sensor is characterized. The three-axis feature representation of the base, the fusion weight determination and the triaxial feature fusion achieve the purpose of improving the accuracy of motion recognition.

DRAWINGS

1 is a flow chart of a feature fusion method of the present invention.

detailed description

The present invention will be further described in detail below with reference to the embodiments and drawings, but the embodiments of the present invention are not limited thereto.

Example

The input device used in this embodiment is a smart phone with a built-in three-axis acceleration sensor, and the present invention can be better implemented by processing with a computer.

The system flow chart of the three-axis feature fusion algorithm for human motion recognition is shown in FIG. 1 , and the specific steps include:

1. Pretreatment;

For the collected three-axis acceleration signal, the three-axis acceleration signal of different people is normalized to the range of [-1, 1] by linearly normalizing the ratio.

2. Determine the coefficient of the feature base of each axis based on the three-axis feature representation of the feature base;

F _x =A _x1 X ₁ +A _x2 X ₂ +...+A _xm X _m +ε _x

F _y =A _y1 X ₁ +A _y2 X ₂ +...+A _ym X _m +ε _y , (1)

F _z =A _z1 X ₁ +A _z2 X ₂ +...+A _zm X _m +ε _z

1) Randomly initialize A;

2) According to the given A of step 1), solve the X which can minimize J(A, x);

4) Repeat steps 2) and 3) until AX converges to F.

3. Determine the fusion weight of the triaxial feature based on the variance contribution rate of the characteristic basis coefficient

among them,

Indicates the mean of the characteristic base coefficients for each axis.

By compressing the variance contribution rate of the three-axis characteristic base coefficient, the fusion weight matrix of the triaxial feature can be obtained, which is denoted as W=[W _x , W _y , W _z ], W _x , W _y , W _z represents the feature fusion weight of the x-axis, the y-axis, and the z-axis, respectively, and the fusion weight W of the tri-axis feature is also an m×3 matrix. The feature fusion weight after amplitude compression is expressed as:

4. Three-axis feature fusion;

The three-axis feature space F=[F _x , F _y , F _z ] is compressed by the formula (6) into a uniaxial feature EFF:

EFF=[F _x ,F _y ,F _z ][W _x ,W _y ,W _z ] ^T , (7)

5. Bayesian network classification;

The Bayesian network classifier is trained with 80% of the sample feature set generated above, and then the remaining 20% uses the Bayesian network classifier to identify the action class for each test sample.

The superior performance of the present invention was confirmed by experiments in large samples. The results of correlation experiments on a large number of human body motion signal samples using the feature fusion method of the present invention are described below.

There is currently no public database for human motion recognition based on smartphone accelerometers. In this embodiment, five kinds of motion data (walking, running, jumping, going upstairs and downstairs) of 87 people are collected, and a total of 87 sets of data are collected. Each class randomly selected 70 sets of samples (80% of the total number of samples in each class) for training. The total number of training samples was 350, and the remaining 17 sets of samples were used for testing. The total number of test samples was 85.

In the experiment, the motion recognition rate based on the triaxial feature fusion method proposed by the present invention is compared with the recognition rate of the five actions before the feature fusion. As shown in Table 1 (Table 1 is a table comparing the five kinds of motion recognition rates before and after feature fusion), the recognition rates of five kinds of actions before and after feature fusion are given.

Table 1

It can be seen from Table 1 that the feature obtained by the triaxial feature fusion extraction method proposed by the present invention has a recognition rate significantly higher than that of the unfused triaxial acceleration signal. Therefore, the experimental results show that the triaxial feature-based fusion feature extraction method of the present invention is significantly superior to the traditional unfused time-frequency domain feature in performance.

The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and combinations thereof may be made without departing from the spirit and scope of the invention. Simplifications should all be equivalent replacements and are included in the scope of the present invention.

Claims

A three-axis feature fusion method for human motion recognition, comprising the following steps:

(1) Based on the triaxial feature representation of the feature base, the triaxial feature is represented as a linear combination of feature bases, and the coefficients of the feature base of each axis are determined;

(2) Fusion weights, using the coefficients of the feature base of each axis, and calculating the fusion weight of each axis feature based on the variance contribution rate;

(3) Three-axis feature fusion, using the contribution of each axis feature to different motion recognition to fuse the three-axis feature, improve the recognition rate of motion recognition.
The triaxial feature fusion method for human motion recognition according to claim 1, wherein the triaxial feature representation based on the feature base in the step (1) is a linear combination of representing the triaxial feature as a feature base. , thereby determining the coefficient of the characteristic base of each axis, the specific method is as follows:

The time-frequency domain features of each axis are extracted from a large number of three-axis acceleration signal samples composed of multiple types of motions to form a three-axis feature vector space, which is denoted as F=[F x , F y , F z ], F x , F y , F z denotes the eigenvectors of the x-axis, the y-axis and the z-axis, respectively, and the eigenvector dimension of each axis is denoted as m, that is, the triaxial feature vector space F is a matrix of 3×m, then the triaxial feature vector can Expressed as a linear combination of the characteristic bases [X 1 , X 2 ,..., X m ], namely:

F x =A x1 X 1 +A x2 X 2 +...+A xm X m +ε x

F y =A y1 X 1 +A y2 X 2 +...+A ym X m +ε y , (1)

F z =A z1 X 1 +A z2 X 2 +...+A zm X m +ε z

Wherein, the characteristic bases [X 1 , X 2 , . . . , X m ] can uniquely represent the eigenvectors of each axis, and A ij is a 3×m matrix representing the coefficients of the feature base, i∈{x, y, z}, j=[1,2,...,m]; ε i represents the error balance for each axis, i∈{x,y,z};

The linear combination of the eigenbases X = [X 1 , X 2 , ..., X m ] reconstructs the cost function of the triaxial eigenvector space F by sparse coding, expressed in matrix form as follows:

The base vector X is sparsely penalized by the L 1 norm, and the coefficient matrix A of the feature base is constrained by the L 2 norm; to be derivable at 0, the formula (2) is changed to:

By performing the following algorithm on the formula (3), the feature base X and the coefficient matrix A of the feature base which minimize J(A, x) can be determined, and the algorithm includes the following steps:

1) Randomly initialize A;

2) According to the given A of step 1), solve the X which can minimize J(A, x);

3) According to the X obtained in step 2), solve the A which can minimize J(A, x);

4) Repeat steps 2) and 3) until AX converges to F.
The three-axis feature fusion method for human body motion recognition according to claim 1, wherein the step (2) fuses the weight, and the fusion weight coefficient of each axis feature vector is extracted as follows:

The variance contribution rate can be calculated using the coefficients of the characteristic base per axis:

among them,
Indicates the mean of the characteristic basis coefficients of each axis;

The variance contribution rate of the characteristic base coefficient is recalculated as follows:

The variance contribution rate of the three-axis characteristic base coefficient is compressed in the amplitude to obtain the fusion weight matrix of the three-axis feature, which is denoted as W=[W x , W y , W z ], W x , W y , W z respectively The feature fusion weights representing the x-axis, the y-axis, and the z-axis, and the fusion weight matrix W of the triaxial feature are also a 3×m matrix, and the feature fusion weights after amplitude compression are expressed as:

Where [W x , W y , W z ] represents the feature fusion weight after amplitude compression.
The three-axis feature fusion method for human body motion recognition according to claim 1, wherein the three-axis feature fusion in the step (3) obtains the merged feature vector by using the fusion weight of the three-axis feature:

EFF=[F x ,F y ,F z ][W x ,W y ,W z ] T , (7)

Where EFF represents the fused feature vector.