CN111291865A

CN111291865A - Gait recognition method based on convolutional neural network and isolated forest

Info

Publication number: CN111291865A
Application number: CN202010070608.7A
Authority: CN
Inventors: 曹倩; 徐菲; 刘立红; 左敏; 孙践知; 李丹莲
Original assignee: Beijing Technology and Business University
Current assignee: Beijing Technology and Business University
Priority date: 2020-01-21
Filing date: 2020-01-21
Publication date: 2020-06-16
Anticipated expiration: 2040-01-21
Also published as: CN111291865B

Abstract

The invention relates to a gait recognition method based on a convolutional neural network and an isolated forest, which comprises the following steps: step (1), data acquisition: an acceleration sensor and a gyroscope sensor which are arranged in a mobile phone are used for acquiring walking data of a plurality of users; step (2), data preprocessing: filtering the acquired data by adopting a low-pass finite impulse response filter FIR to remove noise information; carrying out interpolation on the acquired data to ensure that the intervals of sampling points of the acquired data are uniform; and (3) direction-independent conversion: the collected sensor data is subjected to direction-independent conversion, so that the influence of the placement position of the mobile phone on the identification accuracy is reduced; step (4), gait cycle division: adopting a fixed sliding window algorithm to periodically divide the gait, and setting the length of a sliding window to be 200, namely the sampling number per second; and (5) performing feature extraction on the collected gait data by using a CNN neural network.

Description

Gait recognition method based on convolutional neural network and isolated forest

Technical Field

The invention relates to a gait recognition algorithm, in particular to a method for carrying out direction-independent conversion on collected gait data, applying CNN which is successfully used in the image field to the gait recognition field to carry out gait cycle characteristic extraction, and carrying out gait recognition by using an unsupervised isolated forest algorithm.

Background

In most gait data acquisition tasks, the placement of the handpiece is controlled and is placed in an artificially defined orientation. However, in reality, this situation is unlikely to occur, and has limitations, while also causing inconvenience to the subject. Therefore, an algorithm is needed, which can prevent the direction of the collected data from changing along with the change of the placement position of the smart phone. To solve this problem, two different approaches are currently available. The first method is to extract the features of the rotation invariant (e.g. fourier transform or correlation matrix of dynamic gait images). The second method is to project the acquired signals into a new direction-invariant three-dimensional coordinate system depending on the transformation of the inertial signals.

The standard method of gait recognition is to use well-processed gait cycles to perform gait recognition by classical supervised machine learning algorithms (KNN, SVM, etc.). However, such an algorithm requires the computation of a large number of pre-established statistical features, of which the most informative features are selected and used to train the classifier. The process requires further feature selection by evaluating classification performance and iterating through the process. In this way, the features used for classification are progressively refined until a final set of features is obtained. However, these statistical characteristics are usually evaluated by designers through an educated guess or trial and error method, which is heavily artificial and computationally intensive. The invention uses a convolutional neural network for gait cycle feature selection. The convolutional neural network is successfully applied in the field of video processing, so that the convolutional neural network is considered to be applied to the field of inertial sensor data analysis. One of the advantages of using convolutional neural networks is that the statistical features of the gait cycle are automatically evaluated by the CNN as a result of the supervised training phase.

Supervised algorithms are commonly used in the gait recognition field for classification, such as K-nearest neighbors (KNNs), Support Vector Machines (SVMs), multi-layered perceptrons (MLPs), and Classification Trees (CTs). However, considering that the detection problem of abnormal gait only needs to distinguish the positive class or the negative class of gait, and the negative class is less, the invention uses unsupervised isolated forest algorithm to identify gait.

Disclosure of Invention

The technical problems to be solved by the invention are as follows: the invention adopts a method for extracting gait cycle characteristics based on a convolutional neural network, and can improve the traditional method for manually selecting the gait cycle characteristics. And finally, detecting abnormal gait by using an unsupervised isolated forest algorithm.

The solution of the invention is as follows: a gait recognition method based on a convolutional neural network and an isolated forest comprises the following steps:

step (1), data acquisition: the method comprises the steps that an acceleration sensor and a gyroscope sensor which are arranged in a mobile phone are used for collecting walking data of multiple users, the ages of the users are 20-26 years old, the sampling frequency of a smart phone is 200Hz, the mobile phone is placed in a trouser pocket of a subject when the data are collected, the collection time length is 10 minutes, the collected place is a corridor outside a laboratory, the collected data are stored in a mobile phone memory in a txt format, and the collected data comprise 8 lines which are time, acceleration x, y and z directions, gyroscope x, y and z directions and names of the users;

step (2), data preprocessing: filtering the acquired data by adopting a low-pass finite impulse response filter FIR to remove noise information; carrying out interpolation on the acquired data to ensure that the intervals of sampling points of the acquired data are uniform;

and (3) direction-independent conversion: the collected sensor data is subjected to direction-independent conversion, so that the influence of the placement position of the mobile phone on the identification accuracy is reduced;

step (4), gait cycle division: adopting a fixed sliding window algorithm to periodically divide the gait, and setting the length of a sliding window to be 200, namely the sampling number per second;

and (5) performing feature extraction on the collected gait data by using a CNN neural network, wherein the CNN neural network consists of two cascaded convolutional layers, a pooling layer and a full-connection layer, the convolutional layers execute a feature extraction task, the full-connection layer serves as a classifier, the convolutional layers are represented by CL, the full-connection layer is represented by FL, and an input matrix X is used:

X＝(a_ξ,a_ψ,a_ζ,a_mag,g_ξ,g_ψ,g_ζ,g_mag)^T

represents a gait cycle, wherein a represents an acceleration vector, g represents a gyro vector, x, y, z are coordinate axes before transformation, ξ, ψ, ζ is a coordinate axis after transformation, a represents a gait cycle_ξ,a_ψ,a_ζRepresenting gyroscope vectors in the ξ, ψ, ζ directions after direction-independent conversion, g_ξ,g_ψ,g_ζRepresenting the gyroscope vectors in the ξ, ψ, ζ directions, respectively, after direction-independent transformation, where a_magThe calculation formula is shown in formula (7), i is 1,2, and … … are sampling points;

a_mag(i)＝(a_x(i)²+a_y(i)²+a_z(i)²)^1/2(7)

a_x、a_y、a_zrespectively representing the components of the acceleration vector a in the x direction, the y direction and the z direction;

g_magthe calculation formula of (2):

g_mag(i)＝(g_x(i)²+g_y(i)²+g_z(i)²)^1/2；

g_x、g_y、g_zrespectively representing the components of a gyroscope vector g in the x, y and z directions;

step (6), constructing a gait recognition model: and constructing a gait recognition model by using an isolated Forest Isolation Forest algorithm.

Further, the step5 comprises:

step (5.1) setting the number of convolution kernels of each layer of the convolution neural network, and initializing the convolution neural network;

step (5.2) training a convolutional neural network by adopting a 10-fold cross validation mode;

step (5.3) adjusting parameters of each layer step by step according to the identification accuracy of the convolutional neural network, and repeating the step (5.1) and the step (5.2) until the convolutional neural network reaches a preset accuracy a;

and (5.4) removing the last full connection layer of the convolutional neural network, and taking the convolutional neural network as a gait cycle feature extraction tool.

Further, step6, inputting: a gait data set X; a sub-sampling size; the number L of iTrees generated; the method specifically comprises the following steps:

step (6.1) setting the maximum height of itree and initializing iForest;

step (6.2) constructing a first itree;

step (6.3) repeating the step (6.2), and constructing L-1 trees to form an initial forest;

step (6.4) training the initial forest T by using a training set X _ train, calculating the AUC of each iTree according to a cross-validation method, arranging the AUC in a descending order, and selecting the first 80% iTrees;

step (6.5) calculating the similarity between the reserved iTrees, storing each tree as a dictionary structure, calculating the vector inner product between the parent node and the child node, storing the vector inner product in the list, and comparing the same number in the two lists; the similarity s of the two trees can be obtained; setting a threshold t, if s is less than t, determining that the two similar iTrees are similar, deleting the tree with low AUC in the two similar iTrees, and reserving the tree with high AUC; finally, forming a new iForest from the remaining trees;

and (6.6) detecting the data to be detected, calculating the path length of the data in each itree, obtaining an abnormal score, and further judging whether the data is abnormal.

Compared with the prior art, the invention has the advantages that:

(1) the invention carries out direction-independent conversion on the data after data preprocessing, and can reduce the influence of the placement position of the mobile phone on the gait recognition rate;

(2) according to the gait cycle feature extraction method, the CNN which is successfully used in the image field is used for extracting the gait cycle feature, and the method is different from the conventional method for manually selecting the gait cycle feature, so that the influence of subjective factors during feature selection is avoided;

(3) the method uses the improved unsupervised isolated forest algorithm to detect abnormal gaits, is different from the previous mode of using the supervised algorithm to identify gaits, can save the time of identifying the gaits, and can provide a new idea for the field of identifying the gaits.

Drawings

FIG. 1: an acceleration sensor x, y, z-axis data map of a subject;

FIG. 2: comparing the x-axis data after cubic spline interpolation;

FIG. 3 is a graph of x-axis data after digital filtering;

FIG. 4 is a graph comparing x, y, z axis data after direction independent transformation of the data;

FIG. 5: a schematic of a step period;

FIG. 6: an ROC curve for gait recognition is carried out by using the constructed model;

FIG. 7: the invention is a general step flow chart.

Detailed Description

The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, rather than all embodiments, and all other embodiments obtained by a person skilled in the art based on the embodiments of the present invention belong to the protection scope of the present invention without creative efforts.

The invention relates to a gait recognition method based on a convolutional neural network and an isolated forest, which comprises the following steps:

step 1) gait data is collected through an Android APP designed by the user.

Step 2) design and verification of an original gait data preprocessing technology, wherein the design and verification comprise the following steps: and 4, extracting a gait cycle with good robustness, and mapping the motion signal to a reference system with invariable direction.

Step 3) a new Convolutional Neural Network (CNN) based feature extraction tool that is trained only once on a representative set of users and then serves as a generic feature extractor at runtime.

And 4) combining the features extracted by the CNN with an isolated Forest (Isolation Forest) trained only for the target object to construct a gait recognition model.

The gait recognition method based on the convolutional neural network and the isolated forest specifically comprises the following steps:

first step data acquisition

The method comprises the steps that an acceleration sensor and a gyroscope sensor which are built in a HUAWEI Mate9 mobile phone are used for collecting walking data of 10 users, the age is 20-26 years old, the sampling frequency of a smart phone is 200HZ, the mobile phone is placed in a trouser pocket of a subject when data are collected, the collection time is 10 minutes, the collection place is a corridor outside a laboratory, data collection software developed by android studio is used for collecting data, the collected data are stored in a mobile phone memory in a txt format, the collected data comprise 8 lines of time, the acceleration x, y and z directions, the gyroscope x, y and z directions, the name of the user, and the acceleration sensor x, y and z axis data of a certain subject are shown in figure 1.

Second step data preprocessing

(2.1) cubic spline interpolation

The set collection frequency in the data collection process of the invention is 200HZ, that is, 200 data are collected per second, the interval of each row is 5ms, but the data stored in the memory of the mobile phone is observed to find that the interval of every two data in the first column is not 5ms, which indicates that the intelligent mobile phone cannot ensure the uniformity of data sampling due to the inherent defects of software and hardware. Therefore, when the cubic spline interpolation method is adopted, the number of sampling values per second is fixed to 200, and the interpolated data pair is shown in fig. 2.

(2.2) digital Filtering

The digital filter is divided into a low-pass filter, a band-pass filter and a high-pass filter. And a low pass filter is required for processing gait noise. The invention adopts a low-pass Finite Impulse Response (FIR) filter to process gait information, the cut-off frequency of the filter is set to be 40HZ, and the data pair before and after filtering is shown in figure 3.

Third step of direction independent conversion

The purpose of the conversion is to find a four-component system consisting of three orthogonal quaternions

A coordinate system represented therein

Pointing upwards (parallel to the torso of the subject),

pointing forward (in line with the subject's direction of motion),

orthogonal to the other two.

Will gravity

And taking the average direction of gravity in the current gait cycle as the direction of the first coordinate axis of the new reference system as the starting point of data conversion. Suppose n_kIs the number of sampling points in the current gait cycle k (k is 1,2, … …), a_x，a_y，a_zRespectively representing the sampling points of the current gait cycle k along the x-axis, the y-axis and the z-axis, and | a_x|＝|a_y|＝|a_z|＝n_k。 g_x，g_y，g_zRepresents the gyroscope sample point in the current gait cycle, g_x|＝|g_y|＝|g_z|＝n_kThen the gravity loss in the period k

As shown in equation (1):

first quaternion of the new coordinate system

As shown in equation (2):

defining an acceleration matrix A ═ a_x,a_y,a_z]^TThe gyroscope matrix G ═ G_x,g_y,g_z]^TThen the acceleration and the gyroscope are in matrix

The projection of (c) is shown in equation (3):

the horizontal acceleration data is expressed, and the calculation formula is shown in formula (4).

A is calculated by Principal Component Analysis (PCA)^fFeature vector corresponding to maximum feature value

Then quaternion

As shown in equation (5):

quaternion

As shown in equation (6):

the data after direction independent conversion of the data is shown in fig. 4.

Fourth step dynamic cycle partitioning

The walking behavior of a person is not a regular periodic behavior, and the gait needs to be divided periodically in order to find information contained in the gait. The gait is periodically divided by adopting a fixed sliding window algorithm, and the length of a sliding window is set to be 200, namely the sampling number of one second.

Fifth step feature extraction and feature selection

The CNN consists of two cascaded convolutional layers, one pooling layer and one fully-connected layer. The convolutional layer performs the feature extraction task, while the fully-connected layer acts as a classifier, denoted CL for the convolutional layer and FL for the fully-connected layer. Gait cycle use input matrix X ═ (a)_ξ,a_ψ,a_ζ,a_mag,g_ξ,g_ψ,g_ζ,g_mag)^TWherein a denotes an acceleration vector, g denotes a gyro vector, x, y, z are coordinate axes before transformation, ξ, ψ, ζ are coordinate axes after transformation, wherein a denotes an acceleration vector, g denotes a gyro vector, x, y, z denotes a gyro vector, and_magthe formula is shown in formula (7), and the formula for gmag is the same as a_mag I 1,2, … … is the sample point:

a_mag(i)＝(a_x(i)²+a_y(i)²+a_z(i)²)^1/2(7)

g_magthe calculation formula of (2):

g_mag(i)＝(g_x(i)²+g_y(i)²+g_z(i)²)^1/2；

CL1 is the first layer convolutional layer, performs the first filtering of the input using a one-dimensional convolutional kernel of 1 × 10, and processes each input vector. The activation function is linear, the number of convolution kernels being N_k1。

CL2 is the second convolution layer, and uses a convolution kernel size of 4 × 10, and the activation function is a nonlinear tanh function. Max pooling is applied to the output of CL2 for the purpose of dimensionality reduction and reduced spatial invariance. The number of convolution kernels is N_k2。

FL1 is the first fully connected layer, to which each output of CL2 is connected (weights not shared). The activation function used for this layer is tanh. The output vector of FL1 is f ═ f (f)₁,...,f_F)^T。

FL2 second fully connected layer, one class for each output neuron, for a total of K neurons. K is the number of subjects in the training phase. K-dimensional output vector y ═ y₁,...,y_K)^TIs obtained through the softmax activation function, y_j∈(0,1),j＝1,...,K，

Indicating the probability that the current input belongs to a certain class.

The invention uses 10-fold cross validation to train CNN, directly uses the well-trained CNN network reaching a certain accuracy as a feature extraction tool, and removes the final full connection FL 2. That is, given an input matrix X, a feature vector F associated with the input is returned, where the number F of feature vectors F is defined as 40. And then, reducing the dimension of the feature vector by adopting principal component analysis, wherein the number S of the finally selected feature vectors S is 20.

Step six identification model construction

The gait recognition model is constructed by using an improved isolated forest algorithm, the algorithm can be roughly divided into two stages, and t isolated trees are trained in the first stage to form an isolated forest. Each sample point is then brought into each isolated tree in the forest, the average height is calculated, and then the outlier score for each sample point is calculated. The Isolation Forest procedure is as follows.

The first stage is as follows:

(1) for a given data set X ═ X₁,...,x_n},

x_i＝(x_i1,...,x_id) Where n denotes the number of subjects collected and d denotes the number of features per subject, a subset X' of ψ sample points are randomly drawn from X and placed in the root node.

(2) Randomly assigning a dimension q from d dimensions, and randomly generating a cut point p, min (x) in the current data_ij,j＝q,x_ij∈X')＜p＜max(x_ij,j＝q,x_ij∈X')。

(3) This cut point p generates a hyperplane, dividing the current data space into two subspaces: sample points with dimensions smaller than p are designated to be placed in the left child node, and sample points with dimensions larger than or equal to p are designated to be placed in the right child node.

(4) Recursion (2) and (3) until all leaf nodes have only one sample point or the isolated tree (iTree) has reached a specified height.

(5) And (4) circulating from (1) to (4) until t isolated trees are generated.

And a second stage:

and for each data point x, traversing each isolated tree, calculating the average height h (x) of the point in the forest, namely the number of edges from the root node to the external nodes, and normalizing the average height of all the points. The outlier score s (x, ψ) is shown in equation (8):

wherein c (ψ) is as shown in formula (9):

let T be a binary tree and N be a node of T, which is called an external node if N is a leaf node and an internal node if N is a node with two children. E (H (x)) is the average value of H (x) in the iTree set, H (ψ) ═ Ln (ψ) + γ, γ is the euler constant; n is the number of leaf nodes; c (n) is the average of h (x) given n to normalize h (x).

Although an isolated Forest (hereinafter, referred to as iForest) algorithm reduces abnormal covering and inundation effects to a certain extent and has low linear time complexity, random factors are artificially introduced, so that the problems of low precision, poor stability and the like are caused. And when constructing the iForest, redundant iTrees exist, so that memory space waste and calculation amount increase are caused, and the execution efficiency of the algorithm is influenced. The invention improves the traditional iForest algorithm,

in the process of realizing gait recognition of the isolated forest model, after an Area Under Cut (AUC) value (defined as an area enclosed by coordinate axes under an ROC Curve) of each iTree is calculated, descending order sorting is performed on the AUC, and itrees with high AUC are selected to form a new iForest. Because the training samples for generating the iTrees are random each time, and the node feature selection is also random, the iTrees have certain correlation. The greater the correlation between any two trees in a forest, the greater the error rate.

The correlation is obtained by the similarity, and the method for calculating the similarity in the experiment is as follows:

each tree is stored as a dictionary structure, each node has corresponding index and value values to represent features and division values, the vector inner product between a parent node and a child node is calculated and stored in a list, and the same quantity in the two lists is compared. The similarity of the two trees can be obtained.

The inner product of the parent node and the child node is calculated as follows:

suppose a parent node

Child node

The inner product calculation formula is shown as formula (10):

inner product is parent^T·child (10)

Therefore, by setting a threshold, if within a certain degree of correlation, they are considered similar, and the tree with low AUC in the two similar itrees is deleted, and the tree with high AUC is retained. This reduces the correlation between trees. Finally, a new iForest is composed of the remaining trees. The specific steps for constructing the gait model are shown in algorithm 1:

algorithm 1: improved isolated forest algorithm

Inputting: a gait data set X; a sub-sample size ψ; number of iTrees generated L

Step1 sets the maximum height of itree, initializing iForest.

Step2 constructs the first itree.

Step3 Step2 is repeated, and L-1 trees are constructed to form an initial forest.

Step4, training the initial forest T by using a training set X _ train, calculating the AUC of each iTree according to a cross-validation method, arranging the AUC in a descending order, and selecting the first 80% iTrees.

Step5 calculates the similarity between the reserved iTrees, stores each tree as a dictionary structure, calculates the vector inner product between the parent node and the child node, stores the vector inner product in the list, and compares the same number in the two lists. The similarity s of the two trees can be obtained. And (3) setting a threshold value t, if s < t, the two similar iTrees are considered to be similar, deleting the tree with low AUC from the two similar iTrees, and reserving the tree with high AUC. Finally, a new iForest is composed of the remaining trees.

Step6, detecting the data to be detected, calculating the path length h (x) of the data x in each itre, and obtaining an abnormal score s (x, ψ), thereby judging whether the data x is abnormal.

Results and analysis of the experiments

The accuracy is the most common evaluation index, and generally speaking, the higher the accuracy, the better the classifier. The classification index we choose is the accuracy.

The calculation formula is as follows:

accuracy＝(TP+TN)/(P+N)

note: (1) if an instance is a positive class and is predicted to be a positive class, it is a True class (True Positive TP);

(2) if an instance is a positive class, but is predicted to be a Negative class, i.e., a False Negative class (False Negative FN);

(3) if an instance is a negative class, but is predicted to be a positive class, i.e., a False positive class (False positive FP);

(4) if an instance is a Negative class, but is predicted to be a Negative class, namely a True Negative class (True Negative TN);

(5) p is the number of positive classes in the sample and N is the number of negative classes in the sample.

In the experimental process, different iTrees are set, the number of the iForest is respectively 40, 50, 60, 70, 80, 90 and 100, the iForest before improvement is tested firstly, then the iForest after improvement is tested, and the accuracy of the iForest after improvement is improved by 3.02% at most through comparison of the accuracy. Overall, the improved iForest performance is better.

The experimental data are shown by a graph, so that the comparison result can be displayed more visually. Aiming at the gait recognition problem, the improved iForest is improved in accuracy compared with the prior iForest.

TABLE 1 model accuracy comparison

iTree number	iForest before improvement (%)	Improved iForest(％)
			40	91.60	94.62
50	91.85	93.23
			60	92.82	94.65
70	93.42	94.89
			80	92.82	94.26
90	92.82	94.26
			100	93.32	96.00

Finally, the number of itrees is set to 100 to create a gait recognition model, and the final result is shown in fig. 6, where the ROC curre Curve has false positive rate on the abscissa (FPR, FPR: the proportion of all samples that are actually negative (0) that are erroneously determined to be positive (1)), true positive rate on the ordinate (TPR, TPR: the proportion of all samples that are actually positive (1) that are correctly determined to be positive (1)), solid line representing a Curve composed of TPR and FPR obtained by isolated forest at different thresholds, and dotted line representing TPR-FPR (i.e., the proportion of samples that are predicted to be positive regardless of whether the samples are positive or negative).

It can be seen that the accuracy of the final identification of the gait model constructed by the isolated forest is 96%.

Although illustrative embodiments of the present invention have been described above to facilitate the understanding of the present invention by those skilled in the art, it should be understood that the present invention is not limited to the scope of the embodiments, but various changes may be apparent to those skilled in the art, and it is intended that all inventive concepts utilizing the inventive concepts set forth herein be protected without departing from the spirit and scope of the present invention as defined and limited by the appended claims.

Claims

1. A gait recognition method based on a convolutional neural network and an isolated forest is characterized by comprising the following steps:

X＝(a_ξ,a_ψ,a_ζ,a_mag,g_ξ,g_ψ,g_ζ,g_mag)^T

a_mag(i)＝(a_x(i)²+a_y(i)²+a_z(i)²)^1/2(7)

g_magthe calculation formula of (2):

g_mag(i)＝(g_x(i)²+g_y(i)²+g_z(i)²)^1/2；

2. A gait recognition method based on convolutional neural network and isolated forest as claimed in claim 1, characterized in that: the step5 comprises the following steps:

3. A gait recognition method based on convolutional neural network and isolated forest as claimed in claim 1, characterized in that: the step6 is to input: a gait data set X; a sub-sampling size; the number L of iTrees generated; the method specifically comprises the following steps:

step (6.1) setting the maximum height of itree and initializing iForest;

step (6.2) constructing a first itree;