CN118245850A - Human lower limb movement intention recognition method and system under non-ideal condition - Google Patents

Abstract

The invention relates to the field of artificial intelligence and intelligent rehabilitation, and particularly discloses a method and a system for identifying movement intention of human lower limbs under non-ideal conditions. Aiming at the problems of low precision, poor quality and the like of the continuous movement intention recognition of the lower limbs under non-ideal conditions, the human body lower limb movement intention recognition method based on the improved neural network is designed. The method comprises the following steps: a. collecting physiological signals of continuously walking lower limbs; b. analyzing muscle and physical states of a subject and performing signal preprocessing; c. establishing an improved neural network model according to requirements; d. inputting preprocessed signals for training and searching optimal super parameters of an improved neural network model; e. inputting the preprocessed surface electromyographic signal characteristics to predict the knee joint angle and verifying the prediction quality and accuracy of the model.

Description

Human lower limb movement intention recognition method and system under non-ideal condition

Technical Field

The invention relates to the field of artificial intelligence and intelligent rehabilitation, and particularly discloses a method and a system for identifying movement intention of human lower limbs under non-ideal conditions.

Technical Field

The intention recognition technology has been one of the key problems of man-machine interaction solved by the exoskeleton rehabilitation robot, and the core purpose of the technology is to enable the exoskeleton rehabilitation robot to understand and predict the action intention of a user, so as to provide proper assistance or enhance the exercise capacity of the user. The human body movement intention recognition means that the motion to be completed by a human body is predicted in advance by using a model through the acquired human body physiological signals, so that the exoskeleton rehabilitation robot is helped to respond, and the human-computer interaction capability is improved. In the field of medical rehabilitation, the user is helped to perform rehabilitation actions better by enhancing the response speed and coordination of the exoskeleton robot. For example, during rehabilitation training of a cerebral apoplexy patient, the exoskeleton robot can assist the patient in rehabilitation exercise, and the intention recognition method with good prediction quality can enable the rehabilitation exercise to be more standard, so that a better rehabilitation effect is achieved.

In the practical use of an exoskeleton rehabilitation robot, an early human lower limb movement intention recognition method cannot adapt to complex reality conditions. At present, the method for identifying the lower limb movement intention is mostly continuous movement prediction and classification prediction under ideal conditions, and continuous movement prediction under non-ideal conditions cannot be achieved. Non-ideal conditions refer to electrode deflection, muscle fatigue, individual differences and other comprehensive disturbances, such as reduced muscle function of the user due to long-term rehabilitation movements when using the exoskeleton rehabilitation robot, and failure of the predictive model to accurately identify intent causes reduced accuracy to cause inconsistent and deformed movements of the exoskeleton rehabilitation robot and even injury to the person.

The existing lower limb movement intention recognition technology has the problems of lower precision, poor quality and the like under non-ideal conditions. Therefore, no mature and applicable technical scheme exists in the fields of intelligent rehabilitation and the like.

Disclosure of Invention

The invention discloses a human body lower limb movement intention recognition method under non-ideal conditions, which aims at the problems of low precision, complex physiological signals and the like of human body lower limb continuous movement intention recognition under fatigue state. Based on the root mean square characteristic of the collected human body lower limb surface electromyographic signals as model input, training is performed by using an improved long-short-term memory recurrent neural network, and finally, high-precision lower limb continuous motion prediction is realized. The technical scheme of the invention is as follows:

A method for identifying the movement intention of the lower limbs of a human body under non-ideal conditions comprises the following steps:

a. collecting lower limb surface electromyographic signals and knee joint angular displacement signals of continuous walking of a subject in a period of time;

b. calculating the average power of the surface electromyographic signals and shannon entropy, analyzing the muscle and physical state of the subject, and carrying out signal preprocessing;

c. establishing an improved neural network model according to requirements;

d. Inputting the preprocessed surface electromyographic signals and knee joint angular displacement signals, training and searching the optimal super parameters of the improved neural network model;

e. The root mean square characteristic of the surface electromyographic signals after being input and preprocessed predicts the knee joint angle through an improved neural network model and verifies the prediction quality and accuracy of the model.

Further, in the step c, the improved neural network model specifically includes:

s1: the output value of the hidden layer is obtained through a long-short-term memory recurrent neural network, and the specific expression is as follows:

Wherein, Is root mean square characteristicA kind of electronic deviceThe root mean square feature vector of the moment,，，As a matrix of weights, the weight matrix,As a result of the bias term,、AndRespectively isAn input door, a forget door and an output door at moment,For cell state candidates, tanh is a hyperbolic tangent function used to generate an output between-1 and 1,The value is updated for the state of the cell,As a function of the sigmoid,AndThe hidden layer outputs at times t-1 and t respectively,Representing the dot product between elements, i.e., hadamard product;

s2: calculating the output value of the hidden layer by using a multi-head self-attention mechanism, wherein the specific expression is as follows:

Wherein, Is the firstA weight matrix of the individual self-attention heads,Is the firstThe bias term of the individual self-attention head,Is vector quantityIs used in the manufacture of a printed circuit board,Is a linear mapping matrix, n is the number of attention points,Softmax is the activation function, concat is the tensor splice,Is the output of the hidden layer and,；

S3: calculating the output value of the multi-head self-attention by using a dropout layer, wherein the specific expression is as follows:

Wherein, Is one and withThe random vectors of the same shape, each element of which is independently and equidistributed, are derived from Bernoulli distribution with probability of，Representing the dot product between elements, i.e., hadamard product;

s4: the output value of the dropout layer is calculated by using a channel attention mechanism with the maximum value removed, and the specific expression is as follows:

wherein n is a vector The number of elements in the matrix, max is the orientation quantityExp is an exponential function based on the natural exponent e;

S5: the predicted values of the models are output by using residual connection, a dropout layer and a full connection layer, and the specific expression is as follows:

the function of Linear is to perform Linear transformation on input data, and mean is the average value of the input.

The invention also provides a human lower limb movement intention recognition system under non-ideal conditions, which comprises:

The signal acquisition module is used for acquiring lower limb surface electromyographic signals and knee joint angles of the subject during continuous movement;

the signal preprocessing module is used for processing the surface electromyographic signals by using fourth-order Butterworth filtering and average filtering, extracting root mean square characteristics from the processed signals and carrying out maximum and minimum standardization processing;

The continuous motion prediction module is used for carrying out continuous angle prediction by using the preprocessed signals;

And the angle output and quality verification module outputs a predicted angle, calculates root mean square error and a decision coefficient by using the real angle and the predicted angle, and verifies the predicted quality.

Compared with the prior art, the invention has the advantages that:

The invention provides an improved long-term and short-term memory recurrent neural network intention recognition technology combined with an attention mechanism under non-ideal conditions. Is characterized in that the output of the single long-short-term memory recurrent neural network when the intention recognition is carried out on the complex and continuous lower limb movement has huge oscillation. Therefore, a multi-head self-attention mechanism is adopted to extract characteristics from hidden layer output of a single long-short-term memory recurrent neural network, and attention distribution of different subspaces of the hidden layer is obtained, so that association among hidden layer sequences is more comprehensively captured, and vibration is reduced. And because the prediction accuracy of multi-head self-attention is low, a channel attention mechanism is used for calculating the whole output weight and reassigning the output, and the maximum value in the whole sequence is removed when the weight is calculated in consideration of calculation time and stability. Compared with other super-parameter numerical value optimization methods, the TPE algorithm based on the Bayesian idea can search the super-parameter space more effectively. Conventional grid searches or random searches require a large number of iterations to cover a wide parameter space, while TPEs use historical information to guide the search process, focusing more on the optimizable parameter area. The method has small iteration times and can avoid sinking into local optimum.

Drawings

FIG. 1 is a flow chart of a method for identifying the movement intention of the lower limbs of a human body under non-ideal conditions according to the invention;

Fig. 2 (a) shows the acquisition position of the physiological signal acquisition device of a healthy subject on a human body; fig. 2 (b) acquisition position of the physiological signal acquisition device of the stroke subject on the human body;

Fig. 3 (a) is a graph of average power in healthy subjects; fig. 3 (b) is a graph of average power in a stroke subject; fig. 3 (c) is a shannon entropy graph of healthy subjects; fig. 3 (d) shannon entropy plot of stroke subjects;

FIG. 4 is a flow chart of signal preprocessing;

FIG. 5 is a block diagram of an improved neural network;

FIG. 6 is a flowchart of a TPE algorithm;

FIG. 7 is a block diagram of a human lower limb movement intention recognition system under non-ideal conditions according to the present invention;

fig. 8 (a) is a graph of true angle and predicted angle for healthy subject 1; fig. 8 (b) is a graph of true angle and predicted angle for healthy subject 2; fig. 8 (c) is a graph showing the actual angle and the predicted angle of the stroke subject 3.

Detailed Description

The invention is further illustrated by the following examples and the accompanying drawings:

Example 1

Fig. 1 is a flowchart of a method for identifying movement intention of lower limbs of human body under non-ideal condition according to the present invention, as shown in fig. 1, the specific method is as follows:

a. the physiological signal acquisition equipment is used for acquiring the myoelectric signals of the surface of the lower limb and the knee joint angles of the human body walking for thirty minutes, and the specific steps are as follows:

S1: the acquisition equipment is a Biopac human surface myoelectricity information acquisition system and a Biopac angular displacement sensor. As shown in fig. 2 (a) and (b), the muscle positions acquired by the human surface myoelectricity information acquisition system are biceps femoris, semitendinosus, tibialis anterior and gastrocnemius, and the positions acquired by the angular displacement sensor are left leg knee joints of healthy subjects and diseased side (left leg) knee joints of cerebral apoplexy subjects;

S2: healthy subjects performed thirty minutes of continuous exercise on the treadmill, with the treadmill speed set at 2.5 km/h. Since stroke subjects cannot safely exercise on the running machine for physical reasons, continuous exercise on a flat ground is selected until the stroke subjects feel tired and cannot exercise. Limited by the wireless receiving range of the acquisition equipment, a cerebral apoplexy subject can only move around a certain point within a certain range, so that the angle range of the knee joint is inconsistent in some periods.

B. the average power of the surface electromyographic signals and shannon entropy are combined to analyze the muscle fatigue state and the health state, and the collected data are preprocessed, wherein the specific steps are as follows:

S1: the analysis of the relationship between the surface electromyographic signals and the muscle fatigue states can better train the model effectively, and the prediction accuracy under different fatigue conditions is improved. As muscle fatigue increases, the conduction velocity of muscle fibers decreases, and the spectrum of the electromyographic signals tends to shift to lower frequencies, and the average power of the muscle signals decreases as the degree of fatigue increases, as shown in fig. 3 (a) and (b). Meanwhile, as the fatigue of the muscle is gradually increased, the initially activated muscle fiber is fatigued to lose, and the body maintains the same level of output by activating new muscle fiber or activating the same muscle group in a different mode so that the signal becomes more complicated, and thus the entropy value rises as shown in fig. 3 (c) and (d). The analysis of the shannon entropy of the surface electromyographic signals can effectively distinguish the health status of the body. Since healthy physiological systems can flexibly respond to changes in the internal and external environments, exhibit rich dynamic changes, physiological signals of healthy people generally exhibit higher complexity and randomness. In contrast, cerebral stroke or other disease states may lead to impaired certain functions of the physiological system, reflecting lower complexity and more regular, predictable signal patterns. Thus, higher entropy values are typically associated with healthy and adaptable systems, while lower entropy values may indicate impaired system function or poor adaptation, as shown in fig. 3 (c) and (d);

S2: the specific expression of the average power of the surface electromyographic signals is as follows:

wherein the surface electromyographic signal is ，，，At a frequency ofPower spectral density value, frequency at=2000Hz，Is the number of segments used for the averaging,Is the firstSegment surface electromyographic signalsThe fast fourier transform result of (2) is at frequencyAn array comprising complex frequency components,Is the firstFrequency of segmentsIs a periodic chart of (2);

s3: the specific expression of shannon entropy of the surface electromyographic signals is as follows:

Wherein, Is the firstSegment surface electromyographic signalsIs used as a reference to the entropy of (a),Is the firstSegment surface electromyographic signalsTaking a specific valueIs a function of the probability of (1),Is a logarithmic function, base b=2;

S4: the flow chart of the surface electromyographic signal preprocessing step is shown in fig. 4, and the collected surface electromyographic signals are subjected to fourth-order Butterworth filtering and average filtering to remove high-frequency noise generated by power supply, equipment interference and high-frequency muscle tremor, and meanwhile sharp peaks of the signals are reduced, so that the electromyographic signals are smoother;

S5: the root mean square characteristic of the surface area electric signal after filtering is calculated, and the specific expression is as follows:

Wherein, Is the firstSegment surface electromyographic signalsRoot mean square of (a);

s6: and the minimum and maximum standardization is used for the root mean square characteristics, the adjustment range is 0 to 1, the convergence speed of the model is improved, and the numerical stability is improved.

C. An improved long-term and short-term memory recurrent neural network model is established according to the requirements, the structure of the model is shown in figure 5, and the specific steps are as follows:

S1: the output value of the final hidden layer is obtained by using the long-term memory recurrent neural network of the N layers, and the specific expression is as follows:

Wherein, Is root mean square characteristicA kind of electronic deviceThe root mean square feature vector of the moment,，，As a matrix of weights, the weight matrix,As a result of the bias term,、AndRespectively isAn input door, a forget door and an output door at moment,For cell state candidates, tanh is a hyperbolic tangent function used to generate an output between-1 and 1,The value is updated for the state of the cell,As a function of the sigmoid,AndThe hidden layer outputs at times t-1 and t respectively,Representing the dot product between elements, i.e., hadamard product;

S2: the multi-head self-attention mechanism enables models to learn information in different subspaces by processing multiple self-attention heads in parallel, mining features and patterns in the data more carefully. Multi-head self-attention gives the model more flexibility and more expressive power multi-head self-attention mechanisms by splitting the self-attention mechanism into multiple "heads", each learning a representation of a different aspect of the input data, in parallel. Each "head" implements a set of independent linear transformations of the query (Q), key (K) and value (V), then calculates the attention score, and finally outputs the weighted value. The outputs of these different heads are spliced together and passed through another linear transformation to form the final output. Calculating the output value of the hidden layer by using a multi-head self-attention mechanism, wherein the specific expression is as follows:

Wherein, Is the firstA weight matrix of the individual self-attention heads,Is the firstThe bias term of the individual self-attention head,Is vector quantityIs used in the manufacture of a printed circuit board,Is a linear mapping matrix, n is the number of attention points,Softmax is the activation function, concat is the tensor splice,Is the output of the hidden layer and,；

S3: the dropout layer is only used during the model training phase, which should be turned off (i.e., keep all neurons activated) during the verification or test phase. The output is "scaled" by multiplying the outputs of all neurons by the retention probability, thereby counteracting the effects of some cells not being activated during training. Calculating the output value of the multi-head self-attention by using a dropout layer, wherein the specific expression is as follows:

Wherein, Is one and withThe same shape of the random vector, each element of which is independently and equispaced, is derived from a bernoulli distribution, with a probability of p,Representing the dot product between elements, i.e., hadamard product;

S4: omitting extreme maxima can make the model more stable in the face of different inputs, and will not produce excessive output variation due to extreme value fluctuation. The channel attention mechanism can improve the sensitivity of the network to important channels in the input features, adjust the weights of the channels accordingly, strengthen the important features and inhibit the unimportant features. The output value of the dropout layer is calculated by using a channel attention mechanism with the maximum value removed, and the specific expression is as follows:

wherein n is a vector The number of elements in the matrix, max is the orientation quantityExp is an exponential function based on the natural exponent e;

S5: in the model that introduces residual links, the inputs are added directly to the outputs of the later layers by a "jump connection". Thus, the model needs to learn to become the residual error between the input and the output, instead of directly learning the output, so that the performance of the model is improved, and the gradient disappearance problem is relieved. The predicted value of the model is output by using a residual error link, a dropout layer and a full connection layer, and the specific expression is as follows:

the function of Linear is to perform Linear transformation on input data, and mean is the average value of the input.

D. The TPE algorithm based on the Bayesian optimization basic idea is used for training and searching the optimal super-parameters of the neural network model by using the root mean square characteristics of the surface electromyographic signals and the knee joint angles in different fatigue states after pretreatment, and the optimal prediction effect is obtained, and the specific steps are as follows:

S1: the neural network model is trained by using the root mean square characteristics of the surface electromyographic signals and the knee joint angles with different fatigue degrees after pretreatment, so that the neural network model can be helped to better learn and predict the corresponding knee joint angles under different fatigue conditions;

s2: searching the super-parameters (iteration times, hidden layer number, learning rate and neuron number) of the model can enable the model to have better learning and prediction effects on specific input signals, and a TPE algorithm based on a Bayesian optimization basic idea can be used for searching the optimal super-parameters of the model, as shown in fig. 6, and the specific steps are as follows:

S201: and (3) establishing a model: the parameter space is divided into two parts according to the quantiles of the objective function value (e.g. the first 25% are optimizable parameters, the rest are non-optimizable parameters): optimizable parameters And non-optimizable parameters. The probability density function is built for each part using a Parzen window estimator, expressed in:

Wherein, AndRespectively optimizable and non-optimizable parameter sets,Is based onAs a central kernel function, gaussian kernels are typically used,Is the bandwidth of the kernel function;

step S202: generating candidate points and optimizing : Selecting new candidate parametersBy slave ofSampling to generate new parameter points, and calculating each pointSelecting the parameter with the highest expected improvement，The expression is:

Step S203: evaluating and updating: using the selected Running the objective function to obtain new observation result and updating the modelAnd。

E. Root Mean Square Error (RMSE) is used for measuring deviation between predicted value and actual value and determining coefficient [ ]) The ratio of the model predicted variation to the total variation is measured. Collecting lower limb surface electromyographic signals and knee joint angles of walking under different muscle fatigue conditions of a human body through physiological signal collecting equipment, predicting the knee joint angles, verifying the prediction quality and accuracy of a model, and carrying out RMSE and RMSEThe expression is:

Wherein, 、AndRespectively the firstActual observed value, the firstAnd the number of predicted values and the average value of all observed values, wherein n is the total number of the observed values.

Example two

Fig. 7 is a block diagram of a system for recognizing movement intention of lower limbs of human body under a non-ideal condition according to the present invention, as shown in fig. 7, the specific structure includes:

The signal acquisition module is used for acquiring the lower limb surface electromyographic signals and knee joint angles of the subject during continuous movement, wherein the acquisition position of the physiological signal acquisition equipment is the position in the step a of the intention recognition method;

the signal preprocessing module is used for processing the surface electromyographic signals by using fourth-order Butterworth filtering and average filtering, extracting root mean square characteristics from the processed signals and carrying out maximum and minimum standardization processing;

The continuous motion prediction module is used for carrying out continuous angle prediction by using the preprocessed signals, wherein the model is an improved neural network model in the step c of the intention recognition method, and the model hyper-parameters are obtained by optimizing the TPE algorithm in the step d of the intention recognition method;

And the angle output and quality verification module outputs a predicted angle, calculates root mean square error and a decision coefficient by using the real angle and the predicted angle, and verifies the predicted quality. Three subjects use the system of the invention, namely a healthy subject 1, a healthy subject 2 and a cerebral apoplexy subject 3, wherein the healthy subjects 1 and 2 have good exercise capacity in physical health, and the cerebral apoplexy subject 3 is a cerebral apoplexy patient with weak leg functions. The true angle and predicted angle curves of the subject, the root mean square error and the decision coefficients output by this module are shown in fig. 8.

Claims (3)

1. A method for identifying the movement intention of a human lower limb under non-ideal conditions, which is characterized by comprising the following steps:

a. Collecting lower limb surface electromyographic signals and knee joint angular displacement signals of continuous walking of a subject in a period of time; b. calculating the average power of the surface electromyographic signals and shannon entropy, analyzing the muscle and physical state of the subject, and carrying out signal preprocessing; c. establishing an improved neural network model according to requirements; d. inputting the root mean square characteristic of the surface electromyographic signals and the knee joint angular displacement signals after pretreatment, training and searching the optimal super parameters of the improved neural network model; e. the root mean square characteristic of the surface electromyographic signals after being input and preprocessed predicts the knee joint angle through an improved neural network model and verifies the prediction quality and accuracy of the model.

2. The method for recognizing the movement intention of the lower limbs of the human body under the non-ideal condition according to claim 1, wherein the improved neural network model comprises the steps of:

s1: the output value of the hidden layer is obtained through a long-short-term memory recurrent neural network, and the specific expression is as follows:

Wherein/> Is root mean square characteristic/>/>Time vector,/>，The LSTM consists of an input gate, a forgetting gate, an output gate, a candidate value of the cell state and a candidate value cell state update value, wherein the update in the LSTM consists of the sum of the candidate value cell state update value at one moment multiplied by the output gate and the candidate value of the cell state multiplied by the forgetting gate, and the point multiplication among elements is Hadamard product;

s2: calculating the output value of the hidden layer by using a multi-head self-attention mechanism, wherein the specific expression is as follows:

，/>， Wherein/> For/>Weight matrix of individual self-attention header,/>For/>Bias term of self-attention head,/>Is vector/>Dimension,/>Is a linear mapping matrix, n is the attention header number,/>Softmax is the activation function, concat is tensor stitching, linear is the Linear transformation,Is the output of the hidden layer,/>；

S3: calculating the output value of the multi-head self-attention by using a dropout layer, wherein the specific expression is as follows:

wherein r is an AND/> Random vectors of the same shape, each element of which is independently co-distributed, are derived from Bernoulli distribution with a probability p (i.e., each element has a probability of 0 p and a probability of 1-p),/>Representing the dot product between elements, i.e., hadamard product;

s4: the output value of the dropout layer is calculated by using a channel attention mechanism with the maximum value removed, and the specific expression is as follows:

Wherein n is vector/> The number of elements in (a) and (b) max is the orientation quantity/>Exp is an exponential function based on the natural exponent e;

S5: the predicted values of the models are output by using residual connection, a dropout layer and a full connection layer, and the specific expression is as follows:

，/> wherein, the Linear function is to perform Linear transformation on the input data, mean is the average value of the input.

3. A system for identifying movement intent of a lower extremity of a human body under non-ideal conditions, the system comprising:

The signal acquisition module acquires lower limb surface electromyographic signals and knee joint angles when the subject continuously moves;

The signal preprocessing module is used for processing the surface electromyographic signals by using fourth-order Butterworth filtering and average filtering, extracting root mean square characteristics from the processed signals and carrying out minimum and maximum standardization processing;

The continuous motion prediction module is used for carrying out continuous angle prediction by using the preprocessed signals;

And the angle output and quality verification module outputs a predicted angle, calculates root mean square error and a decision coefficient by using the real angle and the predicted angle, and verifies the predicted quality.