CN112232161A - Complex motion continuous estimation method based on electromyography mapping model switching - Google Patents

Complex motion continuous estimation method based on electromyography mapping model switching Download PDF

Info

Publication number
CN112232161A
CN112232161A CN202011066204.7A CN202011066204A CN112232161A CN 112232161 A CN112232161 A CN 112232161A CN 202011066204 A CN202011066204 A CN 202011066204A CN 112232161 A CN112232161 A CN 112232161A
Authority
CN
China
Prior art keywords
estimation
model
lstm
signals
electromyographic
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011066204.7A
Other languages
Chinese (zh)
Other versions
CN112232161B (en
Inventor
张琴
皮特
熊蔡华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huazhong University of Science and Technology
Original Assignee
Huazhong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huazhong University of Science and Technology filed Critical Huazhong University of Science and Technology
Priority to CN202011066204.7A priority Critical patent/CN112232161B/en
Publication of CN112232161A publication Critical patent/CN112232161A/en
Application granted granted Critical
Publication of CN112232161B publication Critical patent/CN112232161B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/12—Classification; Matching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/02—Preprocessing
    • G06F2218/04—Denoising

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • Image Analysis (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)

Abstract

The invention discloses a complex motion continuous estimation method based on electromyography mapping model switching, belonging to the field of man-machine interaction, which comprises the steps of firstly collecting joint angle signals and multi-channel electromyography signals according to joint angles to be estimated and surface muscles corresponding to joint motion, dividing a task into a plurality of subtasks according to the characteristics of the complex task, then, respectively preprocessing the collected electromyographic signals and joint angle signals, establishing a mapping sub-relationship of the electromyographic signals and the joint angle signals by adopting an LSTM deep learning network to obtain an estimation sub-model, then, a mapping relation is established between the electromyographic signals and each estimation submodel by using an RF ensemble learning algorithm to obtain an RF switching model, the characteristics of the electromyographic signals are classified by using the RF switching model, the estimation submodel to which the current movement belongs is judged, and finally, the electromyographic characteristics are input into the corresponding estimation submodel to obtain a joint angle estimation value. The invention effectively improves the estimation precision of each joint angle under complex tasks.

Description

Complex motion continuous estimation method based on electromyography mapping model switching
Technical Field
The invention belongs to the field of human-computer interaction, and particularly relates to a complex motion continuous estimation method based on switching of an electromyography mapping model.
Background
Currently, the myoelectric interface is the only clinically feasible form of neural interface. It uses the surface muscle electromyogram signal as a signal source to estimate the movement of each joint of the human body, which has become a clinically feasible solution, but only the movement pattern recognition based on the electromyogram signal is really put into application. Many students have invested a lot of research and obtained certain achievements in motion continuous estimation and control of external equipment (such as a mechanical arm, a mechanical arm and an exoskeleton) based on electromyographic signals, but still many problems exist when the students are actually invested in clinical application, and one of the main problems is that the functions of the electromyographic-controlled external equipment cannot adapt to complex tasks in real life, for example, people can easily allocate each joint of an upper limb to complete a series of actions of extending an arm (reaching the position of a water cup by a hand), grasping (grasping the water cup by fingers), bending the arm (taking back the water cup), lifting the arm and turning over the wrist (drinking water) and the like, and the task of drinking water is completed. However, the series of actions involves precise fitting and scheduling of multiple joints in time and space. For another example, in daily life, people sometimes need to grip a soft and light object such as a paper cup, sometimes need to grip a hard and heavy object such as a pan handle, and can self-adaptively grip the paper cup with different forces according to different gripping objects.
The task is complicated by the change of factors such as force, posture, speed and the like, and particularly, the task is extremely difficult to be continuously estimated by using the electromyographic signals, meanwhile, in the complicated task, different muscle groups are activated by a human body at different time or space to drive different joint motions, and the mapping relation between the electromyographic signals of each stage and the joint motions is different. Therefore, a single estimation model is used for establishing a mapping relation between the electromyographic signals and the joint angles of the complex task, a good result is often not obtained, the model is poor in robustness, and accurate estimation of the whole task cannot be completed smoothly.
Therefore, it is necessary to develop a new complex motion continuous estimation method to successfully complete the precise estimation of complex task.
Disclosure of Invention
Aiming at the defects and improvement requirements of the prior art, the invention aims to provide a complex motion continuous estimation method based on electromyography mapping model switching, and aims to solve the technical problem that complex tasks cannot be continuously estimated in the prior art.
In order to achieve the above object, the present invention provides a complex motion continuous estimation method based on electromyography model switching, which includes the following steps:
s1: collecting joint angle signals and multi-channel electromyographic signals according to the joint angle to be estimated and surface muscles driving corresponding joints to move, dividing a task into a plurality of subtasks according to the characteristics of a complex task, combining the plurality of subtasks into the whole complex task,
s2: respectively preprocessing the collected electromyographic signals and joint angle signals,
s3: utilizing the preprocessed electromyographic signals and joint angle signals, adopting an LSTM deep learning network to establish a mapping sub-relation of the electromyographic signals and the joint angle signals to obtain an estimation sub-model,
s4: establishing a mapping relation between the electromyographic signals and each estimation submodel by using an RF ensemble learning algorithm to obtain an RF switching model,
s5: and classifying the electromyographic signal characteristics by using an RF switching model, judging an estimation submodel to which the current movement belongs, and inputting the electromyographic characteristics into the corresponding estimation submodel to obtain the joint angle estimation value at the moment.
Further, in step S2, the preprocessing of the electromyographic signals includes filtering, feature extraction, and normalization of the electromyographic signals, specifically, performing notch filtering on the electromyographic signals to eliminate power frequency interference, performing 20Hz to 460Hz band-pass filtering on the electromyographic signals to eliminate low frequency noise and high frequency noise, performing feature extraction on the electromyographic signals subjected to noise reduction, obtaining feature values of the electromyographic signals by using a sliding window method, and then normalizing the electromyographic features of the channels according to the maximum value and the minimum value of the feature values of the channels to obtain the electromyographic features after normalization of the channels.
Further, the preprocessing of the joint angle signal in step S2 is to perform filtering processing on the measured joint angle signal and the estimated joint angle signal, respectively, so that the measured joint angle and the estimated joint angle are input and output to the corresponding models at relatively smooth angles.
Further, in step S3, the obtained estimation submodel is an LSTM estimation submodel, and the specific process of obtaining the LSTM estimation submodel is as follows: firstly, initializing parameters of the LSTM deep learning network, and determining a weight matrix W of a candidate stateCAnd bias bCWeight matrix W of input gatesiAnd bias biForgetting gate weight matrix WfAnd bias bfWeight matrix W of output gatesoAnd bias boInitializing to a random number between 0 and 1, setting the number of neurons in an input layer of an LSTM deep learning network to be M, setting the number of layers to be i, taking the output of each layer as the input of the next layer, training the LSTM network after the initialization of the LSTM network is completed, inputting preprocessed electromyographic signals and joint angle signals into an LSTM model, and optimizing the multiple weight matrixes and the multiple biases by adopting a back propagation algorithm to obtain an LSTM estimation sub-model.
Further, in step S5, a mapping relationship is established between the electromyographic signals and each LSTM estimation submodel by using an RF ensemble learning algorithm to obtain an RF switching model, which specifically includes: and training to obtain an RF switching model by taking the preprocessed electromyographic signals and the corresponding label of the LSTM model as input, wherein the output of the RF switching model is the label of the LSTM estimation sub-model, the label of the LSTM model is a number of 1-n, and n LSTM estimation sub-models are generated.
Furthermore, the joints in the joint angle signals are human body compound joints such as joints of human hands, shoulder and elbow joints and the like, the multichannel electromyographic signals are electromyographic signals generated by relevant surface muscles driving the compound joints to move, and the selection of the number of the channels of the electromyographic signals is determined according to a compound joint angle target estimated according to needs.
Further, the method further includes step S6, where step S6 is: and adopting the Pearson correlation coefficient and the root mean square error as evaluation indexes of the actually measured joint angle and the estimated joint angle so as to evaluate the estimation precision of the estimated joint angle.
Generally, compared with the prior art, the above technical solution conceived by the present invention has the following beneficial effects:
the invention improves the estimation precision of a single estimation submodel by utilizing the advantage of strong long-term memory of the LSTM deep learning network in the electromyographic signals. By adopting a model switching mechanism, the defect that a single model cannot be accurately estimated in a complex task is overcome, and accurate estimation under the complex task is realized. Experimental results show that the method effectively improves the estimation precision of the angle of each joint under the complex task.
Drawings
FIG. 1 is a flow chart of the implementation of the complex motion continuous estimation method based on electromyography mapping model switching;
FIG. 2 is a schematic diagram of an LSTM deep learning network employed by the present invention;
FIG. 3 is a schematic diagram of an RF handoff model employed by the present invention;
FIG. 4 is a graph of the estimation accuracy of the LSTM submodel of the present invention applied to five fingers;
FIG. 5 is a graph of the comparison of the estimation accuracy of the present invention using the LSTM hybrid estimation model in combination with the model switching mechanism with a single LSTM submodel for complex gripping tasks.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention relates to a composite motion continuous estimation method based on electromyographic signals and an LSTM deep learning network under a complex task. And an LSTM hybrid estimation model combined with a model switching mechanism is provided to realize the continuous estimation of the human joint compound motion under complex tasks. The LSTM has great advantages in processing time series problems, introduces the cell state to store long-term memory, and improves the defect that the RNN recurrent neural network can only store short-term memory. The method has the advantages that the mapping model between the electromyographic signals and human body joint motions is built by using the LSTM deep learning network, and the estimation accuracy can be improved compared with other machine learning methods.
Specifically, for different task scenarios in real life, the task is divided into a plurality of subtasks according to task complexity characteristics, for example, the tasks are decomposed into n tasks, and different subtasks are trained by using an LSTM deep learning network respectively to obtain a plurality of LSTM estimation submodels. And training the electromyographic signals and the LSTM estimation submodels to which the electromyographic signals belong by using a Random Forest (RF) ensemble learning algorithm to obtain a switching model.
The technical scheme adopted by the invention comprises the following steps:
s1: and collecting joint angle signals and multi-channel electromyographic signals according to the joint angle to be estimated and surface muscles driving corresponding joint motion. And dividing the task into a plurality of subtasks according to the characteristics of the complex task, and combining the plurality of subtasks into the whole complex task. And selecting human joint angles with multiple degrees of freedom as an estimation target according to the requirements of n subtasks decomposed from the complex tasks, and collecting multi-channel electromyographic signals as system input.
S2: and preprocessing the electromyographic signals, including filtering, feature extraction and normalization of the electromyographic signals. Specifically, the surface electromyogram signal is too weak and is easily interfered by a surrounding electromagnetic field, the surface electromyogram signal is easily interfered by power frequencies of joint angle measuring equipment and other electrical equipment in the process of actually measuring the electromyogram signal, and the interference frequency distribution is 50Hz and integer times of the frequency. Therefore, the electromyographic signals need to be processed for eliminating power frequency interference. Band-pass filtering processing is carried out according to the main frequency of the electromyographic signals so as to eliminate motion artifact noise (low frequency) and other high-frequency noise. Feature extraction is carried out on the electromyographic signals subjected to noise reduction processing, and different single features or combined features such as Root Mean Square (RMS), Waveform Length (WL), absolute value Integral Average Value (IAV), Mean Absolute Value (MAV), zero crossing coefficient (ZC), Slope Sign Change (SSC), average amplitude change rate (AAC) and the like are selected according to different tasks. And normalizing the electromyographic features of the channels according to the maximum value and the minimum value of the characteristic values of the channels respectively to obtain the electromyographic features after normalization of the channels as input signals of the RF switching model and the LSTM model.
And preprocessing the joint angle signal, and filtering the actually measured joint angle signal and the estimated joint angle signal to input and output the joint angle to the model at a relatively smooth angle.
S3: and respectively training an LSTM estimation sub-model by taking the electromyographic features of each sub-task and the corresponding joint angle signals as input.
Initializing parameters of the LSTM deep learning network: weighting matrix W of candidate stateCAnd bias bCWeight matrix W of input gatesiAnd bias biForgetting gate weight matrix WfAnd bias bfWeight matrix W of output gatesoAnd bias boInitializing to a random number between 0 and 1. And setting the number of neurons in an input layer of the LSTM deep learning network as M, setting the number of layers as i, and using the output of each layer as the input of the next layer.
And training the LSTM network after the initialization of the LSTM network is completed, inputting the preprocessed electromyographic signals and joint angle signals into the LSTM model, and optimizing the weight matrix and the bias by adopting a back propagation algorithm to obtain an LSTM estimation sub-model. The LSTM model is also the LSTM network.
Setting the number of training iterations as epoch, loss function as loss, optimizer as optimizer, and batch size as the number of samples selected in one training.
S4: and training an RF switching model, namely taking the preprocessed electromyographic signals and the corresponding label of the LSTM model as model input to train the RF switching model, wherein the output of the RF switching model is the label of the LSTM estimation sub-model. Performing bootstrap sampling on input samples to obtain k sample sets, respectively training the k sample sets by using a decision tree model to obtain k tree classifiers, wherein labels of the LSTM models are numbers 1-n, n LSTM estimation submodels are generated in total, the maximum depth of each tree classifier is set to be m, a gini function calculation method is adopted for judging whether nodes are continuously split, the minimum sample number required by node splitting is set to be f, the maximum leaf node number is not limited, and the maximum feature number participating in judgment during node splitting is equal to all feature numbers.
S5: after the RF switching model and each LSTM estimation sub-model are trained, the whole LSTM hybrid estimation model combined with the model switching mechanism is trained, and the hybrid model is used for carrying out continuous motion estimation on newly input electromyographic signal data under a complex task in the testing stage. Particularly, in the estimation stage, the electromyographic features of all channels are normalized by using the maximum value and the minimum value of the electromyographic features of all channels in the training stage, the normalized electromyographic features are input into the RF switching model, the LSTM estimation submodel to which the corresponding electromyographic signals belong is output, then the normalized electromyographic features are input into the corresponding LSTM estimation submodel for estimation, and the estimation value of each joint angle is output.
In practical engineering practice, the continuous estimation effect of the composite motion of the LSTM hybrid estimation model combined with the model switching mechanism under the complex task is evaluated, and during evaluation, firstly, the estimation accuracy of a single LSTM model in each estimation sub-model is evaluated to verify the estimation accuracy of each estimation sub-model, and secondly, the estimation effects of the single LSTM model and the LSTM hybrid estimation model combined with the model switching mechanism under the complex task are compared to verify whether the LSTM hybrid estimation model combined with the model switching mechanism can better adapt to the change of the task scene or not, so that the estimation accuracy and the robustness for the complex task are higher. The evaluation indexes are Pearson Correlation Coefficient (CC) and Root Mean Square Error (RMSE). The result shows that the method can effectively improve the estimation precision of the angle of each joint under the complex task.
The whole flow chart during estimation is shown in fig. 1, electromyographic signals are preprocessed to obtain electromyographic features, the electromyographic features are input into an RF (radio frequency) switching model to obtain classification results, labels of an LSTM estimation sub-model are output, and then the electromyographic features enter the corresponding LSTM estimation sub-model to output estimated compound motion angles. Because the results need to be compared, the characteristic value obtained after the electromyographic signals are preprocessed can also be directly output by the composite motion angle through a single LSTM model. The angle outputs respectively obtained by the two models are compared in CC and RMSE dimensions so as to highlight the good estimation effect of the LSTM hybrid estimation model combined with the model switching mechanism under the complex task.
The invention is described in further detail below with reference to the figures and the specific embodiments. In particular, the complicated task involved in the embodiments is that people need to grip different objects with different forces in daily life, for example, when gripping a paper cup, a small force is needed to ensure the gripping, when gripping a pot handle to pick up a cooking pot, a large force is needed, and in ordinary life, only a moderate force is needed to take a bottle of water.
(1) And dividing the task into a plurality of subtasks according to the characteristics of the complex task, and combining the plurality of subtasks into the whole complex task. According to the characteristic that people can grasp different objects by adopting different forces in the daily life situation, the joint angle of the hand with five degrees of freedom is selected as an estimation target, and the six-channel electromyographic signal is selected as system input. The five-freedom-degree hand joints are respectively a thumb metacarpophalangeal joint, an index finger metacarpophalangeal joint, a middle finger metacarpophalangeal joint, a ring finger metacarpophalangeal joint and a little finger metacarpophalangeal joint. The six-channel muscle signals are the flexor hallucis longus, the flexor hallucis digitalis, the flexor digitorum profundus, the extensor digitorum, and the extensor hallucis longus, respectively. And collecting joint angle signals and multi-channel electromyographic signals according to the joint angle to be estimated and surface muscles driving corresponding joint motion.
(2) And preprocessing the electromyographic signals, including filtering, feature extraction and normalization of the electromyographic signals.
(2a) Filtering: because the surface electromyogram signal is too weak and is easily interfered by surrounding electromagnetic fields, the surface electromyogram signal is interfered by power frequency of finger angle measuring equipment and other electrical equipment in the process of actually measuring the electromyogram signal, and the interference frequency is distributed to 50Hz and integral multiple frequency thereof. Therefore, a notch filtering process (Comb-notch Filter) is performed on the electromyographic signals to eliminate power frequency interference. As the main frequency of the electromyographic signals is concentrated at 20-460 Hz, the electromyographic signals are subjected to 20-460 Hz band-pass filtering processing to eliminate motion artifact noise (low frequency) and other high-frequency noise.
(2b) Feature extraction is carried out on the electromyographic signals subjected to noise reduction processing, the main variable factor in the complex situation task is force, and the average absolute value feature of the electromyographic signals represents the activity level of muscles so as to reflect the gripping force to a certain extent, and therefore the average absolute value (MAV) is selected as the feature. The electromyographic signals are subjected to characteristic value acquisition by adopting a sliding window method, the size of the sliding window is set to be 200ms, the sliding window is backwards slid for 50ms each time, and 75% of overlapped data exists between adjacent windows.
(2c) And after obtaining the electromyographic features, respectively normalizing the electromyographic features of the channels according to the maximum value and the minimum value of the characteristic values of the channels, and obtaining the electromyographic features after normalization of the channels as input signals of a subsequent RF switching model and an LSTM model (two subsequent models).
Preprocessing a five-degree-of-freedom finger metacarpophalangeal joint angle signal, performing Butterworth low-pass filtering processing (cutoff frequency is 2Hz,4 orders) on an actually measured angle signal, and enabling a joint angle to be input into an LSTM estimation model at a relatively smooth angle.
(3) Dividing the task into 3 subtasks according to the task complexity, wherein the subtasks are respectively a task one: grip the object with less force, task two: grasp the object with moderate force, task three: the object is gripped with a large force. And taking the electromyographic features of each subtask and the corresponding joint angle signals as input to train LSTM estimation submodels respectively, namely three LSTM estimation submodels corresponding to three different levels of force. FIG. 2 shows a schematic diagram of a single LSTM deep learning network, wherein when each model is trained, the preprocessed myoelectric signals and joint angle signals are input into an input gate, a forgetting gate, an output gate and candidate states in the LSTM deep learning network, and output values of the candidate states of the LSTM deep learning network at time t are obtained
Figure BDA0002713820060000081
Output value i of input gatetOutput value f of forgetting gatetAnd the output value o of the output gatetWherein:
it＝σ(Wi·[ht-1,xt]+bi)
ft＝σ(Wf·[ht-1,xt]+bf)
ot＝σ(Wo·[ht-1,xt]+bo)
Figure BDA0002713820060000091
ht＝ot*tanh(Ct)
Figure BDA0002713820060000092
wherein x istThe electromyographic signals after being preprocessed represent the input of the model. WCAnd bCWeight matrix and bias, W, representing candidate states, respectivelyiAnd biRespectively representing the weight matrix and the offset, W, of the input gatefAnd bfWeight matrix and offset, W, representing forgetting gates, respectivelyoAnd boWeight matrix and offset, C, representing the output gates, respectivelyt、Ct-1Respectively representing the output of the cell state of the storage long-term memory at t and t-1,
Figure BDA0002713820060000093
Figure BDA0002713820060000094
respectively representing the outputs of the candidate states at times t and t-1, ht、ht-1Respectively representing the output of the LSTM deep learning network at t and t-1, wherein sigma (DEG) represents a sigmoid activation function, and tanh (DEG) represents a tanh activation function.
Initializing parameters of the LSTM deep learning network: weighting matrix W of candidate stateCAnd bias bCWeight matrix W of input gatesiAnd bias biForgetting gate weight matrix WfAnd bias bfWeight matrix W of output gatesoAnd bias boInitializing to a random number between 0 and 1. And setting the number of neurons in an input layer of the LSTM deep learning network as M, setting the number of layers as i, and using the output of each layer as the input of the next layer.
Training the LSTM network after the LSTM network initialization is completed: inputting the preprocessed electromyographic signals and joint angle signals into an LSTM model, optimizing the weight matrix and the bias by adopting a back propagation algorithm to obtain an LSTM training model, and performing the same operation on the three LSTM models to obtain three different LSTM estimation submodels. Setting the number of training iterations as epoch, loss function as loss, optimizer as optimizer, and batch size as the number of samples selected in one training. In this embodiment, M is 256, i is 2, epoch is 150, loss is mae (mean absolute error), optimizer is Adam (Adam optimizer), and batch size is 16.
(4) And training the RF switching model, namely taking the preprocessed electromyographic signals and the corresponding label of the LSTM model as model input to train the RF switching model, and taking the output of the RF switching model as the label of the LSTM estimation sub-model. The schematic diagram of the RF switching model is shown in fig. 3, bootstrap sampling is performed on input samples to obtain k sample sets, a decision tree model is used to train the k sample sets to obtain k tree classifiers, where the LSTM submodels are labeled with numbers 1-n (n LSTM estimation submodels are generated in total), the maximum depth of each tree classifier is set to m, a gini function calculation method is used to determine whether nodes are continuously split, the minimum number of samples required for splitting is set to f, the maximum number of leaf nodes is not limited, and the maximum number of features participating in determination when nodes are split is equal to all the feature numbers. In this embodiment, the sample set is divided into 10 sample sets according to bootstrap sampling, i.e., k is 10, there are three LSTM estimation submodels, i.e., n is 3, the maximum depth of each tree classifier is set to 2, i.e., m is 2, and the minimum number of samples required for splitting is set to 2, i.e., f is 2.
(5) After the RF switching model and each LSTM estimation sub-model are trained, the LSTM hybrid estimation model combined with the model switching mechanism is trained, and the model is used for carrying out composite motion continuous estimation on newly input electromyographic signal data under a complex task. In the estimation stage, the electromyographic features of all channels are normalized by using the maximum value and minimum value of the electromyographic features of all channels in the training stage, the normalized electromyographic features are input into an RF switching model, an LSTM estimation sub-model to which the corresponding electromyographic features belong is output, then the normalized electromyographic features are input into the corresponding LSTM estimation sub-model for estimation, and the estimation value of the angle of each joint is output.
The myoelectric compound motion continuous estimation effect of the LSTM mixed estimation model of the set model switching mechanism provided by the invention under a complex task is evaluated.
And evaluating the estimation accuracy of the single LSTM model in each estimation submodel by using the Pearson Correlation Coefficient (CC) and the Root Mean Square Error (RMSE). The estimation accuracy of the LSTM deep learning network is illustrated below by a single LSTM deep learning network constructed in the moderate force context of the present embodiment: the average CC is 0.944 and the average RMSE is 11.63 °. CC >0.8 indicates a strong correlation between the estimated angle and the measured angle, and RMSE indicates a root mean square error of 11.63 ° for the five finger angle, as shown in fig. 4.
Compared with the estimation effect of the single LSTM model and the LSTM hybrid estimation model combined with the model switching mechanism under the complex task, the result shows that the single LSTM deep learning network cannot adapt to the accurate estimation under the complex task, and the LSTM hybrid estimation model combined with the model switching mechanism can adapt to the scene change to realize the accurate estimation under the complex task. The estimation effect of the single LSTM and the LSTM hybrid estimation model combined with the model switching mechanism under the complex task of force variation in the embodiment is further described below.
The method comprises the steps of respectively carrying out cross estimation on the grasping motion under different force grade conditions by using a single LSTM submodel obtained by training under different force grade conditions, and comparing the cross estimation with the continuous estimation on the grasping motion under different force grade conditions by using an LSTM mixed estimation model combined with a model switching mechanism, wherein the estimation effects of the models are shown in the following table I and table II, wherein L, M, H models respectively represent LSTM submodels obtained by training with low-grade force, medium-grade force and high-grade force, and S models represent LSTM mixed estimation models combined with the model switching mechanism. L, M, H the force represents the input force at the time of estimation as low level force, medium level force, high level force, respectively. Wherein, the evaluation index of the pair of estimation results in the table is the Pearson Correlation Coefficient (CC), and the evaluation index of the pair of estimation results in the table is the Root Mean Square Error (RMSE).
TABLE-evaluation index of the estimation results is the result of Pearson's correlation coefficient
Figure BDA0002713820060000111
The evaluation index of the second estimation result is the result of the root mean square error
Figure BDA0002713820060000112
The first table and the second table show the estimation effect of each model on the forces of different levels, and it can be seen from the tables that the single LSTM model under each force level condition only has a good estimation effect on the force level condition under the model, and the estimation effect on the force conditions of other levels is poor. The LSTM hybrid estimation model combined with the model switching mechanism has a good estimation effect on different level force conditions, and a comparison result shows that the LSTM hybrid estimation model combined with the model switching mechanism has better robustness and estimation accuracy on complex tasks. A comparison graph of the estimation effect of the estimation model obtained by training under the low-level force condition and the estimation effect of the mixed estimation model of LSTM combined with the model switching mechanism on the grip under the three-level force condition is shown in fig. 5. As can be seen from fig. 5, the angle curve estimated by the LSTM hybrid estimation model combined with the model switching mechanism under the condition of mixing the three force levels fits the actually measured angle curve well, and the angle curve estimated by the single LSTM estimation model trained under the low-level force condition does not fit the actually measured angle curve seriously under the conditions of the medium-level force and the high-level force. Therefore, a single LSTM deep learning network cannot adapt to accurate estimation under a complex task, and an LSTM hybrid estimation model combined with a model switching mechanism can adapt to scene change to realize accurate estimation under the complex task.
The method can automatically switch under different scenes, and accurately estimate by using the LSTM estimation sub-model after determining the scenes, thereby ensuring the algorithm robustness under complex tasks and realizing the continuous estimation of the compound motion under the complex tasks based on the electromyographic signals.
In the invention, in order to accurately distinguish the training stage, the estimation stage and different estimation models, an LSTM deep learning network, an LSTM model and an LSTM submodel are used. Before an accurate estimation model is obtained, namely before all parameters of the model are trained, an LSTM deep learning network is used, the LSTM model refers to an estimation model after all parameters in the LSTM deep learning network are trained, an LSTM submodel refers to an estimation model respectively corresponding to different scenes in the same complex task, an RF switching model refers to an estimation model for accurately identifying the electromyographic features as a certain scene in the complex task by using an RF integrated learning algorithm, and the LSTM is an acronym of longShort-TermMemory. RF is an acronym for Random Forest, representing a Random Forest classifier.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (7)

1. A complex motion continuous estimation method based on electromyography model switching is characterized by comprising the following steps:
s1: collecting joint angle signals and multi-channel electromyographic signals according to the joint angle to be estimated and surface muscles driving corresponding joints to move, dividing a task into a plurality of subtasks according to the characteristics of a complex task, combining the plurality of subtasks into the whole complex task,
s2: respectively preprocessing the collected electromyographic signals and joint angle signals,
s3: utilizing the preprocessed electromyographic signals and joint angle signals, adopting an LSTM deep learning network to establish a mapping sub-relation of the electromyographic signals and the joint angle signals to obtain an estimation sub-model,
s4: establishing a mapping relation between the electromyographic signals and each estimation submodel by using an RF ensemble learning algorithm to obtain an RF switching model,
s5: and classifying the electromyographic signal characteristics by using an RF switching model, judging an estimation submodel to which the current movement belongs, and inputting the electromyographic characteristics into the corresponding estimation submodel to obtain the joint angle estimation value at the moment.
2. The method of claim 1, wherein the step S2 of pre-processing the EMG signal comprises filtering, feature extraction and normalization of the EMG signal,
in particular, the electromyographic signals are subjected to notch filtering processing to eliminate power frequency interference, band-pass filtering processing of 20 Hz-460 Hz is also carried out on the electromyographic signals to eliminate low-frequency noise and high-frequency noise,
extracting the feature of the electromyographic signal after noise reduction, adopting a sliding window method to extract the feature value of the electromyographic signal,
and after the electromyographic features are obtained, normalizing the electromyographic features of the channels according to the maximum value and the minimum value of the characteristic values of the channels respectively to obtain the electromyographic features after normalization of the channels.
3. The method as claimed in claim 2, wherein the step S2 of preprocessing the joint angle signal comprises filtering the measured joint angle signal and the estimated joint angle signal, respectively, so that the measured joint angle and the estimated joint angle are inputted and outputted to the corresponding model at a relatively smooth angle.
4. The method for continuous estimation of complex motion based on electromyography model switching as claimed in claim 3, wherein the estimation submodel obtained in step S3 is an LSTM estimation submodel, and the specific process for obtaining the LSTM estimation submodel is as follows:
firstly, initializing parameters of the LSTM deep learning network, and determining a weight matrix W of a candidate stateCAnd bias bCWeight matrix W of input gatesiAnd bias biForgetting gate weight matrix WfAnd bias bfWeight matrix W of output gatesoAnd bias boIs initialized to
Figure FDA0002713820050000021
The number of neurons in the input layer of the LSTM deep learning network is set to be M, the number of layers is set to be i, the output of each layer is used as the input of the next layer,
after the initialization of the LSTM deep learning network is completed, the LSTM deep learning network is trained, the preprocessed electromyographic signals and joint angle signals are input into the LSTM deep learning network, and the plurality of weight matrixes and the plurality of offsets are optimized by adopting a back propagation algorithm to obtain an LSTM estimation sub-model.
5. The method for continuous estimation of complex motion based on electromyography model switching as claimed in claim 4, wherein in step S5, a RF ensemble learning algorithm is used to establish a mapping relationship between the electromyography signal and each LSTM estimation submodel to obtain an RF switching model, specifically: training to obtain an RF switching model by taking the preprocessed electromyographic signals and the corresponding label of the LSTM model as input, wherein the output of the RF switching model is the label of the LSTM estimation sub-model, and the label of the LSTM model is a number
Figure FDA0002713820050000022
Figure FDA0002713820050000023
A total of n LSTM estimation submodels are generated.
6. The method for continuous estimation of complex motion based on electromyography model switching as claimed in claim 5, wherein the joints in the joint angle signals are human compound joints including joints of human hand and shoulder-elbow joints, and the multi-channel electromyography signals are electromyography signals generated by relevant surface muscles driving the compound joints to move.
7. The method for continuous estimation of complex motion based on electromyography model switching as claimed in claim 6, further comprising step S6, step S6 being: and adopting the Pearson correlation coefficient and the root mean square error as evaluation indexes of the actually measured joint angle and the estimated joint angle so as to evaluate the estimation precision of the estimated joint angle.
CN202011066204.7A 2020-09-30 2020-09-30 Complex motion continuous estimation method based on electromyography mapping model switching Active CN112232161B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011066204.7A CN112232161B (en) 2020-09-30 2020-09-30 Complex motion continuous estimation method based on electromyography mapping model switching

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011066204.7A CN112232161B (en) 2020-09-30 2020-09-30 Complex motion continuous estimation method based on electromyography mapping model switching

Publications (2)

Publication Number Publication Date
CN112232161A true CN112232161A (en) 2021-01-15
CN112232161B CN112232161B (en) 2023-04-07

Family

ID=74119337

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011066204.7A Active CN112232161B (en) 2020-09-30 2020-09-30 Complex motion continuous estimation method based on electromyography mapping model switching

Country Status (1)

Country Link
CN (1) CN112232161B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114055461A (en) * 2021-08-20 2022-02-18 华中科技大学 Robot force and position synchronous teleoperation control method and device based on myoelectric interface
CN114733160A (en) * 2022-04-14 2022-07-12 福州大学 Myoelectric signal-based muscle strength training equipment control method

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108874149A (en) * 2018-07-28 2018-11-23 华中科技大学 A method of continuously estimating human synovial angle based on surface electromyogram signal
CN109171707A (en) * 2018-10-24 2019-01-11 杭州电子科技大学 A kind of intelligent cardiac figure classification method
CN109262618A (en) * 2018-12-12 2019-01-25 武汉理工大学 Upper limb multi-joint synchronous proportional myoelectric control method and system based on muscle collaboration
CN109833031A (en) * 2019-03-12 2019-06-04 西安交通大学 It is a kind of that the sleep mode automatically method by stages of more physiological signals is utilized based on LSTM
CN110464348A (en) * 2019-07-10 2019-11-19 深圳市智能机器人研究院 The continuous amount of exercise recognition methods of joint of lower extremity and system based on electromyography signal
CN111079590A (en) * 2019-12-04 2020-04-28 东北大学 Peripheral vehicle behavior pre-judging method of unmanned vehicle
CN111134666A (en) * 2020-01-09 2020-05-12 中国科学院软件研究所 Emotion recognition method of multi-channel electroencephalogram data and electronic device

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108874149A (en) * 2018-07-28 2018-11-23 华中科技大学 A method of continuously estimating human synovial angle based on surface electromyogram signal
CN109171707A (en) * 2018-10-24 2019-01-11 杭州电子科技大学 A kind of intelligent cardiac figure classification method
CN109262618A (en) * 2018-12-12 2019-01-25 武汉理工大学 Upper limb multi-joint synchronous proportional myoelectric control method and system based on muscle collaboration
CN109833031A (en) * 2019-03-12 2019-06-04 西安交通大学 It is a kind of that the sleep mode automatically method by stages of more physiological signals is utilized based on LSTM
CN110464348A (en) * 2019-07-10 2019-11-19 深圳市智能机器人研究院 The continuous amount of exercise recognition methods of joint of lower extremity and system based on electromyography signal
CN111079590A (en) * 2019-12-04 2020-04-28 东北大学 Peripheral vehicle behavior pre-judging method of unmanned vehicle
CN111134666A (en) * 2020-01-09 2020-05-12 中国科学院软件研究所 Emotion recognition method of multi-channel electroencephalogram data and electronic device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
QIN ZHANG ET AL: ""Simultaneous and Proportional Estimation of Multijoint Kinematics From EMG Signals for Myocontrol of Robotic Hands"", 《IEEE/ASME TRANSACTIONS ON MECHATRONICS》 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114055461A (en) * 2021-08-20 2022-02-18 华中科技大学 Robot force and position synchronous teleoperation control method and device based on myoelectric interface
CN114055461B (en) * 2021-08-20 2023-02-14 华中科技大学 Robot force and position synchronous teleoperation control method and device based on myoelectric interface
CN114733160A (en) * 2022-04-14 2022-07-12 福州大学 Myoelectric signal-based muscle strength training equipment control method
CN114733160B (en) * 2022-04-14 2022-10-18 福州大学 Myoelectric signal-based muscle strength training equipment control method

Also Published As

Publication number Publication date
CN112232161B (en) 2023-04-07

Similar Documents

Publication Publication Date Title
Huang et al. Development of a myoelectric discrimination system for a multi-degree prosthetic hand
Arief et al. Comparison of five time series EMG features extractions using Myo Armband
CN112232161B (en) Complex motion continuous estimation method based on electromyography mapping model switching
CN106293057A (en) Gesture identification method based on BP neutral net
CN101564328B (en) Laptop artificial limb multi-movement-mode identifying method based on support vector data description
CN104537382A (en) Electromyographic signal gait recognition method for optimizing support vector machine based on genetic algorithm
CN107618018B (en) Manipulator action speed proportional control method based on myoelectricity
Guo et al. Long exposure convolutional memory network for accurate estimation of finger kinematics from surface electromyographic signals
CN110826625A (en) Finger gesture classification method based on surface electromyographic signals
CN108874149B (en) Method for continuously estimating human body joint angle based on surface electromyogram signal
Godoy et al. Electromyography-based, robust hand motion classification employing temporal multi-channel vision transformers
Nazemi et al. Artificial neural network classifier in comparison with LDA and LS-SVM classifiers to recognize 52 hand postures and movements
Liang et al. Research on recognition of nine kinds of fine gestures based on adaptive AdaBoost algorithm and multi-feature combination
CN115512440A (en) Cross-user gesture recognition method based on multi-source domain self-adaption
CN114159079B (en) Multi-type muscle fatigue detection method based on feature extraction and GRU deep learning model
Lin et al. A normalisation approach improves the performance of inter-subject sEMG-based hand gesture recognition with a ConvNet
CN112036357B (en) Upper limb action recognition method and system based on surface electromyographic signals
Zhao et al. EMG control for a five-fingered prosthetic hand based on wavelet transform and autoregressive model
Kukker et al. Neural reinforcement learning classifier for elbow, finger and hand movements
CN113951904A (en) sEMG-based method for continuously predicting knee joint angle of lower limb
CN109662710A (en) A kind of EMG Feature Extraction based on convolutional neural networks
Kan et al. EMG pattern recognition based on particle swarm optimization and recurrent neural network
CN111985270B (en) sEMG signal optimal channel selection method based on gradient lifting tree
CN111973388B (en) Hand rehabilitation robot control method based on sEMG
Tan et al. Electromyography based handwriting recognition system using LM-BP Neural Network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant