CN114063787B

CN114063787B - Deep learning processing analysis method based on EMG and IMU data

Info

Publication number: CN114063787B
Application number: CN202111396749.9A
Authority: CN
Inventors: 王鹤翔; 许德新; 张建强; 张原野; 冯帝
Original assignee: Harbin Engineering University
Current assignee: Harbin Engineering University
Priority date: 2021-11-23
Filing date: 2021-11-23
Publication date: 2023-09-19
Anticipated expiration: 2041-11-23
Also published as: CN114063787A

Abstract

The invention provides a deep learning processing analysis method based on EMG and IMU data, which comprises the following steps: obtaining EMG and IMU data signals corresponding to each gesture action; performing bias removal processing and wavelet noise reduction processing; filtering operation is carried out on ACC signals, and data segmentation operation is carried out on the processed signals; fusing the processed EMG signal and ACC signal data, sequentially storing gesture actions corresponding to each row of the data obtained after data fusion into a matrix in sequence, wherein the obtained matrix is in the shape of n rows and 1 column, and n is the number of rows of the data fusion array; normalizing the elements in the data fusion array; and selecting CNN as a recognition model of the data, and modifying the original CNN. After the operations are all completed, the data can be put into the model for training, and the data analyzed through the method is processed through experimental verification, so that the higher recognition rate is ensured on the premise of improving the recognition speed of the model.

Description

Deep learning processing analysis method based on EMG and IMU data

Technical Field

The invention belongs to the technical field of data processing, and particularly relates to a deep learning processing analysis method based on EMG and IMU data.

Background

In order to realize functions such as remote control, a machine system which is based on deep learning and can realize man-machine interaction through gesture recognition is designed, and the machine system has the characteristics of being novel, efficient, flexible, easy to use and the like. Can be applied to medical systems to assist the movement of patients, and is also beneficial to exploring areas where people are inconvenient to operate.

The core of gesture recognition is the selection and design of a gesture recognition model, and a plurality of existing model algorithms are used for recognition aiming at gesture recognition. For example, a network model algorithm such as LSTM, RNN, SVM can be selected for gesture recognition through electromyographic signals, and an image processing model such as CNN can be selected for classification through gesture recognition.

For the traditional text classification model, the processing time of the model such as RNN, LSTM and the like is long, which is unfavorable for the model to process real-time data. The recognition rate of the machine learning model SVM is lower than that of the deep neural network under the condition of processing a large amount of data, so that the problem of low recognition rate can be caused by selecting the SVM. Whereas CNN network models are generally applicable to image data, they have a lower recognition rate for recognizing text data than LSTM models.

Disclosure of Invention

Aiming at the technical problems, the invention provides a deep learning processing analysis method based on EMG and IMU data, so as to ensure higher recognition rate on the premise of improving the recognition speed of a model.

The image processing model CNN is selected to process text data, but if CNN data is directly selected, the problem of recognition rate is caused. Therefore, the method is improved on the traditional CNN model, and a new algorithm is introduced for improvement, so that the recognition rate of the model is improved under the condition of ensuring that the recognition speed is unchanged.

The specific technical scheme is as follows:

an EMG and IMU data-based deep learning processing analysis method comprises the following steps:

step 1: the wearable data acquisition device executes corresponding gesture actions to obtain EMG and IMU data signals corresponding to each gesture action;

step 2: performing unbiasing processing on the acquired EMG data signals, namely subtracting the average value of any 200 sampling points of the EMG signals when each section is in a resting state from the whole section of EMG signals;

step 3: the EMG signal after the offset removal treatment is subjected to wavelet noise reduction treatment, and the specific operation method comprises the following steps:

(3.1) performing discrete wavelet decomposition on the data; namely, selecting a proper wavelet base, and decomposing an EMG signal into 4 layers through discrete wavelet transformation; obtaining a low-frequency wavelet coefficient and a high-frequency wavelet coefficient from the data after discrete wavelet transformation processing; the discrete wavelet transform formula is as follows:

(3.2) selecting a threshold lambda; the threshold value can be determined by a fixed threshold formula:

wherein, media (|s (i) |) is the median of the wavelet coefficient absolute values;

(3.3) modifying the wavelet decomposition coefficients; if the absolute value of the wavelet decomposition coefficient is smaller than the threshold lambda, making the absolute value of the wavelet decomposition coefficient be 0; when the value is larger than lambda, the value is kept unchanged;

(3.4) performing discrete wavelet inverse transformation on the unmodified low-frequency wavelet coefficient and the modified high-frequency wavelet xishu1 to obtain a denoised signal;

step 4: filtering the ACC signal, namely replacing the maximum value and the minimum value of a certain piece of data with the average value of the maximum value and the minimum value of the certain piece of data;

step 5: performing data segmentation operation on the processed signals;

the specific operation method of the step is as follows:

(5.1) summing the three channel EMG signals to yield sum_emg:

wherein a (i) is the maximum value of the EMG signal of each channel;

(5.2) moving average processing is performed on the absolute value of the sum_emg signal to obtain S (k):

(5.3) establishing a comparison threshold th:

th＝μ+1.6σ

in the method, in the process of the invention, _μ is the mean value of SUM_EMG in the resting state; sigma is the standard deviation of SUM_EMG in a resting state;

(5.4) comparing S (k) with th, recording sequence numbers corresponding to sequences from the beginning of S (k) being greater than th to the beginning of S (k) being less than th, extracting the sequence, and successfully segmenting the EMG signals;

(5.5) dividing the ACC data according to the obtained EMG serial number to successfully divide the ACC signal;

step 6: fusing the processed EMG signal and ACC signal data, namely fusing the EMG signal and the ACC signal into a large array, wherein each row of the array represents a gesture action, and the first 60% of elements of each row are EMG signals and the second 30% are IMU signals;

step 7: sequentially storing gesture actions corresponding to each row of data obtained after data fusion into a matrix according to a sequence, wherein the obtained matrix is in the shape of n rows and 1 column, and n is the number of rows of the data fusion array;

step 8: normalizing the elements in the data fusion array to ensure that the absolute values of the elements are all numbers in the range of 0-1:

s in ₁ For the element value after normalization treatment, s is the untreated original element value, s _mean Is the mean value of s, s _var Is the variance of s;

step 9: selecting CNN as a recognition model of data, and modifying the original CNN as follows:

(1) Modifying the size of the convolution kernel, setting the height of the convolution kernel to be 1, and adapting the width of the convolution kernel to the size of specific input data;

(2) The number of layers of the network is properly increased according to the size of the data, and the method is suitable for stacking small convolution kernels to replace large convolution kernels so as to reduce parameters required by model training;

(3) A residual block is introduced, i.e. the input of a certain layer and the outputs of the following two layers are added together as the input of the latest layer.

After the operations are all completed, the data can be put into the model for training, and the data analyzed through the method is processed through experimental verification, so that the higher recognition rate is ensured on the premise of improving the recognition speed of the model.

Drawings

FIG. 1 is a schematic flow chart of the present invention;

FIG. 2 is a schematic diagram of the large array of step 6;

fig. 3 is a flowchart illustrating the modification of the original CNN in step 9.

Detailed Description

The specific technical scheme of the invention is described by combining the embodiments.

As shown in fig. 1, a deep learning processing analysis method based on EMG and IMU data includes the following steps:

step 5: performing data segmentation operation on the processed signals;

the specific operation method of the step is as follows:

(5.1) summing the three channel EMG signals to yield sum_emg:

wherein a (i) is the maximum value of the EMG signal of each channel;

(5.3) establishing a comparison threshold th:

th＝μ+1.6σ

step 6: fusing the processed EMG signal and ACC signal data, namely fusing the EMG signal and the ACC signal into a large array, wherein each row of the array represents a gesture action, and the first 60% of the elements of each row are EMG signals and the second 30% of the elements of each row are IMU signals as shown in FIG. 2;

step 9: CNN was selected as the recognition model for the data, as shown in fig. 3, and the following modifications were made to the original CNN:

In order to verify the actual effect of the model, 4 gesture actions of fist making, left swing, right swing and hand opening are respectively designed, and 4 healthy adults aged 19-21 are selected as experimental objects, wherein the experimental objects comprise two men and two women. In order to increase the collection quantity of data, each person wears the data collection device to execute specific gesture actions 150 times. Considering that repeated execution of a certain action causes muscle soreness and affects the recognition rate of gestures, the test subjects rest for 5 minutes every 10 actions to relieve the muscle soreness. Thus 600 sets of experimental data are available for each gesture. The obtained model data is put into the designed model after pretreatment such as noise reduction and segmentation. In order to verify that introducing a residual network does improve the overall performance of the model, these data are put into a common CNN network and a modified design network, respectively, with the following results:

it can be seen that the improved network is superior to the CNN network model, both in terms of identification rate and identification speed. And for the MYO arm ring, the identification rate and the stability of the invention are better.

The user firstly collects and executes data corresponding to different gestures through the myoelectric sensor and the IMU, the collected data firstly carries out data preprocessing and data segmentation operation, then carries out data processing operation, and after the processing is completed, EMG signal data and IMU signal data are fused together, so that the processing is completed. The improved model can be used for training the processed data by modifying the size of the CNN convolution kernel and introducing a residual block. Multiple experiments prove that the recognition rate of the model to the data is about 84%.

For data processing methods, there are many other functional formulas that can be implemented for noise reduction, filtering, feature segmentation, the key is the problem of selection and combination of these formulas, i.e. how to select the appropriate formula for each data processing step to optimize the overall process

For the selection aspect of the recognition model, other algorithms can be selected as classification recognition models, such as LSTM (long short term memory neural network), SVM (support vector machine) and KNN (K nearest neighbor classifier), but the recognition rate and the recognition speed are not as good as the models designed by adopting the project.

Claims

1. The deep learning processing analysis method based on the EMG and IMU data is characterized by comprising the following steps of:

step 3: performing wavelet noise reduction on the EMG signal subjected to the offset removal treatment;

step 5: performing data segmentation operation on the processed signals;

the specific operation method of the step 5 is as follows:

(5.1) summing the three channel EMG signals to yield sum_emg:

wherein a (i) is the maximum value of the EMG signal of each channel;

(5.3) establishing a comparison threshold th:

th＝μ+1.6σ

step 9: selecting CNN as a recognition model of the data, and modifying the original CNN;

in step 9, the original CNN is modified as follows:

2. The method for deep learning processing analysis based on EMG and IMU data according to claim 1, wherein the specific operation method in step 3 is as follows:

(3.4) performing inverse discrete wavelet transform on the unmodified low-frequency wavelet coefficient and the modified high-frequency wavelet xishu1 to obtain a denoised signal.