CN112370066A

CN112370066A - Brain-computer interface method of stroke rehabilitation system based on generation of countermeasure network

Info

Publication number: CN112370066A
Application number: CN202011056408.2A
Authority: CN
Inventors: 王卓峥; 马卓
Original assignee: Beijing University of Technology
Current assignee: Beijing University of Technology
Priority date: 2020-09-30
Filing date: 2020-09-30
Publication date: 2021-02-19

Abstract

The invention discloses a stroke rehabilitation system brain-computer interface method based on a generation countermeasure network. In the signal acquisition and preprocessing stage, the eye electricity and the muscle electricity are eliminated by using rapid independent component analysis and wavelet packet transformation. In the feature extraction stage, sample data is repeatedly selected for multiple times and divided into a plurality of small data segments, a part of electroencephalogram data samples of other people are introduced when the covariance matrix of a testee is calculated, and then the RCSP features are extracted by utilizing the covariance matrix so as to ensure the feature stability of the small samples. In the motor imagery classification stage, an EEGGANs learning model is designed, a generator of the EEGGANs learning model comprises four layers of transposed Convolution (CNN), a discriminator also comprises four layers of CNN, and an output layer is Softmax. In the accurate quantitative index evaluation stage, a rehabilitation doctor guides the treatment of the patient according to the mechanical movement position, angle and the like controlled by the motor imagery category. The invention can replace the damaged peripheral nerve and muscle access of stroke patients and realize the active rehabilitation treatment of patients with dyskinesia.

Description

Brain-computer interface method of stroke rehabilitation system based on generation of countermeasure network

Technical Field

The invention relates to a system capable of classifying and identifying motor imagery electroencephalogram signals, in particular to a brain-computer interface method which can help stroke patients with motor dysfunction to carry out rehabilitation training and has high precision and high robustness.

Background

Stroke is an acute cerebrovascular disease, which is a disease that causes brain tissue damage because blood cannot flow into the brain due to rupture or blockage of cerebral vessels. The cerebral apoplexy has the characteristics of high morbidity, high mortality and high disability rate, and investigation shows that the urban and rural total cerebral apoplexy becomes the first cause of death in China and is also the first cause of disability of adults in China.

Currently, from the analysis of treatment means, stroke rehabilitation systems at home and abroad mainly focus on clinical rehabilitation training systems. The clinical rehabilitation training system mostly adopts the rehabilitation training based on the basis of neurophysiology, mainly utilizes the common movement, the synergistic action, the posture reflex and other nerve movement mechanisms according to the movement development control principle and the brain plasticity principle, judges the functional state and the potential capability of a patient through the clinical rehabilitation evaluation of a doctor, and then performs the corresponding rehabilitation training by 'taking medicines for the disease'. At present, many methods for assessing the motor function of the cerebral apoplexy clinically are available, such as a simplified Fugl-Meyer motor function assessment method, a Brunnstrom grade assessment method and the like. The scale assessment methods all depend on examination and observation of doctors, belong to manual assessment, are widely used clinically, but assessment results are easily affected by subjective factors of rehabilitation doctors, and scale grading indexes are more, so that the rehabilitation doctors need to participate in the whole process, and limited doctors often feel uneasy in the face of huge patient groups, and even the optimal treatment time is delayed. The appearance of the brain-computer interface technology provides a feasible scheme for diagnosis and rehabilitation training of stroke patients with normal thinking but incomplete motor functions.

At present, a brain-computer interface of a stroke rehabilitation system generally comprises an electroencephalogram signal acquisition and preprocessing module, a feature extraction module and a motor imagery classification module. Currently, mainstream feature extraction methods such as principal component analysis, genetic algorithm, wavelet transformation and the like are not stable enough in small sample electroencephalogram data set, and high algorithm complexity is caused by high-dimensional features. Traditional classifiers such as support vector machines, naive bayes, decision trees, etc. require manual feature selection, which can result in accuracy deviation. Common neural networks such as a convolutional neural network and a cyclic neural network have poor classification effect aiming at the problems that the small sample data is poor in model generalization capability and easy to generate overfitting.

Disclosure of Invention

The invention mainly aims at the rehabilitation training of patients with motor dysfunction and cerebral apoplexy, and researches and realizes a brain-computer interface based on motor imagery electroencephalogram signals. In order to solve the problem of system stability caused by small sample data and sample imbalance, the method utilizes the covariance matrix to extract the RCSP characteristics so as to ensure the stability of the small sample characteristics. In order to solve the balance problem of recognition accuracy and algorithm complexity in a real-time feedback system, the invention provides a method for generating confrontation network EEGGANs, and establishing a semi-supervised learning model for a brain-computer interface of a stroke rehabilitation system, so that the brain-computer interface realizes accurate classification aiming at motor imagery electroencephalogram signals on the premise of ensuring small samples and high robustness.

The technical scheme adopted by the invention is a stroke rehabilitation system brain-computer interface method based on a generation countermeasure network, and the technical scheme adopted for solving the technical problem comprises the following steps:

step 1: preprocessing an EEG signal, namely firstly performing Wavelet Packet Transform (WPT) on an original EEG, determining the number of decomposition layers of the WPT, selecting a proper wavelet basis function according to the characteristics of the EEG and noise, finally determining a frequency band where high-frequency noise is to be filtered, and zeroing the corresponding frequency band. And then, performing fast independent component analysis (FastICA) and inverse FastICA on the signal subjected to high-frequency noise filtering to remove electro-ocular and electromyographic interference so as to obtain an electroencephalogram signal subjected to noise filtering. And then, acquiring the frequency spectrum of the electroencephalogram signal by utilizing Fast Fourier Transform (FFT) to obtain the power spectral density of the electroencephalogram signal.

Step 2: extracting features of the EEG signal, repeatedly selecting sample data for many times by means of the thought of blocking (patches) in sparse representation, dividing the sample data into a plurality of small data segments, and countingWhen the covariance matrix of the testee is calculated, a part of electroencephalogram data samples of other people are introduced, and then the covariance matrix is used for extracting RCSP characteristics. Matrix D for electroencephalogram signal extracted by single training_N×TWhere N represents the number of channels and T represents the number of sample points per channel over a period of time. And calculating the average covariance through the normalized covariance matrix, then calculating the normalized average spatial covariance matrix and decomposing the normalized average spatial covariance matrix, and finally solving the classification characteristic vector.

And step 3: a network structure is designed to classify the electroencephalogram signals, a convolutional neural network CNN is introduced to generate confrontation networks GANs, and the new network is named as EEGGANs. Wherein the generated network G is of a four-layer structure, random noise vectors (1 × 100) are subjected to four-dimensional reshaping by using fractional-distorted convolution layers (Deconv), and then are sent to a generator to be gradually upsampled to a pseudo sample X_fake(64 × 64 × 1) and adopts ReLU as the activation function. The discrimination network D is also a 4-layer CNN with BN (except for an input layer), and is different from the generation network G in that the first three layers are activated by adopting Leaky ReLU, and the last layer uses a Sigmoid activation function to enable a discriminator to output probability.

The EEGGANs network training data designed by the invention needs to be normalized, so that an Adaptive normalization (Adaptive Norm) method is provided to improve the generalization capability of the model and avoid the problem of local replacement of the whole body caused by the adoption of the same normalization method for solving different problems. Unlike reinforcement learning, the AN determines the appropriate normalization operation for each normalization layer in a deep network using differentiable learning.

Drawings

FIG. 1 general technical route chart of brain-computer interface

FIG. 2 electroencephalogram signal feature extraction method based on motor imagery

FIG. 3EEGGANs semi-supervised network learning architecture

FIG. 4EEGGANs Generation network architecture

Detailed Description

In fig. 1, EEG signals generated when a stroke patient is subjected to rehabilitation for dysfunction under the direction of a physician are used as system inputs. In the system training stage, real patient EEG data (with or without labels) and a labeled BCI competition standard motor imagery EEG signal data set form a training sample together after signal preprocessing, feature extraction and dimension reduction operation, and the training sample is used as the input of a semi-supervised learning model EEGGANs classifier. Under the action of joint training and synchronous enhancement of the generation network G and the discrimination network D, a stable state is achieved. The output is the motor imagery category of the stroke patient, the motor imagery category is transmitted to a BCI hardware interface through a control instruction API to drive mechanical auxiliary rehabilitation equipment (such as a mechanical artificial limb), and then the quantitative evaluation indexes of the mechanical movement position, angle and the like are used as important references of a rehabilitation doctor to guide the treatment of the stroke patient.

In the embodiment shown in FIG. 2, EEG data samples collected in accordance with the International 10-20 Standard in multiple channels are first preprocessed using WTP and FastICA to remove electro-ocular and electro-muscular interference; and then, obtaining the frequency spectrum of the electroencephalogram signal by using FFT (fast Fourier transform), and obtaining the power spectral density of the electroencephalogram signal. Performing decomposition segmentation on the power spectral density, repeatedly selecting sample data for multiple times, dividing the sample data into a plurality of small data segments, introducing a part of electroencephalogram data samples of other people when calculating the covariance matrix of a testee, and then extracting RCSP characteristics by using the covariance matrix. Matrix D for electroencephalogram signal extracted by single training_N×TWhere N represents the number of channels and T represents the number of sample points per channel over a period of time. By means of the normalized covariance matrix C_N×NThe mean covariance is calculated.

Where M represents the number of training times and Δ is the motor imagery signal class. Introducing a deviation parameter beta (beta is more than or equal to 0) for controlling the weight of the covariance matrix of the training sample and a deviation parameter gamma (gamma is less than or equal to 1) for adjusting the weight of a plurality of unit matrixes, enabling a classification result to be independent of a sampling sample through an offset estimation item, and calculating a regularized average spatial covariance matrix.

Wherein I represents an identity matrix, X_Δ(β) represents the covariance matrix of the current subject and the other subjects introduced, defined as follows:

sum of regularized mean spatial covariance matrices ∑ S_Δ(beta, gamma) decomposition into a form EVE of an eigenvalue matrix E and an eigenvector matrix V^TAnd further constructing a whitening matrix P such that S of each class_Δ(β, γ) have the same eigenvectors, and the sum of the corresponding eigenvalues is a fixed constant c,

P·∑S_Δ(β,γ)·P^T＝c·I

according to different classes S_ΔThe same eigenvalue matrix U of (β, γ), the projection matrix W becomes U^TAnd P. Mapping a training sample by selecting r columns before and after the projection matrix W, wherein Z is W.D_N×T. The finally extracted classification feature vector y is:

the extracted classification feature vector needs to be subjected to dimension reduction due to large data volume, and then the category of the motor imagery electroencephalogram signal can be output by a feature classification method to be used by a training set or a test set.

In the embodiment shown in fig. 3, the network structure is such that the decision network D is used as a classifier and the generation network G is used to generate pseudo samples X from random noise_fakeThe training set contains a labeled sample X_labelUnlabeled sample X_unlabelAnd a dummy sample X_fake. Wherein X_labelComprises a BCI competition data set and an EEG data set with a label of a real patient; x_unlabelA real EEG data set representing a patient under-fit or a physician unlabeled category. Classifier accepts samples

For the K classification problem, outputting K +1 dimensional estimation, and obtaining a probability set P through a Softmax function: the first K dimension corresponds to the original motor imagery class, and the last one dimension corresponds to the 'pseudo sample' class, p_iCorresponds to the class estimation label y_i. The optimization function of the system is as follows.

For accurately calculating the class output of the motor imagery, three loss functions are defined, and for labeled samples X in a training set_labelCalculating the probability of whether the estimated label is correct, and recording as L_label。

For unlabeled sample X in training set_unlabelWhether the estimated value is 'true' or not is examined, namely, the probability that the estimated value is not K +1 class is calculated and is marked as L_unlabel。

For the pseudo sample X generated by the generator_unlabelWhether or not to estimate as "false" is examined. I.e. calculating the probability of estimating as K +1 class, denoted as L_fake。

In the embodiment shown in fig. 4, the generation network G of the EEGGANs has a 4-layer structure, and the random noise vector (1 × 100) is four-dimensionally reshaped by using a micro-step transposed convolution layer (Deconv), and then is gradually upsampled to the pseudo sample X_unlabel(64X 1). Layer I firstly flattens data intoAnd (1 × 32768) vectors are activated by adopting a modified linear unit ReLU function, so that the trained network has appropriate sparsity, the problem of gradient disappearance possibly generated by the traditional activation function in the process of adjusting and optimizing the back propagation parameters is solved, and the convergence of the network is accelerated. After the adaptation, the data was reshaped to have a shape of (8 × 8 × 0512). Layer ii performs a transposition convolution operation with a step size of 2 using a convolution kernel of a size of 5 × 5, and then performs a ReLU activation, and the shape of the data output after adaptation is (16 × 16 × 256). Layer III and Layer II have the same structure, and the shape of data output by Layer III is (32X 128). Layer IV firstly uses convolution kernel with the size of 5 multiplied by 5 to carry out transposition convolution operation with the step of 2, then the depth of the Layer IV is reduced from 512 to 64, and then the Layer IV is activated by using a hyperbolic tangent function Tanh, so that the numerical value is compressed between-1 and 1. Outputting a (64 × 64 × 64) tensor, reducing the dimension to 1 by RPCA, and finally outputting a pseudo sample X_unlabel(64×64×1)。

The invention has the beneficial effects that: the method can realize the classification effect of high precision and high robustness on high-dimensional small sample electroencephalogram data, construct a brain-computer interface of a stroke rehabilitation system aiming at limb, visual, language and cognitive dysfunction, and help a patient with an impaired channel and unable to normally output to complete the active rehabilitation training of functional, visual, language and cognitive dysfunction, thereby improving motor function, recovering normal human body function, and solving the problems of low precision of magnitude indexes and untimely treatment caused by manual evaluation.

Claims

1. A stroke rehabilitation system brain-computer interface method based on a generation countermeasure network is characterized by comprising the following steps: the method comprises the following steps:

step 1: preprocessing the electroencephalogram signal to obtain the electroencephalogram signal after noise is filtered;

step 2: extracting RCSP characteristics by using a covariance matrix by means of a blocking idea;

and step 3: and adding the convolutional neural network into the generated countermeasure network to obtain an EEGGANs network model, and classifying the electroencephalogram signals.

2. The brain-computer interface method of stroke rehabilitation system based on generation of countermeasure network of claim 1, wherein: preprocessing an electroencephalogram signal, comprising three processes of wavelet packet transformation, fast independent component analysis and fast Fourier transformation: firstly, WPT is carried out on an original EEG, the number of decomposition layers of the WPT is determined, a proper wavelet basis function is selected according to the characteristics of the EEG and noise, finally, a frequency band where high-frequency noise is to be filtered is determined, and the corresponding frequency band is set to be zero; performing FastICA transformation and FastICA inverse transformation on the signal subjected to high-frequency noise filtering to obtain an electroencephalogram signal subjected to noise filtering; and finally, carrying out FFT (fast Fourier transform) on the denoised electroencephalogram signal to obtain a frequency spectrum of the electroencephalogram signal and calculating the power spectral density.

3. The brain-computer interface method of stroke rehabilitation system based on generation of countermeasure network of claim 1, wherein: extracting RCSP characteristics by using a covariance matrix; matrix D for electroencephalogram signal extracted by single training_N×TRepresenting, wherein N represents the number of channels, and T represents the number of sampling points of each channel in a period of time; by means of the normalized covariance matrix C_N×NCalculating an average covariance;

wherein M represents the training times, and delta is the motor imagery signal category; introducing a deviation parameter beta (beta is more than or equal to 0) for controlling the weight of the covariance matrix of the training sample and a deviation parameter gamma (gamma is less than or equal to 1) for adjusting the weight of a plurality of unit matrixes, enabling a classification result to be independent of a sampling sample through an offset estimation item, and calculating a regularized mean space covariance matrix;

P·∑S_Δ(β,γ)·P^T＝c·I

according to different classes S_ΔThe same eigenvalue matrix U of (β, γ), the projection matrix W becomes U^TP; mapping a training sample by selecting r columns before and after the projection matrix W, wherein Z is W.D_N×T(ii) a The finally extracted classification feature vector y is:

4. the brain-computer interface method of stroke rehabilitation system based on generation countermeasure network as claimed in claim 1, wherein the convolutional neural network is added to the generation countermeasure network to improve the robustness and classification accuracy of the model; the generation network G of EEGGANs is of four-layer structure, and random noise vector (1 × 100) is subjected to four-dimensional reshaping by using micro-step transposition convolution layer, namely Deconv, and then is sent into a generator to be gradually up-sampled to a pseudo sample X_fake(64 × 64 × 1); the corrected linear unit is used as an activation function, so that the trained network has appropriate sparsity, the problem of gradient disappearance possibly generated by the traditional activation function in the process of adjusting and optimizing the back propagation parameters is solved, and network convergence is accelerated; each upsampling layer uses a convolution kernel with the size of 5 multiplied by 5 to carry out transposition convolution operation with the stride of 2; the depth of the artificial tooth is gradually reduced from 512 to 64, and then the dimension is reduced to 1 by RPCA; outputting a 64 multiplied by 64 tensor by the last layer through RPCA dimension reduction output and compressing the numerical value between-1 and 1 by using a hyperbolic tangent function Tanh; the discriminating network D is also a network with4 layers of CNN are normalized in batches, and LeakyReLU is adopted for activation; the last layer is classified using the Softmax function.

5. The stroke rehabilitation system brain-computer interface method based on generation of the countermeasure network according to claim 4, characterized in that: in EEGGANs network, normalization processing is needed during data training, a self-adaptive normalization method is provided to improve the generalization capability of the model, and AN determines normalization operation for each normalization layer in a deep network by using differentiable learning.