CN117520891A - Motor imagery electroencephalogram signal classification method and system - Google Patents

Motor imagery electroencephalogram signal classification method and system Download PDF

Info

Publication number
CN117520891A
CN117520891A CN202311487339.4A CN202311487339A CN117520891A CN 117520891 A CN117520891 A CN 117520891A CN 202311487339 A CN202311487339 A CN 202311487339A CN 117520891 A CN117520891 A CN 117520891A
Authority
CN
China
Prior art keywords
classification
eegnet
loss
source domain
domain data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311487339.4A
Other languages
Chinese (zh)
Inventor
魏爱荣
杨浩田
彭福来
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong University
Original Assignee
Shandong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong University filed Critical Shandong University
Priority to CN202311487339.4A priority Critical patent/CN117520891A/en
Publication of CN117520891A publication Critical patent/CN117520891A/en
Pending legal-status Critical Current

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A61B5/372—Analysis of electroencephalograms
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/10—Pre-processing; Data cleansing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/096—Transfer learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2123/00—Data types
    • G06F2123/02—Data types in the time domain, e.g. time-series data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/02—Preprocessing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/08—Feature extraction
    • G06F2218/10—Feature extraction by analysing the shape of a waveform, e.g. extracting parameters relating to peaks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/12—Classification; Matching

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • General Physics & Mathematics (AREA)
  • Molecular Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mathematical Physics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Software Systems (AREA)
  • Animal Behavior & Ethology (AREA)
  • Surgery (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Medical Informatics (AREA)
  • Psychiatry (AREA)
  • Pathology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Psychology (AREA)
  • Signal Processing (AREA)
  • Physiology (AREA)
  • Fuzzy Systems (AREA)
  • Dermatology (AREA)
  • Neurology (AREA)
  • Neurosurgery (AREA)
  • Human Computer Interaction (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a motor imagery electroencephalogram signal classification method and a motor imagery electroencephalogram signal classification system, which relate to the fields of artificial intelligence and pattern recognition, wherein an EEGNet structure is Improved to obtain an I mproved-EEGNet model, then correlation alignment loss of source domain features and target domain features is introduced behind a full-connection layer of the Improved-EEGNet model, classification models are trained by combining classification loss of source domain data, offset between the source domain features and the target domain features is reduced, and the target domain is helped to train the classification models by utilizing source domain data labels; aiming at the problems of long training time and poor cross-test classification effect in the field of motor imagery electroencephalogram signal classification, the invention provides a migration learning algorithm based on an Improved-EEGNet model, and a reliable classification model is trained by utilizing source domain information to help a target domain.

Description

Motor imagery electroencephalogram signal classification method and system
Technical Field
The invention belongs to the field of artificial intelligence and pattern recognition, and particularly relates to a motor imagery electroencephalogram signal classification method and system.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
Brain-computer interface (Brain-Computer Interface, BCI) is a communication system established between the human Brain and external devices, which can collect, analyze and recognize nerve signals of the human Brain, and convert them into control commands to control the external devices such as manipulators, wheelchairs and the like to move. With the development of computer technology, signal processing, artificial intelligence, and other subjects, brain-computer interfaces have been widely used in a plurality of fields such as medical rehabilitation, fatigue detection, military defense, and the like.
The complete BCI system comprises five parts of electroencephalogram signal acquisition, preprocessing, feature extraction, mode identification and external control equipment, wherein non-invasive equipment is widely used in the electroencephalogram signal acquisition equipment; the preprocessing part can remove some interference and noise signals and improve the signal-to-noise ratio; the feature extraction algorithm comprises a co-space mode algorithm, wavelet transformation, fuzzy entropy and the like; training of the classifier can be performed by utilizing the feature vector and the sample label, and the common classifier comprises a support vector machine, linear discriminant analysis, a decision tree and the like; and then converting the result obtained by pattern recognition into a control instruction to control the external equipment to move.
The motor imagery is a widely used BCI paradigm, when a subject imagines to perform or actually performs a certain action, the region corresponding to the cerebral cortex is activated, an Event-related desynchronization (Event-Related Desynchronization, ERD) phenomenon is generated, and a region not performing motor imagery generates an Event-related desynchronization (Event-related Synchronization, ERS) phenomenon. The ERD/ERS phenomenon mainly occurs in the μ -rhythm and β -rhythm of motor imagery brain electrical signals.
At present, a plurality of mature algorithms can accurately decode motor imagery electroencephalogram signals, and most of the algorithms are trained by adopting a supervised learning method; because of characteristics of non-stationarity, randomness and the like of the brain-computer signal, different tested brain-computer data or the same tested brain-computer data in different periods have great difference, a new user needs to perform a training task for a long time before using a brain-computer interface to establish a reliable classification model, which is a huge burden for the user, especially for a patient needing medical rehabilitation.
Therefore, the existing motor imagery electroencephalogram signal classification method requires a long training process and a large number of training samples, is not suitable for patients needing medical rehabilitation, and has a limited adaptation surface.
Disclosure of Invention
Aiming at the problems of long training time and poor cross-test classification effect in the field of motor imagery electroencephalogram classification, the invention provides a motor imagery electroencephalogram classification method and a motor imagery electroencephalogram classification system, and designs a migration learning algorithm based on an Improved-EEGNet model to realize effective migration of source domain information, so that a reliable classification model is trained by utilizing the source domain information to help a target domain, the effects of reducing the number of samples to be trained in the target domain and shortening the time to be trained in the target domain are achieved, and the algorithm is named as Improved-EEGNet-transfer.
To achieve the above object, one or more embodiments of the present invention provide the following technical solutions:
the first aspect of the invention provides a motor imagery electroencephalogram signal classification method.
A motor imagery electroencephalogram signal classification method comprises the following steps:
acquiring target domain motor imagery electroencephalogram signals to be classified;
classifying the preprocessed target domain motor imagery electroencephalogram signals by using a classification model trained by an Improved-EEGNet-transfer algorithm to obtain and output classification results;
the classification model is obtained by adding a time convolution layer and a space convolution layer on an EEGNet neural network, namely an Improved-EEGNet model, wherein the Improved-EEGNet-transfer algorithm introduces correlation alignment loss of source domain data and target domain data, the classification model is trained jointly by utilizing the correlation alignment loss and the classification loss of the source domain data, the characteristic with good separability is formed, the offset between the source domain characteristic and the target domain characteristic is reduced, and the classification model is trained by utilizing the source domain data label to help the target domain.
Further, the preprocessing includes bandpass filtering the electroencephalogram signal using a bandpass filter and removing artifacts of the electroencephalogram signal with a common average reference.
Further, a time convolution layer and a space convolution layer are added on the EEGNet neural network, specifically: and improving one time convolution layer and one space convolution layer of the EEGNet neural network into two time convolution layers and two space convolution layers.
Further, the correlation alignment loss is defined as a distance between second order statistics of the source domain features and the target domain features:
wherein,represents the square of the matrix Frobenius norm, C S And C T The feature covariance matrix representing the source and target domains, d representing the number of dimensions of the features of the source and target domains.
Further, the correlation alignment loss, relative to the gradient of the input features, is calculated using the chain law as follows:
wherein,and->Is source domain data D T And destination domain data D T Through a mapping function ψ (D S ) And psi (D) T ) The resulting d-dimensional features, ψ represents the total representation of all feature extraction layers in the Improved-EEGNet, +.>The j-th dimension representing the i-th source domain data,>the j-th dimension, C, representing the i-th target domain data S And C T A feature covariance matrix representing the source domain and the target domain.
Further, the Improved-EEGNet-transfer algorithm uses a correlation alignment loss and classification loss combined training classification model of source domain data, and the loss function is as follows:
L＝L CLASS +λL CORAL
wherein L is CLASS Representing the classification loss of source domain data, L CORAL Representing the correlation alignment loss, λ is an adjustable parameter that adjusts the importance of the classification loss and the correlation alignment loss.
Further, the Improved-EEGNet-transfer algorithm is based on a source domain data set D S And a target domain data set D T The following was carried outIterative learning:
from source domain dataset D S Target domain dataset D T Respectively randomly acquiring a preset number of source domain data and target domain data, wherein a source domain data set D S Composed of labeled source domain data, target domain data set D T Consists of target domain data without labels;
inputting the labeled source domain data into a classification model, calculating classification loss, and obtaining source domain characteristics;
inputting target domain data without labels into a classification model to obtain target domain characteristics, and calculating correlation alignment loss by combining source domain characteristics;
based on the classification loss and the correlation alignment loss, a total loss is calculated, a back propagation algorithm is used to calculate the gradient and update the parameters of the model until the number of iterations is reached.
The second aspect of the invention provides a motor imagery electroencephalogram signal classification system.
A motor imagery electroencephalogram signal classification system comprises an acquisition module and a classification module:
an acquisition module configured to: acquiring target domain motor imagery electroencephalogram signals to be classified;
a classification module configured to: classifying the preprocessed target domain motor imagery electroencephalogram signals by using a classification model trained by an Improved-EEGNet-transfer algorithm to obtain and output classification results;
the classification model is obtained by adding a time convolution layer and a space convolution layer on an EEGNet neural network, namely an Improved-EEGNet model, wherein the Improved-EEGNet-transfer algorithm introduces correlation alignment loss of source domain data and target domain data, and the classification model is trained by combining the correlation alignment loss and the classification loss of the source domain data to form a characteristic with good separability, reduce the offset between the source domain characteristic and the target domain characteristic and further help the target domain to train the classification model by using a source domain data label.
A third aspect of the present invention provides a computer-readable storage medium having stored thereon a program which, when executed by a processor, implements steps in a motor imagery electroencephalogram classification method according to the first aspect of the present invention.
A fourth aspect of the present invention provides an electronic device comprising a memory, a processor and a program stored on the memory and executable on the processor, the processor implementing the steps in a motor imagery electroencephalogram classification method according to the first aspect of the present invention when the program is executed.
The one or more of the above technical solutions have the following beneficial effects:
the invention provides an Improved-EEGNet-based migration learning algorithm for realizing effective migration of source domain information, so that a reliable classification model is trained by utilizing the source domain information to help a target domain; the algorithm optimizes the EEGNet model, and adds a time convolution layer and a space convolution layer on the basis of the EEGNet model to obtain an Improved-EEGNet model, so that more complex features can be extracted; and then designing a transfer learning algorithm on the basis of an Improved-EEGNet model, specifically, adding correlation alignment loss of the source domain features and the target domain features behind a full-connection layer of the Improved-EEGNet model, and performing joint training on second order statistics of Ji Yuanyu data and target domain data deep features by combining the correlation alignment loss with classification loss of the source domain data, wherein the final features have good separability and can be well applied to the target domain, so that the accuracy of the motor imagery electroencephalogram signal crossing the tested classification is Improved.
Additional aspects of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention.
Fig. 1 is a flow chart of a method of a first embodiment.
FIG. 2 is a graph of 8-30Hz band pass filtering used during the pretreatment of the first embodiment;
FIG. 3 is a network configuration diagram of the EEGNet of the first embodiment;
FIG. 4 is a diagram showing the comparison of EEGNet and Improved-EEGNet frameworks according to the first embodiment;
FIG. 5 is a schematic diagram showing the structure and parameter sharing of the Improved-EEGNet model according to the first embodiment;
FIG. 6 is a histogram of classification accuracy for EEGNet-not-plus-transfer learning in the first embodiment;
FIG. 7 is a histogram of classification accuracy for the first embodiment Improved-EEGNet with no-migration learning;
FIG. 8 is a graph showing the accuracy of the three algorithms of the first embodiment across the categories under test;
Detailed Description
It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present application. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the present application. As used herein, the singular is also intended to include the plural unless the context clearly indicates otherwise, and furthermore, it is to be understood that the terms "comprises" and/or "comprising" when used in this specification are taken to specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof.
Example 1
How to reduce the number of samples required for training the BCI system, shorten the training time of the BCI system, become an important research direction, and the migration learning can effectively solve the problems, and can help the target domain to train the classification model by utilizing the knowledge of the source domain; the embodiment designs a migration learning algorithm based on an Improved-EEGNet model to realize effective migration of source domain information, and names the algorithm as an Improved-EEGNet-transfer algorithm, so that a reliable classification model is trained by utilizing the source domain information to help a target domain, as shown in fig. 1, and the motor imagery electroencephalogram signal classification method based on the classification model comprises the following steps:
step S1: acquiring target domain motor imagery electroencephalogram signals to be classified;
step S2: classifying the preprocessed target domain motor imagery electroencephalogram signals by using a classification model trained by an Improved-EEGNet-transfer algorithm to obtain and output classification results;
the classification model is obtained by adding a time convolution layer and a space convolution layer on an EEGNet neural network, namely an Improved-EEGNet model, wherein the Improved-EEGNet-transfer algorithm introduces CORAL (CORrelation ALignment) loss of source domain data and target domain data, namely correlation alignment loss, and the classification model is jointly trained by utilizing CORAL loss and classification loss of the source domain data to form a characteristic with good separability, reduces the offset between the source domain characteristic and the target domain characteristic, and further utilizes a source domain data label to help the target domain to train the classification model.
In order to solve the problems of long training time and poor cross-test classification effect in the BCI field, the embodiment designs a single-source-domain unsupervised migration learning algorithm based on an Improved-EEGNet model, which is used for training a classification model with good cross-test classification performance, and firstly, aiming at the problem of poor feature extraction capability of EEGNet time domain and space domain, a time convolution layer and a space convolution layer are added on the basis of the EEGNet model to obtain the Improved-EEGNet model; secondly, calculating CORAL losses of the source domain features and the target domain features behind a full-connection layer of the Improved-EEGNet model, and jointly training the CORAL losses and the classification losses of the source domain data to reduce the offset between the source domain features and the target domain features, so that the source domain labels are effectively utilized to train the model; finally, the trained model is applied to a target domain test set to obtain the classification accuracy of the model on the target domain test set, and the specific steps are as follows:
step 1: two types of motor imagery electroencephalogram for source domain and target domainPreprocessing the signal to obtain preprocessed source domain data D S And destination domain data D T 。
The data set used in this example is BCI Competition III DataSets IVa, the data set is from five healthy subjects aa, al, av, aw and ay, the collected effective electroencephalogram signals are 118 channels, the electrode positions are placed according to the extended international 10-20 standard, the sampling rate is 1000Hz, and the data set also provides the original electroencephalogram data downsampled to 100 Hz; the embodiment researches that the electroencephalogram signals of the right-hand and right-foot classification tasks provided by the data set are analyzed by selecting 64 channels in 118 channels in order to reduce the complexity of calculation;
the preprocessing process is band-pass filtering firstly, the data set used in the embodiment is an electroencephalogram signal acquired based on a motor imagery paradigm, ERD/ERS phenomena of a cerebral cortex corresponding area related to motor imagery mainly occur in a mu rhythm and a beta rhythm of the electroencephalogram signal, corresponding frequency bands are 8-12Hz and 13-28Hz respectively; secondly, because the electroencephalogram signal is weak in amplitude and easy to interfere, and the actions such as blinking and movement of a tested person inevitably occur in the acquisition process, artifacts such as electrooculogram and myoelectricity are often doped in the electroencephalogram signal, and the artifacts of the electroencephalogram signal are removed by using a common average reference in the preprocessing process.
Step 2: and improving the EEGNet network structure.
The invention improves the EEGNet structure, adds a time convolution layer and a space convolution layer on the basis of the EEGNet model, and obtains an Improved-EEGNet model, aiming at extracting deeper, more comprehensive and more complex features and improving the complexity and fitting capacity of the model.
EEGNet is a compact convolutional neural network suitable for various BCI (binary-coded decimal) modes, and because the EEGNet model has a simple structure and poor characteristic extraction capability in time domain and space domain, the EEGNet model is improved firstly, and aims to extract more comprehensive and complex characteristics. The specific structure of EEGNet is shown in FIG. 3, the Input of EEGNet network is motor imagery electroencephalogram with dimension of N×T, where N is the number of channels of electroencephalogram, and T is the number of sampling points.
The first layer is a two-dimensional convolution layer Conv2D, i.e. a temporal convolution layer, the size of the convolution kernel is 1× (f s 2), wherein f s For the sampling frequency of the electroencephalogram signal, the two-dimensional convolution layer carries out row-by-row time convolution on the original electroencephalogram signal, and the two-dimensional convolution layer is shown in the following formula:
y j ＝f((X*W j )+b j )
wherein y is j Represents the j-th feature map, f (·) represents the activation function, X represents the input signal, W j A parameter matrix representing the jth convolution kernel, b j Representing the deviation value of the jth convolution kernel. Here, an ELU activation function is used.
The second layer is a depth convolution layer DepthwiseConv2D, which is a spatial convolution layer, with a convolution kernel size of Nx1, as shown in the following formula:
y j,h ＝f(∑ j (X j *W j,h )+b j,h )
where h represents the depth of the depth convolution, f (·) represents the activation function, y j,h Represents the output of depth h corresponding to the jth convolution kernel, X j Represents the j-th feature map of the input, W j,h A parameter representing the jth convolution kernel of depth h, b j,h Representing the deviation value of the j-th convolution kernel of depth h.
The middle layer is a depth separation convolution, the layer has two layers of convolution processes, namely a depth convolution process and a point convolution process, the feature map output by the previous layer is subjected to the depth convolution process at first, the time domain feature is further extracted, and then the feature map is subjected to the point convolution process shown in the following formula:
y j ＝f((∑ i (X i *W j )+b j ))
the last layer is an output layer, and mainly performs the operation of full connection and output classification.
The invention improves the structure of the EEGNet model, adds a time convolution layer and a space convolution layer on the basis of the EEGNet model to obtain an Improved-EEGNet model, and improves one time convolution layer and one space convolution layer into two time convolution layers and two space convolution layers as shown in figure 4 by the framework of the EEGNet model and the framework pair of the Improved-EEGNet model, so as to extract deeper, more comprehensive and more complex characteristics and improve the complexity and fitting capability of the model.
Step 3: and (3) calculating CORAL losses of the source domain characteristics and the target domain characteristics after the fully connected layer of the Improved-EEGNet neural network, and carrying out weighted summation on the CORAL losses and the classification losses of the source domain data to obtain joint losses.
Firstly, defining CORAL loss between two domains on a single feature layer, and realizing alignment of source domain features and target domain features through the CORAL loss; assume a given source domain datasetCorresponding source domain tag setTarget Domain dataset +.> The number of source domain and target domain samples is n respectively S And n T Let->And-> Is input D S And D T Through a mapping function ψ (D S ) And psi (D) T ) The resulting d-dimensional features, ψ, represent the intermediate of Improved-EEGNetTotal representation with feature extraction layer, +.>The j-th dimension representing the i-th source domain data,>the j-th dimension, C, representing the i-th target domain data S And C T The feature covariance matrix representing the source and target domains defines the CORAL penalty as the distance between the second order statistics (covariance) of the source and target domain features:
wherein,the covariance matrix, representing the square of the matrix Frobenius norm, of the source and target domains is calculated by:
where 1 represents a column vector with all elements being 1.
The gradient of CORAL loss with respect to input features can be calculated using the chain law as follows:
the neural network without transfer learning generally minimizes the classification loss, so as to obtain a classification model with good fitting effect with the true value, and the characteristic degradation can be caused by independently minimizing the CORAL loss; the feature obtained by training the CORAL loss and the classification loss of the source domain data has good separability and good performance on the target domain, so the loss function of the Improved-EEGNet-transfer algorithm is defined as follows:
L＝L CLASS +λL CORAL
wherein L is cLASS Representing the loss of classification, L CORAL Representing the CORAL loss, λ is an adjustable parameter that adjusts the importance of the classification loss and the CORAL loss, which are matched to each other, and the final feature has good separability and can be well applied to the target domain after the training process is finished.
Step 4: tagging source domain EEG data D S And target domain unlabeled EEG data D T Inputting the data into an Improved-EEGNet model, and training the transfer learning model by using an Improved-EEGNet-transfer algorithm. The Improved-EEGNet-transfer algorithm is shown in Table 1:
TABLE 1Improved-EEGNet-transfer Algorithm
The iterative process of the Improved-EEGNet-transfer algorithm is specifically as follows:
from source domain dataset D S Target domain dataset D T Randomly acquiring a preset number of source domain data and target domain data respectively;
inputting the labeled source domain data into an Improved-EEGNet classification model, calculating classification loss, and obtaining source domain characteristics;
inputting target domain data without labels into an Improved-EEGNet classification model to obtain target domain characteristics, and calculating CORAL loss by combining source domain characteristics;
based on the classification loss and the CORAL loss, the total loss is calculated, the gradient is calculated using a back propagation algorithm and the parameters of the model are updated until the number of iterations is reached.
In the process of calculating the classification loss and the CORAL loss, as shown in fig. 5, the model structure and parameters are shared between the model corresponding layers, that is, the model structure and parameters for calculating the classification loss are the same as the model structure and parameters for calculating the CORAL loss.
Step 5: and applying the trained model to a target domain test set to obtain classification accuracy as a model performance evaluation index.
In order to verify the improvement effect of the Improved-EEGNet classification model relative to the EEGNet classification model performance and the effect of the migration learning algorithm based on the CORAL loss, three groups of experiments are designed for comparison in the embodiment.
The first group is the comparison of EEGNet and Improved-EEGNet classification performance, and the training set and the test set of the group of experiments obey the same distribution, namely, a scene which does not need to carry out transfer learning; the purpose of this set of experiments was to verify the improvement in classification accuracy of Improved-EEGNet relative to EEGNet. Thereby proving the superiority of the Improved-EEGNet model proposed by the invention. In order to verify the classification effect when different training sample amounts are used, the number of training set sample trail is respectively determined to be 24, 36, 48, 60, 72, 96, 120, 140 and the like, the corresponding number of test set sample trail is respectively 256, 244, 232, 220, 208, 184, 160 and 140, and the training set accounts for 8.6%, 12.9%, 17.1%, 21.4%, 25.7%, 34.3%, 42.9% and 50%; the experimental results are shown in table 2, table 3, fig. 6, and fig. 7.
The second group is that the EEGNet model directly compares the cross-test classification effect with the cross-test classification effect of the transfer learning algorithm of EEGNet combined CORAL loss, so as to verify the effect of the CORAL loss in the cross-test transfer learning scenario. In this set of experiments, the training data for both the target test and the auxiliary test were 140 trail, and the test data was 140 trail for which the target test did not participate in training. The experimental results are shown in tables 4, 5 and 8.
The third group is the comparison of three methods of EEGNet direct cross-test classification, EEGNet combined with CORAL loss transfer learning classification and Improved-EEGNet-transfer algorithm transfer learning classification, which are named as algorithm 1, algorithm 2 and algorithm 3 in sequence, and the purpose of the group of experiments is to verify the superior performance of the transfer learning algorithm based on Improved-EEGNet model combined with CORAL loss in the field of motor imagery brain electrical signal cross-test classification, wherein the dividing method of training data and test data is the same as that of the second group of experiments. The experimental results are shown in FIG. 8.
The embodiment researches the problem of migration learning of motor imagery electroencephalogram data, so that five tested objects are in turn used as target tested objects, the motor imagery electroencephalogram data of the target tested objects are not provided with labels, the other four tested objects are sequentially used as auxiliary tested objects, the motor imagery electroencephalogram data of the auxiliary tested objects are provided with labels, and the migration learning method used in the embodiment is a single-source-domain unsupervised migration learning method, so that only one auxiliary tested object can be selected each time; the evaluation index of the algorithm performance used in this embodiment is the accuracy, that is, the ratio of the number of trail (samples) predicted to be correct on the test set to the number of trail (samples) in the test set, and the experimental results are shown in tables 2-6 and fig. 6-8.
Experimental results show that compared with EEGNet, the Improved-EEGNet-transfer algorithm provided by the invention directly spans the tested classification accuracy, the effective migration of source domain information can be realized, and therefore, the problem of low span tested identification accuracy in the field of the current motor imagery brain-computer interface is solved to a certain extent.
TABLE 2 EEGNet Classification accuracy
TABLE 3 Improved-EEGNet Classification accuracy
TABLE 4 EEGNet Cross-test Classification accuracy
TABLE 5 EEGNet combined CORAL penalty transfer learning accuracy
TABLE 6 Improved-EEGNet combined with CORAL penalty transfer learning accuracy
Example two
In one or more embodiments, a motor imagery electroencephalogram signal classification system is disclosed, comprising an acquisition module and a classification module:
an acquisition module configured to: acquiring target domain motor imagery electroencephalogram signals to be classified;
a classification module configured to: classifying the preprocessed target domain motor imagery electroencephalogram signals by using a classification model trained by an Improved-EEGNet-transfer algorithm to obtain and output classification results;
the classification model is obtained by adding a time convolution layer and a space convolution layer on an EEGNet neural network, namely an Improved-EEGNet model, wherein the Improved-EEGNet-transfer algorithm introduces CORAL loss of source domain features and target domain features, the classification model is trained by combining the CORAL loss and the classification loss of source domain data, the features with good separability are formed, the offset between the source domain features and the target domain features is reduced, and the target domain training classification model is helped by using a source domain data tag.
Example III
An object of the present embodiment is to provide a computer-readable storage medium.
A computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps in a motor imagery electroencephalogram classification method according to an embodiment of the present disclosure.
Example IV
An object of the present embodiment is to provide an electronic apparatus.
An electronic device includes a memory, a processor, and a program stored on the memory and executable on the processor, wherein the processor implements steps in a motor imagery electroencephalogram classification method according to an embodiment of the present disclosure when the program is executed.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. The motor imagery electroencephalogram signal classification method is characterized by comprising the following steps of:
acquiring target domain motor imagery electroencephalogram signals to be classified;
classifying the preprocessed target domain motor imagery electroencephalogram signals by using a classification model trained by an Improved-EEGNet-transfer algorithm to obtain and output classification results;
the classification model is obtained by adding a time convolution layer and a space convolution layer on an EEGNet neural network, namely an Improved-EEGNet model, wherein the Improved-EEGNet-transfer algorithm introduces correlation alignment loss of source domain data and target domain data, and the classification model is trained by combining the correlation alignment loss and the classification loss of the source domain data to form a characteristic with good separability, reduce the offset between the source domain characteristic and the target domain characteristic and further help the target domain to train the classification model by using a source domain data label.
2. A motor imagery electroencephalogram classification method as set forth in claim 1, wherein the preprocessing includes bandpass filtering the electroencephalogram with a bandpass filter and removing artifacts of the electroencephalogram with a common average reference.
3. The motor imagery electroencephalogram signal classification method as set forth in claim 1, wherein a temporal convolution layer and a spatial convolution layer are added on the EEGNet neural network, specifically: and improving one time convolution layer and one space convolution layer of the EEGNet neural network into two time convolution layers and two space convolution layers.
4. A motor imagery electroencephalogram classification method as set forth in claim 1, wherein the correlation alignment loss is defined as a distance between second order statistics of source domain features and target domain features:
wherein,represents the square of the matrix Frobenius norm, C s And C T The feature covariance matrix representing the source and target domains, d representing the number of dimensions of the features of the source and target domains.
5. A motor imagery electroencephalogram classification method according to claim 4, wherein the correlation alignment loss is calculated with respect to gradients of input features using a chain law as follows:
wherein,and->Is input source domain data D S And destination domain data D T Through a mapping function ψ (D S ) And psi (D) T ) The resulting d-dimensional features, ψ represents the total representation of all feature extraction layers in the Improved-EEGNet, +.>The j-th dimension representing the i-th source domain data,>the j-th dimension, C, representing the i-th target domain data S And C T A feature covariance matrix representing the source domain and the target domain.
6. The motor imagery electroencephalogram classification method according to claim 1, wherein the Improved-EEGNet-transfer algorithm uses a correlation alignment loss and classification loss combined training classification model of source domain data, and a loss function is:
L＝L CLASS +λL CORAL
wherein L is CLASS Representing source domain data classification loss, L CORAL Representing the correlation alignment loss, λ is an adjustable parameter that adjusts the importance of the classification loss and the correlation alignment loss.
7. The motor imagery electroencephalogram classification method as set forth in claim 1, wherein the Improved-EEGNet-transfer algorithm is based on a source domain data set D S And a target domain data set D T The following iterative learning is performed:
from source domain dataset D S Target domain dataset D T Respectively randomly acquiring a preset number of source domain data and target domain data, whichIn, source domain dataset D S Composed of labeled source domain data, target domain data set D T Consists of target domain data without labels;
inputting the labeled source domain data into a classification model, calculating classification loss, and obtaining source domain characteristics;
inputting target domain data without labels into a classification model to obtain target domain characteristics, and calculating correlation alignment loss by combining source domain characteristics;
based on the classification loss and the correlation alignment loss, a total loss is calculated, a back propagation algorithm is used to calculate the gradient and update the parameters of the model until the number of iterations is reached.
8. The motor imagery electroencephalogram signal classification system is characterized by comprising an acquisition module and a classification module:
an acquisition module configured to: acquiring target domain motor imagery electroencephalogram signals to be classified;
a classification module configured to: classifying the preprocessed target domain motor imagery electroencephalogram signals by using a classification model trained by an Improved-EEGNet-transfer algorithm to obtain and output classification results;
the classification model is obtained by adding a time convolution layer and a space convolution layer on an EEGNet neural network, namely an Improved-EEGNet model, wherein the Improved-EEGNet-transfer algorithm introduces correlation alignment loss of source domain data and target domain data, and the classification model is trained by combining the correlation alignment loss and the classification loss of the source domain data to form a characteristic with good separability, reduce the offset between the source domain characteristic and the target domain characteristic and further help the target domain to train the classification model by using a source domain data label.
9. An electronic device, comprising:
a memory for non-transitory storage of computer readable instructions; and
a processor for executing the computer-readable instructions,
wherein the computer readable instructions, when executed by the processor, perform the method of any of the preceding claims 1-7.
10. A storage medium, characterized by non-transitory storing computer-readable instructions, wherein the instructions of the method of any one of claims 1-7 are performed when the non-transitory computer-readable instructions are executed by a computer.
CN202311487339.4A 2023-11-08 2023-11-08 Motor imagery electroencephalogram signal classification method and system Pending CN117520891A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311487339.4A CN117520891A (en) 2023-11-08 2023-11-08 Motor imagery electroencephalogram signal classification method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311487339.4A CN117520891A (en) 2023-11-08 2023-11-08 Motor imagery electroencephalogram signal classification method and system

Publications (1)

Publication Number Publication Date
CN117520891A true CN117520891A (en) 2024-02-06

Family

ID=89760100

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311487339.4A Pending CN117520891A (en) 2023-11-08 2023-11-08 Motor imagery electroencephalogram signal classification method and system

Country Status (1)

Country Link
CN (1) CN117520891A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117725490A (en) * 2024-02-08 2024-03-19 山东大学 Cross-test passive pitch-aware EEG automatic classification method and system
CN117892183A (en) * 2024-03-14 2024-04-16 南京邮电大学 Electroencephalogram signal identification method and system based on reliable transfer learning
CN117892183B (en) * 2024-03-14 2024-06-04 南京邮电大学 Electroencephalogram signal identification method and system based on reliable transfer learning

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117725490A (en) * 2024-02-08 2024-03-19 山东大学 Cross-test passive pitch-aware EEG automatic classification method and system
CN117725490B (en) * 2024-02-08 2024-04-26 山东大学 Cross-test passive pitch-aware EEG automatic classification method and system
CN117892183A (en) * 2024-03-14 2024-04-16 南京邮电大学 Electroencephalogram signal identification method and system based on reliable transfer learning
CN117892183B (en) * 2024-03-14 2024-06-04 南京邮电大学 Electroencephalogram signal identification method and system based on reliable transfer learning

Similar Documents

Publication Publication Date Title
CN111012336B (en) Parallel convolutional network motor imagery electroencephalogram classification method based on spatio-temporal feature fusion
CN109165556B (en) Identity recognition method based on GRNN
CN112120694B (en) Motor imagery electroencephalogram signal classification method based on neural network
Miao et al. A spatial-frequency-temporal optimized feature sparse representation-based classification method for motor imagery EEG pattern recognition
CN109598222B (en) EEMD data enhancement-based wavelet neural network motor imagery electroencephalogram classification method
CN114533086B (en) Motor imagery brain electrolysis code method based on airspace characteristic time-frequency transformation
CN112515685A (en) Multi-channel electroencephalogram signal channel selection method based on time-frequency co-fusion
CN109325586B (en) System for denoising electroencephalogram signal
CN105956624A (en) Motor imagery electroencephalogram classification method based on space-time-frequency optimization feature sparse representation
Pan et al. MAtt: A manifold attention network for EEG decoding
CN113536882B (en) Multi-class motor imagery electroencephalogram signal feature extraction and classification method
CN112528834B (en) Electroencephalogram signal cross-test classification method of subband target alignment co-space mode
CN117520891A (en) Motor imagery electroencephalogram signal classification method and system
CN113208593A (en) Multi-modal physiological signal emotion classification method based on correlation dynamic fusion
Janapati et al. Towards a more theory-driven BCI using source reconstructed dynamics of EEG time-series
CN113180692A (en) Electroencephalogram signal classification and identification method based on feature fusion and attention mechanism
Fauzi et al. Channel selection for common spatial pattern Based on energy calculation of motor imagery EEG signal
CN115238796A (en) Motor imagery electroencephalogram signal classification method based on parallel DAMSCN-LSTM
CN113128384B (en) Brain-computer interface software key technical method of cerebral apoplexy rehabilitation system based on deep learning
CN116919422A (en) Multi-feature emotion electroencephalogram recognition model establishment method and device based on graph convolution
CN117195099A (en) Electroencephalogram signal emotion recognition algorithm integrating multi-scale features
CN111543985A (en) Brain control hybrid intelligent rehabilitation method based on novel deep learning model
CN109685031A (en) A kind of brain-computer interface midbrain signal characteristics classification method and system
CN115105093A (en) EEG signal classification and identification method based on power spectral density predetermined frequency band
Mu et al. A Channel Selection Method for Motor Imagery EEG Based on Fisher Score of OVR-CSP

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination