CN117150346A - EEG-based motor imagery electroencephalogram classification method, device, equipment and medium - Google Patents

EEG-based motor imagery electroencephalogram classification method, device, equipment and medium Download PDF

Info

Publication number
CN117150346A
CN117150346A CN202311178742.9A CN202311178742A CN117150346A CN 117150346 A CN117150346 A CN 117150346A CN 202311178742 A CN202311178742 A CN 202311178742A CN 117150346 A CN117150346 A CN 117150346A
Authority
CN
China
Prior art keywords
module
eeg
tcn
electroencephalogram
layer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311178742.9A
Other languages
Chinese (zh)
Inventor
王力
张准
黎瑾
冯玉杰
黄明阳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou University
Original Assignee
Guangzhou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou University filed Critical Guangzhou University
Priority to CN202311178742.9A priority Critical patent/CN117150346A/en
Publication of CN117150346A publication Critical patent/CN117150346A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2123/00—Data types
    • G06F2123/02—Data types in the time domain, e.g. time-series data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/08—Feature extraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/12—Classification; Matching

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Psychiatry (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Pathology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Surgery (AREA)
  • Public Health (AREA)
  • Probability & Statistics with Applications (AREA)
  • Psychology (AREA)
  • Fuzzy Systems (AREA)
  • Physiology (AREA)
  • Signal Processing (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)

Abstract

The embodiment of the specification provides a motor imagery electroencephalogram classification method, a motor imagery electroencephalogram classification device, motor imagery electroencephalogram classification equipment and a motor imagery electroencephalogram classification medium based on EEG, wherein the method comprises the following steps: acquiring EEG signals to obtain a sample set, and dividing the sample set into a training set and a testing set; inputting a training set into an initial electroencephalogram signal classification model, and training the electroencephalogram signal classification model; performing performance evaluation on the trained electroencephalogram signal classification model through a test set to obtain the trained electroencephalogram signal classification model; inputting signals to be classified into a trained electroencephalogram signal classification model to obtain a signal classification result; the electroencephalogram signal classification model comprises a feature extraction unit, a feature fusion module and a classification module, wherein the feature extraction unit comprises three parallel feature extraction branch networks; each feature extraction branch network comprises an EEGNet module, a SE module and a Bi-TCN module. So as to solve the problem of low prediction classification precision.

Description

EEG-based motor imagery electroencephalogram classification method, device, equipment and medium
Technical Field
The application relates to the technical field of deep learning, in particular to a motor imagery electroencephalogram classification method, device, equipment and medium based on EEG.
Background
Brain-computer interface (BCI) is a new technology that has emerged in the last decades. The device can directly communicate with the brain, interpret brain electrical activity and convert the brain electrical activity into external instructions to control peripheral equipment, such as exercise rehabilitation, wheelchair control, robot control, man-machine interaction and the like. Electroencephalogram (EEG) is a non-invasive method of recording brain electrical activity. EEG has been widely used in nerve engineering, neuroscience and BCI systems. EEG has advantages over other methods such as functional magnetic resonance imaging (fMRI), brain magnetic imaging (MEG), and near infrared spectroscopy (NIRS) in that surgical implantation of electrodes is unnecessary, high spatial-temporal resolution, portability, and affordability, and is therefore more popular. However, EEG signals also suffer from limitations such as low signal-to-noise ratio, nonlinearity, weak signal amplitude, low spatial resolution, and large individual differences. These problems present significant challenges for the processing and classification of EEG signals in a BCI system.
To address the difficulties of EEG task classification, traditional Machine Learning (ML) and Deep Learning (DL) methods have been proposed. In the conventional ML method, manual feature extraction is required, and time-domain and frequency-domain features are extracted from the original EEG signal using techniques such as wavelet transform, power spectral density, and fast fourier transform. For feature extraction of spatial domain EEG signals, filter Bank Common Spatial Pattern (FBCSP) and variants thereof have proven to be effective in improving accuracy. These representative features may be classified using supervised learning methods such as Support Vector Machines (SVMs) and Linear Discriminant Analysis (LDA) or unsupervised learning methods such as K Nearest Neighbors (KNNs). However, conventional methods rely on expert knowledge, and therefore DL methods are favored over manual feature extraction because they can build end-to-end models that learn complex patterns from multi-dimensional data without preprocessing or manual feature extraction.
In terms of EEG signal classification, various DL models have been proposed, including Convolutional Neural Networks (CNNs), recurrent Neural Networks (RNNs), deep Belief Networks (DBNs), self-encoders (AEs), and hybrid DL models. However, the method proposed by the existing study has low classification accuracy of signals, and in the study of classifying the electroencephalogram signals by using a model comprising TCN, only unidirectional TCN is often used (namely, only information acquired from past sequences is considered, and information acquired from future sequences is ignored.
Disclosure of Invention
To overcome the problems in the related art, the present disclosure provides a motor imagery electroencephalogram classification method, apparatus, device, and medium based on EEG.
One or more embodiments of the present specification provide a motor imagery electroencephalogram classification method based on EEG, comprising the steps of:
s, acquiring EEG signals to obtain a sample set, and dividing the sample set into a training set and a testing set;
s2, inputting a training set into an initial electroencephalogram signal classification model, and training the electroencephalogram signal classification model according to set convergence conditions;
s3, performing performance evaluation on the trained electroencephalogram signal classification model through a test set to obtain the trained electroencephalogram signal classification model;
s4, inputting EEG signals to be classified into a trained EEG signal classification model to obtain signal characteristics of the EEG signals; wherein,
the electroencephalogram signal classification model comprises a feature extraction unit, a feature fusion module and a classification module, wherein the feature extraction unit comprises three feature extraction branch networks of which the output ends are connected with the feature fusion module;
each feature extraction branch network comprises an EEGNet module, an SE module and a Bi-TCN module, wherein the EEGNet module is used for extracting time-frequency features of original signals, and selecting high-weight features by using the SE module;
and then, through a feature fusion module, the features extracted from the three feature extraction branch networks are received and combined to obtain comprehensive features, and finally, the comprehensive features are classified by a classifier to obtain an obtained classification result.
One or more embodiments of the present specification provide an EEG-based motor imagery electroencephalogram classification apparatus comprising:
sample acquisition module: acquiring EEG signals to obtain a sample set, and dividing the sample set into a training set and a testing set;
model training module: inputting a training set into an initial electroencephalogram signal classification model, and training the electroencephalogram signal classification model according to set convergence conditions;
model test module: performing performance evaluation on the trained electroencephalogram signal classification model through a test set to obtain the trained electroencephalogram signal classification model;
and a prediction module: inputting EEG signals to be classified into a trained EEG signal classification model to obtain signal characteristics of the EEG signals; wherein,
the electroencephalogram signal classification model comprises a feature extraction unit, a feature fusion module and a classification module, wherein the feature extraction unit comprises three feature extraction branch networks of which the output ends are connected with the feature fusion module;
each feature extraction branch network comprises an EEGNet module, an SE module and a Bi-TCN module, wherein the EEGNet module is used for extracting time-frequency features and space features of original signals, and selecting high-weight features by using the SE module; and then, through a feature fusion module, the features extracted from the three feature extraction branch networks are received and combined to obtain comprehensive features, and finally, the comprehensive features are classified by a classifier to obtain an obtained classification result.
One or more embodiments of the present specification provide a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, which when executed by the processor implements an EEG-based motor imagery electroencephalogram classification method as described above.
One or more embodiments of the present specification provide a computer readable storage medium storing a computer program which, when executed by a processor, implements the steps of an EEG-based motor imagery electroencephalogram classification method as described above.
The application provides a motor imagery electroencephalogram classification method, a motor imagery electroencephalogram classification device, motor imagery electroencephalogram classification equipment and a motor imagery electroencephalogram classification medium based on an EEG.
Drawings
For a clearer description of one or more embodiments of the present description or of the solutions of the prior art, the drawings that are necessary for the description of the embodiments or of the prior art will be briefly described, it being apparent that the drawings in the description that follow are only some of the embodiments described in the description, from which, for a person skilled in the art, other drawings can be obtained without inventive faculty.
FIG. 1 is a flow diagram of a motor imagery electroencephalogram classification method based on EEG provided in one or more embodiments of the present disclosure;
FIG. 2 is a network architecture diagram of an EEG classification model in an EEG classification method according to one or more embodiments of the present disclosure;
FIG. 3 is a network architecture diagram of EEGNet model in an EEG-based motor imagery electroencephalogram classification method according to one or more embodiments of the present disclosure;
FIG. 4 is a block diagram of a Bi-TCN module in an EEG based motor imagery electroencephalogram classification method according to one or more embodiments of the present disclosure; fig. 5 is a schematic diagram of a network structure of a Bi-TCN module in an EEG-based motor imagery electroencephalogram classification method according to one or more embodiments of the present disclosure;
fig. 6 is a network parameter setting diagram of an electroencephalogram classification model according to one or more embodiments of the present disclosure in a comparative case provided by one or more embodiments of the present disclosure;
FIG. 7 is a block diagram of an EEG-based motor imagery electroencephalogram classification apparatus according to one or more embodiments of the present disclosure;
fig. 8 is a schematic structural diagram of a computer according to one or more embodiments of the present disclosure.
Detailed Description
In order to enable a person skilled in the art to better understand the technical solutions in one or more embodiments of the present specification, the technical solutions in one or more embodiments of the present specification will be clearly and completely described below with reference to the drawings in one or more embodiments of the present specification, and it is obvious that the described embodiments are only some embodiments of the present specification, not all embodiments. All other embodiments, which can be made by one or more embodiments of the present disclosure without inventive faculty, are intended to be within the scope of the present disclosure.
The application is described in detail below with reference to the detailed description and the accompanying drawings.
Method embodiment
According to an embodiment of the present application, as shown in fig. 1, there is provided an EEG-based motor imagery electroencephalogram classification method, which is a flowchart of the EEG-based motor imagery electroencephalogram classification method provided in the present embodiment, and according to the embodiment of the present application, the EEG-based motor imagery electroencephalogram classification method includes:
s1, acquiring EEG signals to obtain a sample set, and dividing the sample set into a training set and a testing set;
s2, inputting a training set into an initial electroencephalogram signal classification model, and training the electroencephalogram signal classification model according to set convergence conditions;
s3, performing performance evaluation on the trained electroencephalogram signal classification model through a test set to obtain the trained electroencephalogram signal classification model;
s4, inputting EEG signals to be classified into a trained EEG signal classification model to obtain classification results of the EEG signals; wherein,
the electroencephalogram signal classification model comprises a feature extraction unit, a feature fusion module and a classification module, wherein the feature extraction unit comprises three parallel feature extraction branch networks;
each feature extraction branch network comprises an EEGNet module, an SE (channel attention) module and a Bi-TCN module, wherein the EEGNet module is used for extracting time-frequency features and space features of original signals, the SE module is used for selecting high-weight features, and the Bi-TCN module is provided with a forward TCN residual block and a reverse TCN residual block which are used for respectively processing sequences output by the SE module and extracting time features through two directions simultaneously; the model gives the model a larger weight to the characteristics which can improve the classification accuracy, namely the characteristics which are more valuable.
And then, through a feature fusion module, the high-level time features extracted from the three feature extraction branch networks are received and combined to obtain comprehensive features, and finally, the comprehensive features are classified by a classifier to obtain an obtained classification result.
The method provided by the application adopts a multi-branch network structure to fully learn more features in a deep learning model, and uses a channel attention module to select the features, and is different from the existing unidirectional TCN model in that the Bi-TCN module is provided with a forward TCN residual block and a reverse TCN residual block which respectively process sequences output by the SE module so as to realize that time features are extracted simultaneously in two directions, so that information can be obtained from past sequences and information of future sequences can be obtained, thereby fully utilizing context information, better capturing long-distance dependence in the sequences and generating richer and more accurate feature representations. In addition, the embodiment combines the features from different branches through the feature fusion block to construct the feature representation with more information and characterization capability. Compared with the prior art, the model parameters are relatively fewer, a variable network is not needed, each subject uses the same model hyper-parameters, and the process of searching the optimal hyper-parameters for each subject is avoided. In addition, the electroencephalogram signal classification model provided by the application performs subject-specific evaluation on a private data set, and performs better than the existing algorithm, so that the effectiveness of the electroencephalogram signal classification model in EEG decoding is proved.
In some embodiments, the electroencephalogram classification model provided in this embodiment is an end-to-end model, and its overall network structure includes three parallel branches, as shown in fig. 2, and is a network structure diagram of the electroencephalogram classification model provided in this embodiment, and the parallel three feature extraction branch networks can learn different space-time feature information according to different input EEG signals, so as to increase the number of feature graphs, improve classification accuracy, and simultaneously, make the network have stronger generalization performance and robustness. Each branch has the same structure and comprises three modules: EEGNet module, SE module and Bi-TCN module. By means of a feature Fusion module (Fusion Block), partial features from the three branches are combined to get a feature representation with more information and characterization capabilities. Finally, a classifier Block (classifier Block) is used to obtain the classification result. The specific structure of the electroencephalogram classification model is specifically described below.
In some embodiments, referring to fig. 2, each EEGNet module includes a time Conv layer, a depth separable layer, a first dropout layer, a first averaging pooling layer, a separable Conv layer, a second averaging pooling layer, and a second dropout layer connected in sequence, where the time Conv layer is used to learn a frequency filter, the depth separable convolution layer is used to learn a spatial filter, and the averaging pooling layer is used to reduce a feature dimension;
the size of the convolution kernel of the time convolution layer and the separable convolution layer in each EEGNet module is set differently.
In this embodiment, the EEGNet is used as backbone network to encode low-level spatiotemporal information in the original EEG signal. The network structure of EEGNet referring to fig. 3, EEGNet first learns the frequency filter through the temporal convolution layer and then learns the spatial filter through the depth convolution layer. EEGNet learns the time summary of each feature map using separable convolutions prior to combining and classifying the feature maps, and by using first and second averaging pooling layers, the feature dimensions can be reduced to prevent overfitting and reduce computation time, using different convolution kernel sizes on each branch, using mixed convolution scales to improve classification accuracy. For the temporal convolution layers of each branch, the convolution kernel sizes of 64, 32 and 16 are used to learn the different bandpass frequencies, respectively, and for the separable convolution layers we use the convolution kernel sizes of 16, 8 and 4.
In some embodiments, the SE module is configured to perform compression, excitation, and weighting operations on the time-frequency characteristics, the SE module focusing on useful characteristics and suppressing unimportant characteristics.
In this embodiment, the SE module is an attention module, originally used for image classification, which is mainly composed of two steps: first (compression operation), the entire channel is compressed by global pooling (typically average pooling) to form a compression vector m q The method comprises the steps of carrying out a first treatment on the surface of the I.e.
Wherein F is sq Representing the compression operation, U q L is the feature mapping size for the input feature map;
then excitation operation is performed, the dimension of the input vector is reduced through the first full-connection layer, the dimension of the dimension-reduced vector is increased through the second full-connection layer, and then a weight vector S is generated through a Sigmoid activation function q Weight vector S q The following formula is defined:
S q ＝F ex (m q w) formula 2;
wherein F is ex For excitation operation, W is the ratio parameter to adjust the excitation operation;
finally, through the weight vector S q Element-wise multiplication with the original feature map assigns weights to the original features:
f q ＝F scale (U q ,S q ) Formula 3;
wherein f q For weighted output vector, F scale To add weights.
By mapping the characteristics of different channels by the weighted characteristics, the SE network can concentrate more on useful characteristics and inhibit unimportant characteristics, thereby improving the performance and generalization capability of the whole model. By introducing an SE block after the EEGNet module, the weight of each channel in each feature map can be adaptively adjusted, thereby improving network performance.
In some embodiments, referring to fig. 4 to 5, fig. 4 is a block diagram of a Bi-TCN module provided by this embodiment, and fig. 5 is a network structure schematic diagram of the Bi-TCN module provided by this embodiment, where the Bi-TCN module includes two forward TCN residual blocks and two reverse TCN residual blocks, the two forward TCN residual blocks implement a sequence that is output by a forward sequence processing SE module, the two reverse TCN residual blocks process the sequence output by the SE module after being turned over in reverse sequence, and the processing results are synchronously output to a feature fusion module;
the two TCN residual blocks comprise a depth separable expansion causal convolution layer, a batch normalization layer, an ELU activation function layer and a Dropout layer; the dilation-causal convolution uses progressively increasing dilation factors to skip adjacent time steps to achieve a reduction in time information of the extraction sequence.
The forward TCN residual block processes the sequence (i.e., the signal form after SE) in such a way that from front to back, the forward TCN residual block only processes the point in time before the current point in time when the convolution is performed, and does not process the point in time after the current point in time (this is where the causal convolution is different from the normal convolution). The reverse TCN residual block is extracted from the future sequence, and only the time point after the current time point is processed when convolution is performed, and the time point before the current time point is not processed, wherein the two TCNs in the forward direction or the reverse direction are different, the expansion factors (the parameter D in fig. 5) of the first TCN and the second TCN are different, the first TCN is 1, and the second TCN is 2.
The novel Bi-directional time convolutional neural network proposed in this embodiment is called Bi-TCN. Bi-TCN is mainly used for extracting high-level time features in time series data. And extracting time-frequency characteristics (mainly low-level characteristics) of the original signal by using EEGNet before Bi-TCN, and carrying out abstraction and combination on the basis of the low-level characteristics through Bi-TCN to obtain high-level time characteristics with more information and characteristic representation. Unlike unidirectional TCNs which only consider past sequences, bi-TCNs can extract information from future sequences, which means that the model can make predictions or decisions using information after the current time step when analyzing data, whereas unidirectional TCNs only consider information before the current time step, in particular Bi-TCNs can learn different characteristic information before and after the current time step by combining forward and reverse processing sequences. Fig. 5 shows the structure of Bi-TCN, which uses multiple residual blocks to capture Bi-directional temporal features. The Bi-TCN generates an output of the same length as the input, and each TCN block includes an dilation causal convolution, a batch normalization (Batch Normalization), an ELU activation function, and a Dropout layer. In the dilation-causal convolution, adjacent time steps are skipped using progressively increasing dilation factors, thereby reducing the extracted time information. The causal convolution limits that the convolution kernel can only convolve at a time step before the current time step, does not introduce information from a future time step, maintains causality, namely only relies on past information for analysis, and can extract information from a future sequence by introducing Bi-TCN; the dilation convolution may cause the receptive field of the network to increase proportionally with depth by an exponentially increasing dilation factor. The forward TCN residual block and the reverse TCN residual block in Bi-TCN consist of multi-layer dilation convolutions. By stacking residual blocks, the expansion rate of each block increases exponentially, so the receptive field increases exponentially. Wherein, the receptive field is calculated as follows:
wherein, the residual blocks are composed of a plurality of expansion convolution layers, m is the number of expansion convolution layers in each residual block, K is the size of the expansion convolution layer convolution kernel, D is the size of the expansion base, and n is the number of residual blocks.
The residual block in this embodiment leads out a series of branches of transform F, defined as:
output=ψ (x+f (x)) formula 5;
where x is the input, ψ is the activation function, and in this embodiment, the used activation function is ELU, and ELU of 0< α is defined as:
wherein the super parameter alpha controls the value of ELU saturation when the input is negative.
By introducing batch normalization into the model, the training of the network can be accelerated, gradient disappearance can be relieved, the regularization effect can be achieved, and overfitting can be prevented. The BN algorithm is as follows:
wherein m represents the size of batch_size, x i Mu for the i-th sample of the input data B As a mean value of the lot data,for the variance of the batch data, n i For single samples after batch normalization, E is a small value, mainly preventing the denominator from appearing as 0, y i And (3) obtaining the single sample after linear change through neural network training, wherein gamma and beta are super parameters.
Dropout is a regularization technique that reduces over-fitting of the model. The principle of operation is to randomly set the output of a portion of neurons to 0 during the training process. By such random 'discarding' of the output of neurons, the forced model is independent of the particular neuron, forcing the network to learn a more robust and generalized representation of the features. Slightly different from the TC blocks in ATCNet networks, the proposed Bi-TCN uses depth separable dilation causal convolution instead of the original dilation causal convolution, since we find this approach more efficient and can reduce the number of parameters, reducing the overfitting.
In this embodiment, in the feature fusion module, we splice and fuse three advanced time features processed through Bi-TCN branches. And finally, sending the fused features to a classifier block for prediction. The classifier in this embodiment is implemented through a fully connected network, and is the last part of the proposed model for classifying tasks. For multi-class classification tasks, we use Softmax activation functions. Therefore, the tag with the highest probability is considered as the final decoding result.
The technical scheme proposed in this embodiment is described below by way of a comparative example.
The case is compared with the existing EEGNet, deepConvNet and MBEEGSE electroencephalogram characteristic extraction by the electroencephalogram classification model (model parameters are shown in FIG. 6), and the model provided by the application achieves 74.89% of overall accuracy and 0.67 k fraction and is higher than the average classification results of other models by referring to the following table 1. Compared with EEGNet, deepConvNet, the accuracy is improved by 11.9 percent and 10.96 percent respectively. The accuracy score standard deviation between the tested samples is 10.3 and is lower than 15.1 of EEGNet and 11.3 of MBEEGSE, and the EEGNet adopts a convolution kernel with a single scale, so that the implicit characteristic information of the electroencephalogram signal can not be fully extracted, and the model provided by the application has good performance in four time sequence signal identification.
Table 1, four kinds of time sequence signal recognition results
Device embodiment
According to an embodiment of the present application, there is provided an EEG-based motor imagery electroencephalogram classification apparatus, as shown in fig. 7, according to an embodiment of the present application, including:
sample acquisition module 10: the EEG signal acquisition device is used for acquiring an EEG signal to obtain a sample set, and dividing the sample set into a training set and a testing set;
model training module 20: the method comprises the steps of inputting a training set into an initial electroencephalogram signal classification model, and training the electroencephalogram signal classification model according to set convergence conditions;
model test module 30: the method comprises the steps of performing performance evaluation on a trained electroencephalogram signal classification model through a test set to obtain the trained electroencephalogram signal classification model;
prediction module 40: inputting EEG signals to be classified into a trained EEG signal classification model to obtain signal characteristics of the EEG signals; wherein,
the electroencephalogram signal classification model comprises a feature extraction unit, a feature fusion module and a classification module, wherein the feature extraction unit comprises three feature extraction branch networks of which the output ends are connected with the feature fusion module;
each feature extraction branch network comprises an EEGNet module, an SE module and a Bi-TCN module, wherein the EEGNet module is used for extracting time-frequency features and space features of original signals, and selecting high-weight features by using the SE module;
and then, through a feature fusion module, the features extracted from the three feature extraction branch networks are received and combined to obtain comprehensive features, and finally, the comprehensive features are classified by a classifier to obtain an obtained classification result.
The device provided by the application adopts a multi-branch network structure to fully learn more features in a deep learning model, and uses a channel attention module to select the features, and is different from the existing unidirectional TCN model in that the Bi-TCN module is provided with a forward TCN residual block and a reverse TCN residual block which respectively process sequences output by the SE module so as to realize that the time features are extracted in two directions simultaneously, so that not only information can be obtained from past sequences, but also information of future sequences can be obtained, thereby fully utilizing context information, better capturing long-distance dependence in the sequences and generating richer and more accurate feature representations. In addition, the embodiment combines the features from different branches through the feature fusion block to construct the feature representation with more information and characterization capability. Compared with the prior art, the model parameters are relatively fewer, a variable network is not needed, each subject uses the same model hyper-parameters, and the process of searching the optimal hyper-parameters for each subject is avoided.
In some embodiments, each EEGNet module includes a time Conv layer, a depth separable Conv layer, a first dropout layer, a first averaging pooling layer, a separable Conv layer, a second averaging pooling layer, and a second dropout layer connected in sequence, where the time Conv layer is used to learn a frequency filter, the depth separable convolution layer is used to learn a spatial filter, and the averaging pooling layer is used to reduce a feature dimension;
the size of the convolution kernel of the time convolution layer and the separable convolution layer in each EEGNet module is set differently.
Specifically, for each branch time Conv layer, the convolution kernel sizes of 64, 32 and 16 are used to learn the different bandpass frequencies, respectively, and for separable convolution layers, we use the convolution kernel sizes of 16, 8 and 4.
In some embodiments, the SE module is an attention module, originally used for image classification, consisting essentially of two steps: first (compression operation), the entire channel is compressed by global pooling (typically average pooling) to form a compression vector m q The method comprises the steps of carrying out a first treatment on the surface of the I.e.
Wherein F is sq Representing the compression operation, U q L is the feature mapping size for the input feature map;
then excitation operation is performed, the dimension of the input vector is reduced through the first full-connection layer, the dimension of the dimension-reduced vector is increased through the second full-connection layer, and then a weight vector S is generated through a Sigmoid activation function q Weight vector S q The following formula is defined:
S q ＝F ex (m q w) formula 2;
wherein F is ex For excitation operation, W is the ratio parameter to adjust the excitation operation;
finally, through the weight vector S q Element-wise multiplication with the original feature map assigns weights to the original features:
f q ＝F scale (U q ,S q ) Formula 3;
wherein f q For weighted output vector, F scale To add weights.
In some embodiments, the Bi-TCN module includes two TCN networks, each implementing a forward sequence of processing sequences, one in reverse order;
the forward and reverse TCN residual blocks comprise a depth separable expansion causal convolution layer, a batch normalization layer, an ELU activation function layer and a Dropout layer; the dilation-causal convolution skips adjacent time steps using progressively increasing dilation factors, reducing the time information of the extracted sequence.
In this embodiment, the dilation convolution increases the receptive field of the TCN network in proportion to depth by an exponentially increasing dilation factor, wherein the receptive field is calculated as follows:
wherein, the residual blocks are composed of a plurality of expansion convolution layers, m is the number of expansion convolution layers in each residual block, K is the size of the expansion convolution layer convolution kernel, D is the size of the expansion base, and n is the number of residual blocks.
The residual block in this embodiment leads out a series of branches of transform F, defined as:
output=ψ (x+f (x)) formula 5;
where x is the input, ψ is the activation function, and in this embodiment, the used activation function is ELU, and ELU of 0< α is defined as:
wherein the super parameter alpha controls the value of ELU saturation when the input is negative.
By introducing batch normalization into the model, the training of the network can be accelerated, gradient disappearance can be relieved, the regularization effect can be achieved, and overfitting can be prevented. The BN algorithm is as follows:
y i ＝γn i +β formula 10;
wherein m represents the size of batch_size, x i Mu for the i-th sample of the input data B As a mean value of the lot data,for the variance of the batch data, n i For single samples after batch normalization, E is a small value, mainly preventing the denominator from appearing as 0, y i And (3) obtaining the single sample after linear change through neural network training, wherein gamma and beta are super parameters.
The embodiment of the present application is an embodiment of a device corresponding to the embodiment of the method, and specific operations of processing steps of each module may be understood by referring to descriptions of the embodiment of the method, which are not repeated herein.
As shown in fig. 8, the present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the motor imagery electroencephalogram classification method based on EEG in the above embodiment, or which, when executed by a processor, implements the motor imagery electroencephalogram classification method based on EEG in the above embodiment, and which, when executed by the processor, implements the following method steps:
s1, acquiring EEG signals to obtain a sample set, and dividing the sample set into a training set and a testing set;
s2, inputting a training set into an initial electroencephalogram signal classification model, and training the electroencephalogram signal classification model according to set convergence conditions;
s3, performing performance evaluation on the trained electroencephalogram signal classification model through a test set to obtain the trained electroencephalogram signal classification model;
s4, inputting EEG signals to be classified into a trained EEG signal classification model to obtain signal characteristics of the EEG signals; wherein,
the electroencephalogram signal classification model comprises a feature extraction unit, a feature fusion module and a classification module, wherein the feature extraction unit comprises three feature extraction branch networks of which the output ends are connected with the feature fusion module;
each feature extraction branch network comprises an EEGNet module, an SE (channel attention) module and a Bi-TCN module, wherein the EEGNet module is used for extracting time-frequency features and space features of original signals, and selecting high-weight features by using the SE module;
and then, through a feature fusion module, the high-level time features extracted from the three feature extraction branch networks are received and combined to obtain comprehensive features, and finally, the comprehensive features are classified by a classifier to obtain an obtained classification result.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for apparatus or system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, with reference to the description of method embodiments in part. The apparatus and system embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present application without undue burden.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the application and are not specifically described in the present specification and will be apparent to those skilled in the art from the scope of the present application.

Claims (10)

1. EEG-based motor imagery electroencephalogram classification method is characterized by comprising the following steps:
s1, acquiring EEG signals to obtain a sample set, and dividing the sample set into a training set and a testing set;
s2, inputting a training set into an initial electroencephalogram signal classification model, and training the electroencephalogram signal classification model according to set convergence conditions;
s3, performing performance evaluation on the trained electroencephalogram signal classification model through a test set to obtain the trained electroencephalogram signal classification model;
s4, inputting EEG signals to be classified into a trained EEG signal classification model to obtain classification results of the EEG signals; wherein,
the electroencephalogram signal classification model comprises a feature extraction unit, a feature fusion module and a classification module, wherein the feature extraction unit comprises three parallel feature extraction branch networks;
each feature extraction branch network comprises an EEGNet module, an SE module and a Bi-TCN module, wherein the EEGNet module is used for extracting time-frequency features and space features of original signals, and selecting high-weight features by using the SE module;
and then, through a feature fusion module, the features extracted from the three feature extraction branch networks are received and combined to obtain comprehensive features, and finally, the comprehensive features are classified by a classifier to obtain an obtained classification result.
2. The EEG-based motor imagery electroencephalogram classification method of claim 1, wherein each EEGNet module comprises a temporal Conv layer, a depth separable Conv layer, a first dropout layer, a first averaging pooling layer, a separable Conv layer, a second averaging pooling layer, and a second dropout layer connected in sequence, wherein the temporal Conv layer is used to learn a frequency filter, the depth separable convolution layer is used to learn a spatial filter, and the averaging pooling layer is used to reduce a feature dimension;
the size of the convolution kernel of the time convolution layer and the separable convolution layer in each EEGNet module is set differently.
3. The EEG-based motor imagery electroencephalogram classification method of claim 1, wherein the SE module is configured to perform compression, excitation and weighting operations on the time-frequency features to achieve acquisition of valuable features and suppression of unimportant features.
4. An EEG-based motor imagery electroencephalogram classification method according to any one of claim 3, wherein the compressing, exciting and weighting operations are specifically calculated as follows:
compression operation, compressing whole channel to form compression vector m through global pooling q The calculation is as follows:
wherein F is sq Representing the compression operation, U q L is the feature mapping size for the input feature map;
excitation operation: the input vector is subjected to dimension reduction through the first full-connection layer, the dimension reduction vector is subjected to dimension increase through the second full-connection layer, and a weight vector S is generated through a Sigmoid activation function q Weight vector S q The following formula is defined:
S q ＝F ex (m q w) formula 2;
wherein F is ex For excitation operation, W is the ratio parameter to adjust the excitation operation;
and (3) adding weight: by a weight vector S q Element-wise multiplication with the original feature map assigns weights to the original features:
f q ＝F scale (U q ，S q ) Formula 3;
wherein f q For weighted output vector, F scale To add weights.
5. An EEG based motor imagery electroencephalogram classification method according to any one of claims 1-3, wherein the Bi-TCN module comprises:
the two forward TCN residual blocks realize forward sequence processing of sequences output by the SE module, processing results are output to the feature fusion module, the two reverse TCN residual blocks process the sequences output by the SE module after overturning processing in reverse sequence, and processing results are output to the feature fusion module;
the forward and reverse TCN residual blocks comprise a depth separable expansion causal convolution layer, a batch normalization layer, an ELU activation function layer and a Dropout layer; the dilation-causal convolution uses progressively increasing dilation factors to skip adjacent time steps to reduce the temporal information of the extracted sequence.
6. The EEG-based motor imagery electroencephalogram classification method of claim 5, wherein the dilation-cause convolution increases the receptive field of the TCN network in proportion to depth by an exponentially increasing dilation factor, wherein the receptive field is calculated as:
wherein, the forward TCN residual block and the reverse TCN residual block are composed of a plurality of expansion convolution layers, m is the number of expansion convolution layers in each residual block, K is the size of the expansion convolution layer convolution kernel, D is the size of the expansion base, and n is the number of residual blocks.
7. EEG-based motor imagery electroencephalogram classification device is characterized by comprising
Sample acquisition module: the EEG signal acquisition device is used for acquiring an EEG signal to obtain a sample set, and dividing the sample set into a training set and a testing set;
model training module: the method comprises the steps of inputting a training set into an initial electroencephalogram signal classification model, and training the electroencephalogram signal classification model according to set convergence conditions;
model test module: the method comprises the steps of performing performance evaluation on a trained electroencephalogram signal classification model through a test set to obtain the trained electroencephalogram signal classification model;
and a prediction module: the EEG signal classification method comprises the steps of inputting EEG signals to be classified into a trained EEG signal classification model to obtain classification results of the EEG signals; wherein,
the electroencephalogram signal classification model comprises a feature extraction unit, a feature fusion module and a classification module, wherein the feature extraction unit comprises three parallel feature extraction branch networks;
each feature extraction branch network comprises an EEGNet module, an SE module and a Bi-TCN module, wherein the EEGNet module is used for extracting time-frequency features and space features of original signals, and selecting high-weight features by using the SE module;
and then, through a feature fusion module, the features extracted from the three feature extraction branch networks are received and combined to obtain comprehensive features, and finally, the comprehensive features are classified by a classifier to obtain an obtained classification result.
8. The EEG based motor imagery electroencephalogram classification apparatus of claim 7, wherein the Bi-TCN module comprises:
the two forward TCN residual blocks realize forward sequence processing of sequences output by the SE module, processing results are output to the feature fusion module, the two reverse TCN residual blocks process the sequences output by the SE module after overturning processing in reverse sequence, and processing results are output to the feature fusion module; the forward and reverse TCN residual blocks comprise a depth separable expansion causal convolution layer, a batch normalization layer, an ELU activation function layer and a Dropout layer; the dilation-causal convolution uses progressively increasing dilation factors to skip adjacent time steps to reduce the temporal information of the extracted sequence.
9. Computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the EEG-based motor imagery electroencephalogram classification method of any one of claims 1 to 6 when the computer program is executed.
10. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the EEG-based motor imagery electroencephalogram classification method of any one of claims 1 to 6.
CN202311178742.9A 2023-09-12 2023-09-12 EEG-based motor imagery electroencephalogram classification method, device, equipment and medium Pending CN117150346A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311178742.9A CN117150346A (en) 2023-09-12 2023-09-12 EEG-based motor imagery electroencephalogram classification method, device, equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311178742.9A CN117150346A (en) 2023-09-12 2023-09-12 EEG-based motor imagery electroencephalogram classification method, device, equipment and medium

Publications (1)

Publication Number Publication Date
CN117150346A true CN117150346A (en) 2023-12-01

Family

ID=88886729

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311178742.9A Pending CN117150346A (en) 2023-09-12 2023-09-12 EEG-based motor imagery electroencephalogram classification method, device, equipment and medium

Country Status (1)

Country Link
CN (1) CN117150346A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117725490A (en) * 2024-02-08 2024-03-19 山东大学 Cross-test passive pitch-aware EEG automatic classification method and system

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117725490A (en) * 2024-02-08 2024-03-19 山东大学 Cross-test passive pitch-aware EEG automatic classification method and system
CN117725490B (en) * 2024-02-08 2024-04-26 山东大学 Cross-test passive pitch-aware EEG automatic classification method and system

Similar Documents

Publication Publication Date Title
Abdelhameed et al. A deep learning approach for automatic seizure detection in children with epilepsy
Amin et al. Deep Learning for EEG motor imagery classification based on multi-layer CNNs feature fusion
Li et al. A channel-projection mixed-scale convolutional neural network for motor imagery EEG decoding
Alvarez et al. Learning the number of neurons in deep networks
CN108256629B (en) EEG signal unsupervised feature learning method based on convolutional network and self-coding
CN112244873A (en) Electroencephalogram time-space feature learning and emotion classification method based on hybrid neural network
CN114176607B (en) Electroencephalogram signal classification method based on vision transducer
CN117150346A (en) EEG-based motor imagery electroencephalogram classification method, device, equipment and medium
CN114041795A (en) Emotion recognition method and system based on multi-modal physiological information and deep learning
CN115804602A (en) Electroencephalogram emotion signal detection method, equipment and medium based on attention mechanism and with multi-channel feature fusion
CN112465069A (en) Electroencephalogram emotion classification method based on multi-scale convolution kernel CNN
Taherisadr et al. EEG signal dimensionality reduction and classification using tensor decomposition and deep convolutional neural networks
Deepthi et al. An intelligent Alzheimer’s disease prediction using convolutional neural network (CNN)
Huang et al. Modeling task fMRI data via mixture of deep expert networks
CN112541566B (en) Image translation method based on reconstruction loss
Khalkhali et al. Low latency real-time seizure detection using transfer deep learning
WO2023148471A1 (en) Classification of brain activity signals
CN114224354B (en) Arrhythmia classification method, arrhythmia classification device, and readable storage medium
Zhang et al. Hierarchical model compression via shape-edge representation of feature maps—an enlightenment from the primate visual system
CN114742107A (en) Method for identifying perception signal in information service and related equipment
CN114757273A (en) Electroencephalogram signal classification method based on collaborative contrast regularization average teacher model
CN114283301A (en) Self-adaptive medical image classification method and system based on Transformer
CN116058852B (en) Classification system, method, electronic device and storage medium for MI-EEG signals
CN116304558B (en) Epileptic brain magnetic map spike detection method and device
CN116421152B (en) Sleep stage result determining method, device, equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination