CN115017960A - Electroencephalogram signal classification method based on space-time combined MLP network and application - Google Patents
Electroencephalogram signal classification method based on space-time combined MLP network and application Download PDFInfo
- Publication number
- CN115017960A CN115017960A CN202210849649.5A CN202210849649A CN115017960A CN 115017960 A CN115017960 A CN 115017960A CN 202210849649 A CN202210849649 A CN 202210849649A CN 115017960 A CN115017960 A CN 115017960A
- Authority
- CN
- China
- Prior art keywords
- electroencephalogram
- layer
- sample
- mlp
- sequence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 238000012549 training Methods 0.000 claims abstract description 26
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 9
- 238000001914 filtration Methods 0.000 claims abstract description 4
- 238000013145 classification model Methods 0.000 claims abstract description 3
- 230000006870 function Effects 0.000 claims description 18
- 239000000126 substance Substances 0.000 claims description 16
- 239000011159 matrix material Substances 0.000 claims description 13
- 230000009467 reduction Effects 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 10
- 230000004913 activation Effects 0.000 claims description 9
- 210000004556 brain Anatomy 0.000 claims description 7
- 238000011176 pooling Methods 0.000 claims description 7
- 230000009466 transformation Effects 0.000 claims description 7
- 238000010606 normalization Methods 0.000 claims description 5
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 abstract description 4
- 238000005070 sampling Methods 0.000 abstract description 2
- 238000013136 deep learning model Methods 0.000 abstract 1
- 239000010410 layer Substances 0.000 description 74
- 238000000537 electroencephalography Methods 0.000 description 8
- 238000013135 deep learning Methods 0.000 description 6
- 230000002452 interceptive effect Effects 0.000 description 5
- 230000000875 corresponding effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000002123 temporal effect Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000002349 favourable effect Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000002346 layers by function Substances 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 108010076504 Protein Sorting Signals Proteins 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000008909 emotion recognition Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000007917 intracranial administration Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 210000004761 scalp Anatomy 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 238000010792 warming Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/14—Fourier, Walsh or analogous domain transformations, e.g. Laplace, Hilbert, Karhunen-Loeve, transforms
- G06F17/141—Discrete Fourier transforms
- G06F17/142—Fast Fourier transforms, e.g. using a Cooley-Tukey type algorithm
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Optimization (AREA)
- Mathematical Analysis (AREA)
- Computational Mathematics (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Discrete Mathematics (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention discloses an electroencephalogram classification method based on a space-time combined MLP network and application thereof, and the method comprises the following steps: 1, preprocessing original electroencephalogram data, including selection of data to be classified, sliding window slicing, data up-sampling and selection of data input shapes; 2, establishing a deep learning model of the multilayer perceptron network; 3, in a training stage, inputting data and continuously optimizing model parameters through cross entropy loss to obtain a final classification model for classifying the electroencephalogram signals to be tested; and 4, calibrating the prediction result sequence of the model by using a moving average filtering algorithm. The invention applies the time-space information of the electroencephalogram data to the multilayer sensor network, and can obviously improve the classification accuracy of the electroencephalogram signals, thereby increasing the application value of the electroencephalogram signals in the fields of medical treatment and the like.
Description
Technical Field
The invention relates to the field of electroencephalogram signal classification, in particular to a method for predicting and classifying electroencephalogram signals by combining an MLP (multi-level label layer) network of time and space information of multi-channel electroencephalogram signals.
Background
Electroencephalography (EEG) is a physiological technique for recording physiological electrical signals of the brain. The recognition and prediction of physiological and psychological states from patterns of neural activity observed in the scalp and intracranial electroencephalograms are widely used in the field of brain-computer interfaces such as emotion recognition, motor imagery, medical health, and the like. Conventional machine learning methods, such as auto-regressive coefficients and Lyapunov indices, which are manually extracted, have been used with some success in a tightly controlled experimental environment. However, these manually extracted features often require researchers to have extensive expertise and to perform a large number of experimental attempts. In addition, in real electroencephalogram recording influenced by various artifacts, manually extracted features often only cover part of electroencephalogram information, so that the system has poor robustness.
The deep learning algorithm is stimulated to be widely applied to electroencephalogram classification and prediction due to the excellent generalization ability and the strong ability of automatically learning high-efficiency characteristics. At present, most deep learning methods for classification of electroencephalogram signals carry out feature preprocessing, such as short-time Fourier transform, public space mode and the like. These preprocessing operations on the original brain wave, while resulting in more "clean" data, may also lose some important information. Models that use feature pre-processing and directly use raw brain signals in recent years typically have more complex architectures and larger kernels, resulting in greater memory resource consumption and computational power requirements.
At present, most deep learning algorithms for classification of electroencephalogram signals are generally used as feature classifiers. Researchers extract time domain features, frequency domain features or time-frequency domain features from electroencephalogram signals through existing professional knowledge, and then use a deep learning algorithm to perform classification tasks. Although this method also achieves good classification performance, it requires deep mathematical knowledge for extracting features, and it neglects the strong data driving capability of deep learning algorithms. The simultaneously extracted features, although being a better data representation to some extent, also lose much of the spatial correlation information and temporal correlation information present in the original multi-channel electroencephalogram data. There are also a few deep learning algorithms that use an end-to-end architecture, but none of them fully utilize the spatial correlation information and temporal correlation information present in multi-channel brain electrical signals. Due to the limitation of various conditions, the total data amount of the electroencephalogram signals is seriously insufficient, so that the development of the classification method of the electroencephalogram signals is greatly limited. And because the electroencephalogram data of different types have the serious data imbalance problem, the development of the classification method is greatly limited.
Disclosure of Invention
The invention aims to overcome the defects and provides the electroencephalogram signal classification method based on the space-time joint MLP network and the application thereof, so that the spatial correlation information and the time correlation information of multi-channel electroencephalograms can be extracted from the original electroencephalogram signals and applied to the multilayer perceptron network, the electroencephalogram signal classification accuracy can be obviously improved, and the application value of the electroencephalogram signals in the fields of medical treatment and the like is increased.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention relates to an electroencephalogram classification method based on a space-time combined MLP network, which is characterized by comprising the following steps:
step 1, acquiring an electroencephalogram signal data set with labeled category information, performing sliding slicing processing on the electroencephalogram signal of each category, reconstructing the input shape of the sliced electroencephalogram signal, obtaining N segments of electroencephalogram signal samples with total duration T, and recording the N segments of electroencephalogram signal samples as a training sample set X{X 1 ,X 2 ,...,X i ,...,X N And denote the label set of the training sample X as Y ═ Y 1 ,Y 2 ,...,Y i ,...,Y N }; wherein, X i ∈R C×1×L Representing the ith section of electroencephalogram signal sample after input shape reconstruction, C representing the number of channels of the electroencephalogram signal sample, and L representing the length of the electroencephalogram signal sample; y is i As the ith segment of EEG signal sample X i A corresponding label;
step 2, establishing a space-time combination-based MLP network, comprising: the system comprises a denoising weighting module, a space-time joint MLP module and a classification module;
step 2.1, the denoising weighting module comprises: a denoising layer, a weighting layer and a dimensionality reduction layer;
the de-noising layer comprises a manually set matrix filter with randomly initialized element values, and firstly, the de-noising layer converts a training sample set X (X) through fast Fourier transform 1 ,X 2 ,...,X i ,...,X N Converting the time domain into the frequency domain, multiplying the training sample set transformed into the frequency domain by a learnable matrix filter to obtain a denoised training sample set, and transforming the denoised training sample set into the time domain through inverse fast Fourier transform to obtain a time domain denoised electroencephalogram sample sequence
Wherein the content of the first and second substances,
representing the denoised i-th segment of time domain electroencephalogram signal sample;
the brain electrical signal sequence
Converting from three dimensions to two dimensions to obtain a two-dimensional de-noised electroencephalogram sample sequence
Wherein the content of the first and second substances,
representing a two-dimensional ith segment of electroencephalogram signal sample;
the weighting layer comprises a channel weight matrix which is preset manually and can learn diagonal element values. The weighting layer firstly carries out two-dimensional electroencephalogram sample sequence
Multiplying the weighted sequence by the channel weight matrix to obtain a weighted sequence
Wherein the content of the first and second substances,
representing the ith segment of electroencephalogram signal sample after channel weighting;
the dimensionality reduction layer comprises a group of 1 xk convolution kernels and weights the channel to the electroencephalogram sample sequence
The redundant information is removed in the time dimension (length dimension) to obtain the redundancy-removed electroencephalogram sample sequence
Wherein the content of the first and second substances,
representing the ith section of electroencephalogram sample after redundant information is removed;
step 2.2, the space-time combined MLP module comprises: an inter-channel MLP layer and an intra-channel MLP layer;
the inter-channel MLP layer sequentially comprises: a layer norm layer, a transformation full-link layer, a GELU nonlinear activation function and a restoration full-link layer;
layer pair redundancy removal electroencephalogram sample sequence
After normalization processing, the spatial correlation electroencephalogram samples are obtained after processing of the transformation full-connection layer, the GELU activation function and the restoration full-connection layer in sequenceSequence of
Wherein the content of the first and second substances,
representing the ith segment of electroencephalogram signal sample which is extracted, integrated and associated with the channel space;
the intra-channel MLP layer and the inter-channel MLP layer have the same structure, and are used for spatially correlated electroencephalogram sample sequences
After normalization processing, the time information electroencephalogram sample sequence is obtained after processing of the transformation full-connection layer, the GELU activation function and the restoration full-connection layer in sequence
Wherein the content of the first and second substances,
representing the ith section of electroencephalogram signal sample extracted through time information in the channel;
step 2.3, the classification module comprises: an averaging pooling layer, a full link layer and a Softmax layer;
time information electroencephalogram sample sequence
After the average pooling layer and the full-connection layer are sequentially processed, the score of each electroencephalogram signal sample corresponding to each category is obtained, finally, the score of each electroencephalogram signal sample corresponding to each category is converted into the probability value of each category through the Softmax layer, and the maximum probability value is selected as the prediction classification result of each electroencephalogram signal sample;
step 3, model training:
based on the training sample set X and the label set Y thereof, adopting cross entropy as a loss function, training the space-time combined MLP network by using an ADAM optimizer, and calculating the gradient of the loss function to update network parameters until the maximum iteration times or the loss function convergence is reached, thereby obtaining a trained electroencephalogram signal classification model;
step 4, calibrating the prediction result sequence of the model by using a moving average filtering algorithm:
taking the ith segment of electroencephalogram sample X i And its following M-1 segment of electroencephalogram samples { X i+1 ,X i+2 ,...,X i+M-1 The mean value of probability values of each category corresponding to each electroencephalogram signal sample in the data is correspondingly used as the X of the ith sample i Each category probability value of (1).
The electronic device comprises a memory and a processor, and is characterized in that the memory is used for storing programs for supporting the processor to execute the electroencephalogram classification method, and the processor is configured to execute the programs stored in the memory.
The invention relates to a computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when executed by a processor, performs the steps of the electroencephalogram classification method.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention provides a new end-to-end electroencephalogram signal classification method based on an MLP network, which designs a denoising weighting module before electroencephalogram signal classification, wherein the denoising weighting module comprises three independent functional layers: the denoising layer is used for reducing various artifact noises in the original electroencephalogram signal; the weighting layer automatically learns channels more favorable for making correct inference from the multi-channel electroencephalogram, and strengthens the function of the channels favorable for correct classification by giving larger weight; the dimensionality reduction layer is used for eliminating redundant information in the electroencephalogram signals, so that the robustness of the electroencephalogram signal classification result is improved.
2. Aiming at the characteristics of multi-channel electroencephalogram signals, the invention designs a space-time combined MLP module which comprises two independent functional layers: the inter-channel MLP layer is used for extracting and integrating spatial correlation existing among different channel electroencephalogram signals; the intra-channel MLP layer is used for extracting time information of each channel of electroencephalogram data, and time and space associated information of electroencephalogram signals are extracted and integrated, so that the generalization performance of the model is remarkably improved.
Drawings
FIG. 1 is a schematic diagram of a network architecture according to the present invention;
FIG. 2 is a schematic diagram of a denoising weighting module according to the present invention;
FIG. 3 is a schematic diagram of a spatiotemporal interaction MLP module according to the present invention;
FIG. 4 is a schematic diagram of the input and output of the de-noising layer of the present invention (data from the CHB-MIT data set subject 5).
Detailed Description
In the embodiment, the electroencephalogram signal classification method based on the space-time interaction MLP network firstly reduces noise in the electroencephalogram signals through a denoising weighting module, strengthens the effect of the electroencephalogram signals of important channels, and rejects redundant information in the electroencephalogram signals; extracting spatial correlation and time information of the integrated multi-channel electroencephalogram through a space-time interactive MLP module; the data sequence is converted into a prediction result through a classification module, as shown in fig. 1, specifically, the method comprises the following steps:
step 1, acquiring an electroencephalogram signal data set with labeled category information, performing data selection on the electroencephalogram signals to obtain electroencephalogram signals of C channels, slicing the electroencephalogram signals of the C channels through a sliding window, reconstructing the input shape of the sliced electroencephalogram signals, and accordingly obtaining N sections of electroencephalogram signal samples with the total duration of T, and recording as a training sample set X ═ { X ═ X 1 ,X 2 ,...,X i ,...,X N And set Y of labels for training sample X to { Y ═ Y } 1 ,Y 2 ,...,Y i ,...,Y N }; wherein X i ∈R C ×1×L Representing the ith section of electroencephalogram signal sample after input shape reconstruction, C representing the number of channels of the electroencephalogram signal, and L representing the length of the electroencephalogram signal sample; y is i Is the ith segment of EEG signal sample X i In the embodiment, the number of channels of the electroencephalogram signal is 22, the length of the sliding window is 30s, and the sampling rate of the electroencephalogram signal per second is 256 Hz; the method uses two publicationsAn electroencephalogram data set: a CHB-MIT dataset and a Kaggle dataset;
step 2, establishing a space-time joint based MLP network, the structure of which is shown in FIG. 1 and comprises: the system comprises a denoising weighting module, a space-time joint MLP module and a classification module;
as shown in fig. 2, the denoising weighting module includes: a denoising layer, a weighting layer and a dimensionality reduction layer.
The denoising layer comprises a fast Fourier transform, an inverse fast Fourier transform and a learnable matrix filter; the weighting layer comprises a learnable channel weight matrix; the dimensionality reduction layer comprises a set of convolution kernels;
as shown in fig. 3, the spatiotemporal union MLP module includes: inter-channel MLP layer and intra-channel MLP layer.
The inter-channel MLP layer and the intra-channel MLP layer have the same structure and comprise a layer norm layer, a transformation full-connection layer, a GELU nonlinear activation function and a restoration full-connection layer, and the difference is that the inter-channel MLP layer acts on the space dimension of electroencephalogram data, and the intra-channel MLP layer acts on the time dimension of electroencephalogram signals;
the classification module comprises: an averaging pooling layer, a fully connected layer and a Softmax layer.
Step 2.1, initializing the model parameters:
before training, randomly initializing all weight parameters in the network;
step 2.2, the reconstructed slice sample X i ∈R C×1×L Inputting the data into a network, and obtaining the data after the data is processed by a denoising weighting module
Wherein, C r And L r Respectively representing the number and the length of channels of the processed electroencephalogram samples;
in this embodiment, the denoising weighting module is implemented as follows:
firstly, brain electrical data X is changed into X 1 ,X 2 ,...,X i ,...,X N Converting the frequency domain EEG signal sequence into the frequency domain through Fast Fourier Transform (FFT)
Wherein the content of the first and second substances,
representing the ith segment of EEG signal sample of frequency domain, multiplying the EEG data of frequency domain by a manually preset matrix filter W with learnable element value 1 Then obtaining a frequency domain denoising sequence
Wherein the content of the first and second substances,
the frequency domain denoised electroencephalogram signal sample of the ith segment is represented, the matrix filter can effectively reduce high-frequency noise artifacts in the electroencephalogram signal, as shown in fig. 4, the output of a denoising layer is smoother compared with the input, and high-frequency components are effectively removed.
Converting the denoised frequency domain data into a time domain through Inverse Fast Fourier Transform (IFFT) to obtain a time domain denoising sequence
Wherein
Representing an ith time domain denoised electroencephalogram signal sample; then three-dimensionally
Is removed to obtain two-dimensional
And then has a two-dimensional denoising sequence
Inputting the two-dimensional noise reduction sequence into a weighting layer, multiplying the two-dimensional noise reduction sequence by a channel weight diagonal matrix which is preset manually and can learn diagonal element values (each diagonal element value corresponds to a channel weight value of a channel)To obtain a channel weighting sequence
Wherein the content of the first and second substances,
the weighting layer gives greater weight to electroencephalogram channels containing important information to strengthen the role of the electroencephalogram channels in electroencephalogram classification, a part of channel weight values learned by the weighting layer are shown in table 3, channels with the learned channel weight values being more than or equal to 0.6 are defined as important channels, and the channels have 6 important channels for the 21 st subject in the CHB-MIT data set and have more important significance for model deduction.
Table 3 shows the weight values of the weighting layers of the present invention (data derived from CHB-MIT data set subject 21).
| channel | LOOCV-1 | LOOCV-2 | LOOCV-3 | LOOCV-4 |
| u’FP1-F7’ | 0.233 | 0.230 | 0.225 | 0.213 |
| u’F7-T7’ | 0.212 | 0.210 | 0.206 | 0.202 |
| u’T7-P7’ | 0.202 | 0.200 | 0.198 | 0.191 |
| u’P7-O1’ | 0.600 | 0.594 | 0.594 | 0.591 |
| u’FP1-F3’ | 0.519 | 0.512 | 0.507 | 0.501 |
| u’F3-C3’ | 0.543 | 0.536 | 0.533 | 0.533 |
| u’C3-P3’ | 0.899 | 0.899 | 0.895 | 0.894 |
| u’P3-O1’ | 0.215 | 0.216 | 0.219 | 0.220 |
| u’FP2-F4’ | 0.294 | 0.290 | 0.289 | 0.286 |
| u’F4-C4’ | 0.302 | 0.300 | 0.300 | 0.297 |
| u’C4-P4’ | 0.658 | 0.657 | 0.655 | 0.653 |
| u’P4-O2’ | 0.122 | 0.136 | 0.142 | 0.146 |
| u’FP2-F8’ | 0.429 | 0.418 | 0.413 | 0.402 |
| u’F8-T8’ | 0.207 | 0.206 | 0.201 | 0.198 |
| u’T8-P8’ | 0534 | 0.533 | 0.533 | 0.531 |
| u’P8-O2’ | 0.407 | 0.420 | 0.426 | 0.432 |
| u’FZ-CZ’ | 0.788 | 0.797 | 0.806 | 0.818 |
| u’CZ-PZ’ | 0.433 | 0.433 | 0.434 | 0.435 |
| u’P7-T7’ | 0.948 | 0.944 | 0.937 | 0.929 |
| u’T7-FT9’ | 0.317 | 0.318 | 0.316 | 0.315 |
| u’FT9-FT10’ | 0.811 | 0.811 | 0.804 | 0.803 |
| u’FT10-T8’ | 0.508 | 0.509 | 0.502 | 0.492 |
Finally, the electroencephalogram signal sample after channel weighting is input to a dimensionality reduction layer, a large amount of redundant information in electroencephalogram data is eliminated by using one-dimensional convolution, the robustness of the model is improved, and a redundancy removal sequence is obtained
Wherein
Representing the electroencephalogram signal sample with the i-th segment removed with redundant information, and calculating as shown in formula (1):
in the formula (1), w 1 ,w 2 ,…,w n Representing a learnable channel weight, Conv1D represents a one-dimensional convolution.
Step 2.3, will remove redundant sequence
Input into a spatio-temporal joint MLP module to extract integersThe time-space combination characteristic is implemented as follows: firstly inputting a redundancy removing sequence into an inter-channel MLP layer, firstly standardizing the redundancy removing sequence through a layer standardization (LN) operation, then expanding the number of channels of the redundancy removing sequence through a conversion full-connection layer, then carrying out nonlinear mapping on data by using a nonlinear activation function GELU (global warming) to improve the nonlinear fitting capability of a network on electroencephalogram signals, and finally restoring the expanded number of channels to the number of channels before expansion through a restoration full-connection layer, and realizing the extraction and integration of spatial correlation among channels of the multichannel electroencephalogram signals through the above operations to obtain a spatial correlation sequence
Wherein
Representing the ith section of electroencephalogram signal sample which is extracted and integrated with the spatial correlation between channels;
the intra-channel MLP layer and the inter-channel MLP layer have similar internal structures, except that the inter-channel MLP layer operates on the channel dimension (spatial dimension) of the input data, and the intra-channel MLP layer operates on the length dimension (temporal dimension) of the input data. Through similar operation in the MLP layer between the channels, the MLP layer in the channels extracts time information in each channel of the multi-channel electroencephalogram signals to obtain a time information sequence
Wherein
Representing the ith section of electroencephalogram signal sample after time information extraction, and calculating by the space-time joint MLP module as shown in formula (2):
in the formula (2), W 2 ,W 3 ,W 4 ,W 5 Indicating full connectivity, LN indicates layer normalization, and σ indicates the GELU nonlinear activation function.
Step 2.4,
After being processed by the classification module, the ith section of electroencephalogram sample X is output i So that the final classification result can be obtained; the specific process is as follows:
extracting time information electroencephalogram sample sequence of spatial correlation and time information
Inputting the data into a classification module, and obtaining a time dimension reduction sequence after the data is processed by an average pooling layer
Wherein the content of the first and second substances,
representing the I-th time dimension (length dimension) reduced EEG sample, and then reducing the time dimension sequence through the full connection layer
After space dimension reduction, a category score sequence is obtained
Wherein the content of the first and second substances,
representing each category score of the ith electroencephalogram sample, and finally outputting the ith electroencephalogram sample X through a Softmax layer i Selecting the class with the highest probability as a final classification result, wherein the calculation process is shown as formula (3):
in the formula (3), f averagepooling (. represents average pooling f) softmax (. for) Softmax
Step 3, model training:
in the model training stage, cross entropy is used as a loss function, an ADAM optimizer is used for optimizing the network, and the gradient of the loss function is calculated and used for updating the network weight parameters until the maximum iteration times are reached; in the example, the number of samples trained each time is set to be 16, the initial learning rate of the ADAM optimizer is set to be 0.001, and a model with the lowest verification loss in a test set is selected as an optimal model;
step 4, calibrating the model:
in practical situations, another electroencephalogram signal does not appear in the continuous electroencephalogram signals. However, the condition that another isolated electroencephalogram signal appears in the continuous electroencephalogram signals is easy to appear in the electroencephalogram signal classification task. To solve this problem, a moving average filtering algorithm is used to calibrate the sequence of predictions for the model. Taking the ith segment of electroencephalogram sample X i And the following M-1 segment of electroencephalogram sample { X i+1 ,X i+2 ,...,X i+M-1 Taking the mean value of the probabilities of each category as X of the ith sample i The probability of each class in (c) is calculated as shown in equation (4).
In the formula (4), p i+k Indicates the class probabilities for the i + k sample,
indicating the calculated class probabilities for the ith sample.
The application example of the space-time interactive MLP network is electroencephalogram type prediction based on electroencephalogram signals, experiments are respectively carried out on two public data sets of CHB-MIT and Kaggle, a representative CNN model and a traditional MLP model are used as comparison models, and the performance of the proposed network is verified. In this example, four widely used evaluation indexes were used to measure model performance. The sensitivity (Sn) is the ratio of the number of times of correctly predicting the electroencephalogram and the total number of times of appearing the electroencephalogram, which are obtained by adopting a k-threshold method (continuous k certain predictions mean that the electroencephalogram is detected); the false alarm rate (FPR) is defined as the number of times of wrong prediction per hour; AUC is an important index for accurately balancing the model prediction performance, the AUC value of the random classifier is 0.5, and the AUC value of the perfect classifier can reach 1; the p value can be used for measuring whether the model is superior to a stochastic classifier, and when the p value (p-value) is less than 0.05, the model is obviously superior to the stochastic classifier under the significance level of 0.05. Tables 1 and 2 show the behavior of the spatio-temporal interactive MLP network of the present invention and the two comparative models on the CHB-MIT data set and the Kaggle data set, respectively. Compared with the traditional CNN model, the space-time combined MLP network provided by the invention realizes a better prediction result by using approximately equal parameters and a shallower network structure, and compared with the traditional MLP network, the space-time combined MLP network provided by the invention realizes a better prediction result by using less than 1/4 parameters, so that the validity of the space-time interactive MLP network provided by the invention is fully shown, the space relation and time information are fully extracted, the classification prediction performance is improved, and very excellent performances are obtained on both a CHB-MIT data set and a Kaggle data set, which shows that the space-time interactive MLP network provided by the invention has better identification capability and stronger generalization performance on electroencephalogram categories of a subject.
TABLE 1 average Performance of contrast model on CHB-MIT database for electroencephalogram classification
TABLE 2 average performance of contrast models on Kaggle database for classification of EEG signals
In conclusion, the method can effectively reduce high-frequency noise artifacts in the original electroencephalogram signals, strengthen the effect of important channels on making correct inference, effectively remove redundant information in the electroencephalogram signals, and improve the identification robustness of the model on the original electroencephalogram data; meanwhile, abundant time information and spatial correlation in the electroencephalogram signals can be fully utilized, the accuracy of the prediction result of the model is greatly improved, a remarkable prediction result is obtained in the prediction of the two-classification electroencephalogram types on the data sets CHB-MIT and Kaggle, and meanwhile, the false alarm rate of the electroencephalogram type prediction is effectively reduced; and because the MLP network has a simple structure, the spatio-temporal joint MLP network provided by the invention is easier to train and has less time for training and testing compared with a CNN network.
Claims (3)
1. An electroencephalogram classification method based on a space-time combined MLP network is characterized by comprising the following steps:
step 1, acquiring an electroencephalogram signal data set with labeled category information, performing sliding slicing processing on the electroencephalogram signal of each category, reconstructing the input shape of the sliced electroencephalogram signal, obtaining N segments of electroencephalogram signal samples with total time length T, and recording as a training sample set X ═ { X ═ X 1 ,X 2 ,...,X i ,...,X N And recording the label set of the training sample X as Y ═ Y 1 ,Y 2 ,...,Y i ,...,Y N }; wherein, X i ∈R C×1×L Representing the ith section of electroencephalogram signal sample after input shape reconstruction, C representing the number of channels of the electroencephalogram signal sample, and L representing the length of the electroencephalogram signal sample; y is i As the ith segment of EEG signal sample X i A corresponding label;
step 2, establishing a space-time combination-based MLP network, comprising: the system comprises a denoising weighting module, a space-time joint MLP module and a classification module;
step 2.1, the denoising weighting module comprises: a denoising layer, a weighting layer and a dimensionality reduction layer;
the de-noising layer comprises a manually set matrix filter with randomly initialized element values, and firstly, the de-noising layer converts a training sample set X (X) through fast Fourier transform 1 ,X 2 ,...,X i ,...,X N Converting the time domain to the frequency domain, multiplying the training sample set transformed to the frequency domain by a learnable matrix filter to obtain a denoised training sample set, and then performing inverse fast Fourier transform on the denoised training sample setConverting the training sample set into a time domain to obtain a time domain denoising electroencephalogram sample sequence
Wherein, X i d ∈R C×1×L Representing a denoised i-th segment of time domain electroencephalogram signal sample;
the brain electrical signal sequence
Converting from three dimensions to two dimensions to obtain a two-dimensional de-noised electroencephalogram sample sequence
Wherein the content of the first and second substances,
representing a two-dimensional ith segment of electroencephalogram signal sample;
the weighting layer comprises a channel weight matrix which is manually preset and can be learned by diagonal element values; the weighting layer firstly carries out two-dimensional electroencephalogram sample sequence
Multiplying the weighted sequence by the channel weight matrix to obtain a weighted sequence
Wherein the content of the first and second substances,
representing the ith segment of electroencephalogram signal sample after channel weighting;
the dimensionality reduction layer comprises a group of 1 xk convolution kernels and weights the channel to the electroencephalogram sample sequence
The redundant information is removed in the time dimension (length dimension) to obtain the redundancy-removed electroencephalogram sample sequence
Wherein the content of the first and second substances,
representing the ith section of electroencephalogram sample after removing redundant information;
step 2.2, the space-time combined MLP module comprises: an inter-channel MLP layer and an intra-channel MLP layer;
the inter-channel MLP layer sequentially comprises: a layer norm layer, a transformation full-link layer, a GELU nonlinear activation function and a restoration full-link layer;
layer pair redundancy removal electroencephalogram sample sequence
After normalization processing, the spatial correlation electroencephalogram sample sequence is obtained after processing of the transformation full-connection layer, the GELU activation function and the restoration full-connection layer in sequence
Wherein the content of the first and second substances,
representing the ith segment of electroencephalogram signal sample which is extracted, integrated and associated with the channel space;
the intra-channel MLP layer and the inter-channel MLP layer have the same structure, and are used for spatially correlating electroencephalogram sample sequences
After normalization processing, the time information electroencephalogram sample sequence is obtained after processing of the transformation full-connection layer, the GELU activation function and the restoration full-connection layer in sequence
Wherein the content of the first and second substances,
representing the ith section of electroencephalogram signal sample extracted through time information in the channel;
step 2.3, the classification module comprises: an averaging pooling layer, a full link layer and a Softmax layer;
time information electroencephalogram sample sequence
After the average pooling layer and the full-connection layer are sequentially processed, the score of each electroencephalogram signal sample corresponding to each category is obtained, finally, the score of each electroencephalogram signal sample corresponding to each category is converted into the probability value of each category through the Softmax layer, and the maximum probability value is selected as the prediction classification result of each electroencephalogram signal sample;
step 3, model training:
based on the training sample set X and the label set Y thereof, adopting cross entropy as a loss function, training the space-time combined MLP network by using an ADAM optimizer, and calculating the gradient of the loss function to update network parameters until the maximum iteration times or the loss function convergence is reached, thereby obtaining a trained electroencephalogram signal classification model;
step 4, calibrating the prediction result sequence of the model by using a moving average filtering algorithm:
taking the ith section of electroencephalogram sample X i And its following M-1 segment of electroencephalogram samples { X i+1 ,X i+2 ,...,X i+M-1 The mean value of probability values of each category corresponding to each electroencephalogram signal sample in the data is correspondingly used as the X of the ith sample i Each category probability value of (1).
2. An electronic device comprising a memory and a processor, wherein the memory is configured to store a program that enables the processor to perform the electroencephalogram classification method of claim 1, and the processor is configured to execute the program stored in the memory.
3. A computer-readable storage medium, having a computer program stored thereon, wherein the computer program, when executed by a processor, performs the steps of the electroencephalogram classification method of claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210849649.5A CN115017960B (en) | 2022-07-19 | 2022-07-19 | Electroencephalogram signal classification method based on space-time combined MLP network and application |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210849649.5A CN115017960B (en) | 2022-07-19 | 2022-07-19 | Electroencephalogram signal classification method based on space-time combined MLP network and application |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115017960A true CN115017960A (en) | 2022-09-06 |
| CN115017960B CN115017960B (en) | 2024-02-13 |
Family
ID=83081323
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210849649.5A Active CN115017960B (en) | 2022-07-19 | 2022-07-19 | Electroencephalogram signal classification method based on space-time combined MLP network and application |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115017960B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117547286A (en) * | 2023-12-29 | 2024-02-13 | 中国人民解放军东部战区总医院 | Electroencephalogram signal data analysis management system based on intelligent repair material |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200368491A1 (en) * | 2019-05-24 | 2020-11-26 | Neuroenhancement Lab, LLC | Device, method, and app for facilitating sleep |
| US20210138249A1 (en) * | 2016-04-22 | 2021-05-13 | Newton Howard | System and method for neural stimulation using spike frequency modulation |
| CN114124260A (en) * | 2022-01-11 | 2022-03-01 | 中国人民解放军国防科技大学 | Spectrum prediction method, apparatus, medium, and device based on composite 2D-LSTM network |
| CN114564991A (en) * | 2022-02-28 | 2022-05-31 | 合肥工业大学 | Electroencephalogram signal classification method based on Transformer guide convolution neural network |
| CN114564990A (en) * | 2022-02-28 | 2022-05-31 | 合肥工业大学 | Electroencephalogram signal classification method based on multi-channel feedback capsule network |
-
2022
- 2022-07-19 CN CN202210849649.5A patent/CN115017960B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210138249A1 (en) * | 2016-04-22 | 2021-05-13 | Newton Howard | System and method for neural stimulation using spike frequency modulation |
| US20200368491A1 (en) * | 2019-05-24 | 2020-11-26 | Neuroenhancement Lab, LLC | Device, method, and app for facilitating sleep |
| CN114124260A (en) * | 2022-01-11 | 2022-03-01 | 中国人民解放军国防科技大学 | Spectrum prediction method, apparatus, medium, and device based on composite 2D-LSTM network |
| CN114564991A (en) * | 2022-02-28 | 2022-05-31 | 合肥工业大学 | Electroencephalogram signal classification method based on Transformer guide convolution neural network |
| CN114564990A (en) * | 2022-02-28 | 2022-05-31 | 合肥工业大学 | Electroencephalogram signal classification method based on multi-channel feedback capsule network |
Non-Patent Citations (2)
| Title |
|---|
| 蓝天;彭川;李森;叶文政;李萌;惠国强;吕忆蓝;钱宇欣;刘峤;: "单声道语音降噪与去混响研究综述", 计算机研究与发展, no. 05 * |
| 郑纯军;王春立;贾宁;: "语音任务下声学特征提取综述", 计算机科学, no. 05 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117547286A (en) * | 2023-12-29 | 2024-02-13 | 中国人民解放军东部战区总医院 | Electroencephalogram signal data analysis management system based on intelligent repair material |
| CN117547286B (en) * | 2023-12-29 | 2024-05-28 | 中国人民解放军东部战区总医院 | Electroencephalogram signal data analysis management system based on intelligent repair material |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115017960B (en) | 2024-02-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107844755B (en) | Electroencephalogram characteristic extraction and classification method combining DAE and CNN | |
| CN112001306A (en) | Electroencephalogram signal decoding method for generating neural network based on deep convolution countermeasure | |
| CN112800928B (en) | Epileptic seizure prediction method of global self-attention residual error network integrating channel and spectrum characteristics | |
| CN114947883B (en) | Deep learning electroencephalogram noise reduction method based on time-frequency domain information fusion | |
| CN114564990A (en) | Electroencephalogram signal classification method based on multi-channel feedback capsule network | |
| CN115804602A (en) | Electroencephalogram emotion signal detection method, equipment and medium based on attention mechanism and with multi-channel feature fusion | |
| CN114648048B (en) | Electrocardiosignal noise reduction method based on variational self-coding and PixelCNN model | |
| CN111387975A (en) | Electroencephalogram signal identification method based on machine learning | |
| CN110543831A (en) | brain print identification method based on convolutional neural network | |
| CN114595725B (en) | Electroencephalogram signal classification method based on addition network and supervised contrast learning | |
| CN114841191A (en) | Epilepsia electroencephalogram signal feature compression method based on fully-connected pulse neural network | |
| CN115017960B (en) | Electroencephalogram signal classification method based on space-time combined MLP network and application | |
| CN114595728A (en) | Signal denoising method based on self-supervision learning | |
| CN113128384B (en) | Brain-computer interface software key technical method of cerebral apoplexy rehabilitation system based on deep learning | |
| CN117193537A (en) | Double-branch convolutional neural network motor imagery intention decoding method based on self-adaptive transfer learning | |
| CN116236209A (en) | Method for recognizing motor imagery electroencephalogram characteristics of dynamics change under single-side upper limb motion state | |
| CN115293210A (en) | Instruction prediction output control method based on brain waves | |
| CN115429286A (en) | Electrocardio representation self-supervision learning method | |
| CN115270847A (en) | Design decision electroencephalogram recognition method based on wavelet packet decomposition and convolutional neural network | |
| CN111493864B (en) | EEG signal mixed noise processing method, equipment and storage medium | |
| CN114692682A (en) | Method and system for classifying motor imagery based on graph embedding representation | |
| Berthelot et al. | Filtering-based analysis comparing the DFA with the CDFA for wide sense stationary processes | |
| CN112149724B (en) | Electroencephalogram data feature extraction method based on intra-class compactness | |
| CN117725490B (en) | Cross-test passive pitch-aware EEG automatic classification method and system | |
| CN115130524A (en) | Electroencephalogram signal classification method based on Ghost network and unbalanced learning and application |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |