CN113011330A - Electroencephalogram signal classification method based on multi-scale neural network and cavity convolution - Google Patents

Abstract

The invention discloses an electroencephalogram signal classification method based on a multi-scale neural network and cavity convolution, which comprises the following steps: 1, preprocessing an original electroencephalogram signal in a data set, including channel selection and slice segmentation; 2, establishing a classification model based on a multi-scale neural network and cavity convolution, and initializing network parameters; designing a loss function, and establishing a classification model optimization target; and 4, inputting data to train the network, optimizing network parameters and obtaining an optimal classification model. Compared with the traditional method, the method can obviously improve the classification accuracy of the electroencephalogram signals, thereby improving the application value of the electroencephalogram signals in the fields of medical treatment and the like.

Description

Electroencephalogram signal classification method based on multi-scale neural network and cavity convolution

Technical Field

The invention belongs to the field of electroencephalogram signal processing, and particularly relates to an electroencephalogram signal classification method based on a multi-scale neural network and cavity convolution.

Background

Electroencephalogram (EEG) is a powerful tool for recording electroencephalogram activity and can reflect the brain state of a subject, so that the effective electroencephalogram classification method can greatly improve the application value of the electroencephalogram in the fields of medical treatment and the like, and has great research significance. At present, electroencephalogram classification methods can be divided into two categories: traditional methods and methods based on deep learning.

In the traditional method for classifying the electroencephalogram signals, the key point is feature extraction, and features with high discrimination need to be designed manually. The commonly used electroencephalogram signals are mainly characterized by three types: time domain features, frequency domain features, and time-frequency domain features. The time domain features mainly include expectation, variance, Hjorth descriptors, etc., the frequency domain features mainly include spectral energy, wavelet energy, etc., and the time-frequency domain features mainly include wavelet decomposition coefficients, etc. The above features require artificial design of effective features based on signal characteristics, which requires researchers to have in-depth physiological knowledge and limited generalization ability.

In recent years, deep learning can automatically extract features with high discrimination, so that the features are widely applied to classification of electroencephalogram signals. Common electroencephalogram signal classification network structures include a convolutional neural network, a cyclic neural network and the like. The deep learning drives the artificial neural network to automatically extract the characteristics in the signals in a data driving mode, so that the classification is realized, and a better classification effect is obtained in the classification of the electroencephalogram signals.

The electroencephalogram signal is a complex electrical signal with multi-scale property, wherein the multi-scale property means that the electroencephalogram signal comprises various time and space components, in the time dimension, the time scale of some electroencephalograms is millisecond grade, but some electroencephalograms are several seconds long; in the spatial dimension, an abnormal firing of a neuron may be associated with only one electrode channel or may be associated with multiple electrode channels. However, most current electroencephalogram signal classification methods ignore the multi-scale property of electroencephalogram signals and adopt a single-scale research method. The single-scale method is difficult to fully extract the characteristics of the electroencephalogram signals, and the accuracy of classification of the electroencephalogram signals is limited.

Disclosure of Invention

The invention aims to overcome the defects in the prior art, and provides the electroencephalogram signal classification method based on the multi-scale neural network and the cavity convolution, so that the multi-scale characteristics of the electroencephalogram signal can be considered, the space-time multi-scale characteristics of the electroencephalogram signal are extracted, and the accuracy of electroencephalogram signal classification can be improved.

In order to achieve the purpose, the invention adopts the following technical scheme:

the invention discloses an electroencephalogram signal classification method based on a multi-scale neural network and cavity convolution, which is characterized by comprising the following steps of:

step 1, acquiring an electroencephalogram signal data set with labeled categories, and preprocessing original electroencephalogram signals in the data set, wherein the preprocessing comprises channel selection and slice segmentation;

step 1.1, selecting electroencephalogram signals on C channels shared by original electroencephalogram signals from a data set, dividing the electroencephalogram signals into N sections of electroencephalogram signal segments with time length t by using a sliding window method, and recording the N sections of electroencephalogram signal segments as a training sample set X ═ X₁,X₂,...,X_n,...,X_N}；X_nRepresents the nth EEG signal sample, and X_n＝{X_n ¹,X_n ²,…,X_n ^c,…,X_n ^C}，X_n ^cRepresenting the electroencephalogram signal data of the c-th channel in the n-th electroencephalogram signal sample, and X_n ^c＝{x₁ ^c,x₂ ^c,...,x_r ^c,...,x_t×s ^c}；x_r ^cRepresenting the r-th data point of the c-th channel in the n-th electroencephalogram signal sample, and s represents the sampling rate of the electroencephalogram signal in the data set; n is 1,2,. cndot.n; c1, 2,. C; r 1,2, t × s;

step 2, establishing a classification model based on a multi-scale neural network and a cavity convolution, wherein the classification model comprises the following steps: a characteristic extraction stage consisting of a time multi-scale stage and a space multi-scale stage, and a classification stage consisting of a classification module; the time multi-scale stage and the space multi-scale stage are both composed of a multi-scale neural network module and a cavity convolution module;

step 2.1, initializing weight values: initializing parameters of the convolution layer of the classification model;

2.2, the multi-scale neural network module consists of three parallel network branches, each branch comprises two convolution layers and two pooling layers, and convolution kernels of different branches have different sizes so as to extract features of different scales; the cavity convolution module comprises three parallel network branches and an attention mechanism module for feature fusion, and each branch comprises cavity convolution layers with different expansion rates;

step 2.2.1, inputting the training sample set X into a time multi-scale stage, and outputting a first time characteristic sequence through a multi-scale neural network module of the time multi-scale stage

Second time signature sequence

Third temporal signature sequence

Wherein, F_n ¹Represents the firstTime signature sequence F¹N-th feature map of (1), F_n ²Representing a second temporal signature sequence F²N-th feature map of (1), F_n ³Representing a third temporal signature sequence F³The nth feature map of (1);

step 2.2.2, three time characteristic sequences F¹,F²,F³Respectively sending the time characteristic sequence { F) to the cavity convolution modules in the time multi-scale stage, and outputting the time characteristic sequence { F) through the cavity convolution layers with different expansion rates, namely the first time cavity convolution layer, the second time cavity convolution layer and the third time cavity convolution layer^i,1,F^i,2,F^i,31,2,3, where { F | i ═ 1,2,3}, where^i,1I | -1, 2,3} represents a time signature sequence obtained by the i-th time signature sequence passing through the first time hole convolution layer, { F^i,2I | -1, 2,3} represents a time signature sequence obtained by subjecting the ith time signature sequence to the second time hole convolution layer, { F^i,3I is 1,2,3, which represents the time characteristic sequence obtained by the ith time characteristic sequence passing through the third time hole convolution layer;

time feature series F^i,1,F^i,2,F^i,3Sending 1,2 and 3 to an attention mechanism module for feature fusion: first, the time feature sequence { F^i,1,F^i,2,F^i,3Sending the | i ═ 1,2 and 3} into the global pooling layer, and then obtaining a time characteristic sequence { F ] through the first full-connection layer and the second full-connection layer^i,1,F^i,2,F^i,3Time weighting factor { alpha } of | i ═ 1,2,3}^i,1,α^i,2,α^i,3I ═ 1,2,3}, the temporal signature sequence { F^i,1,F^i,2,F^i,3I | ═ 1,2,3} and the temporal weighting factor { α^i,1,α^i,2,α^i,3Correspondingly multiplying | i ═ 1,2,3} to obtain three output time characteristic sequences F of the cavity convolution module_d ¹,F_d ²,F_d ³(ii) a Wherein, F_d ¹，F_d ²，F_d ³Respectively representing a first time scale characteristic sequence, a second time scale characteristic sequence and a third time obtained by the electroencephalogram signal through a time multi-scale stageA sequence of scale features;

step 2.2.3, outputting the time characteristic sequence F_d ¹,F_d ²,F_d ³Sending the data into a multi-scale neural network module at a spatial multi-scale stage, and outputting a first spatial feature sequence

Second spatial feature sequence

Third spatial signature sequence

Wherein the content of the first and second substances,

representing a first sequence of spatial features G¹The n' th feature map in (1),

representing a second spatial signature sequence G²The n' th feature map in (1),

representing a third spatial signature sequence G³The n' th feature map in (1);

step 2.2.4, three spatial feature sequences G¹,G²,G³Respectively sending into the cavity convolution modules in spatial multi-scale stages, and outputting a spatial characteristic sequence { G ] through the cavity convolution layers with different expansion rates, namely a first spatial cavity convolution layer, a second spatial cavity convolution layer and a third spatial cavity convolution layer^i,1,G^i,2,G^i,31,2,3, where G^i,1I | -1, 2,3} represents a spatial signature sequence obtained by subjecting the ith spatial signature sequence to the first spatial hole convolution layer, { G { (G) }^i,2I | -1, 2,3} represents a spatial signature sequence obtained by subjecting the ith spatial signature sequence to the second spatial hole convolution layer, { G { (G) }^i,3I ═ 1,2,3} means that the ith spatial feature sequence passes through the thirdSpatial feature sequences obtained by the spatial cavity convolution layers;

will space the characteristic sequence G^i,1,G^i,2,G^i,3Sending 1,2 and 3 to an attention mechanism module for feature fusion: first, the spatial feature sequence { G }^i,1,G^i,2,G^i,3Sending the | i ═ 1,2 and 3} into the global pooling layer, and then obtaining a spatial feature sequence { G ] through a third full-connection layer and a fourth full-connection layer^i,1,G^i,2,G^i,3Spatial weighting factor { β } of | i ═ 1,2,3}^i,1,β^i,2,β^i,31,2,3, and converting the spatial feature sequence { G | i ═ G |^i,1,G^i,2,G^i,31,2,3 and spatial weighting factor { beta |^i,1,β^i,2,β^i,3Correspondingly multiplying | i ═ 1,2,3} to obtain three output space characteristic sequences G of the cavity convolution module_d ¹,G_d ²,G_d ³(ii) a Wherein G is_d ¹，G_d ²，G_d ³Respectively representing a first spatial scale characteristic sequence, a second spatial scale characteristic sequence and a third spatial scale characteristic sequence of the electroencephalogram signal obtained through a spatial multi-scale stage;

step 2.3, outputting three output space characteristic sequences G_d ¹,G_d ²,G_d ³The probability of the corresponding category of each electroencephalogram signal sample is obtained through the convolution layer, the global pooling layer and the full-connection layer of the classification module;

step 3, adopting focal length as a loss function of the classification model, and updating each weight in the classification model by using an optimization algorithm at a fixed learning rate to enable the loss function to tend to be stable, so as to obtain a trained classification model;

and 4, classifying any electroencephalogram signal sample by using the trained classification model to obtain a probability value of a corresponding class, and performing binarization classification on the probability value according to a set threshold value to obtain a final classification result.

Compared with the prior art, the invention has the beneficial effects that:

1. the invention takes the time-space multi-scale characteristics of the electroencephalogram signal into consideration, constructs the electroencephalogram signal classification method based on the multi-scale neural network and the cavity convolution, and fully excavates the time-space multi-scale characteristics of the electroencephalogram signal. Compared with the method using the same database reported in periodicals, the electroencephalogram signal classification method provided by the invention greatly improves the accuracy of electroencephalogram signal classification;

2. the invention constructs a multi-scale neural network module, and extracts the time-space multi-scale characteristics of the electroencephalogram signal from two dimensions of time and space through convolution kernels with different sizes to obtain electroencephalogram signal characteristics with more discrimination;

3. according to the invention, a cavity convolution module based on an attention mechanism is constructed, so that the receptive field of the network is enlarged, and the global and local information of the electroencephalogram signal is effectively integrated.

Drawings

FIG. 1 is a flow chart of the method of the present invention;

FIG. 2 is an overall framework diagram of the classification model of the present invention;

FIG. 3 is a block diagram of a hole convolution module of the method of the present invention;

FIG. 4 is a schematic diagram of a modular design of the present invention.

Detailed Description

In this embodiment, a method for classifying electroencephalograms based on a multi-scale neural network and a cavity convolution, as shown in fig. 1, includes the following steps:

step 1, acquiring an electroencephalogram signal data set with labeled categories, and preprocessing original electroencephalogram signals in the data set, wherein the preprocessing comprises channel selection and slice segmentation;

step 1.1, selecting electroencephalogram signals on C channels shared by original electroencephalogram signals from a data set, dividing the electroencephalogram signals into N sections of electroencephalogram signal segments with time length t by using a sliding window method, and recording the N sections of electroencephalogram signal segments as a training sample set X ═ X₁,X₂,...,X_n,...,X_N}；X_nRepresents the nth EEG signal sample, and X_n＝{X_n ¹,X_n ²,…,X_n ^c,…,X_n ^C}，X_n ^cRepresenting the electroencephalogram signal data of the c-th channel in the n-th electroencephalogram signal sample, and X_n ^c＝{x₁ ^c,x₂ ^c,...,x_r ^c,...,x_t×s ^c}；x_r ^cRepresenting the r-th data point of the c-th channel in the n-th electroencephalogram signal sample, and s represents the sampling rate of the electroencephalogram signal in the data set; n is 1,2,. cndot.n; c1, 2,. C; r 1,2, t × s;

in specific implementation, for an 18-lead electroencephalogram signal data set, selecting C-18; for the sliding window method, the length of a window can be taken as 4 seconds, and the sliding step length is 2 seconds;

step 2, establishing a classification model based on a multi-scale neural network and a cavity convolution, wherein an overall frame diagram is shown in fig. 2, and the classification model comprises the following steps: a characteristic extraction stage consisting of a time multi-scale stage and a space multi-scale stage, and a classification stage consisting of a classification module; the time multi-scale stage and the space multi-scale stage are both composed of a multi-scale neural network module and a cavity convolution module;

step 2.1, initializing weight values: initializing parameters of the convolution layer of the classification model;

in a specific implementation, the weights in the classification model are initialized using a uniform distribution.

Step 2.2, the multi-scale neural network module is composed of three parallel network branches, each branch comprises two convolution layers and two pooling layers, sizes of convolution kernels of different branches are different so as to extract features of different scales, and exemplarily, in a time multi-scale stage, the size of a two-dimensional convolution kernel is as follows: 32 × 1, 64 × 1, 128 × 1, three convolution kernel sizes corresponding to three time scales; in the spatial multi-scale stage, the size of the two-dimensional convolution kernel is: the sizes of the three convolution kernels are 1 × 2, 1 × 3 and 1 × 5, and correspond to three spatial scales. The hole convolution module comprises three parallel network branches and an attention mechanism module for feature fusion, wherein each branch comprises hole convolution layers with different expansion rates, and the expansion rates are 1,2 and 5 respectively. Specifically, the network structure of the void convolution module is shown in fig. 3, and the network structure of the attention mechanism module is shown in fig. 4;

step 2.2.1, inputting the training sample set X into a time multi-scale stage, and outputting a first time characteristic sequence through a multi-scale neural network module of the time multi-scale stage

Second time signature sequence

Third temporal signature sequence

Wherein, F_n ¹Representing a first time characteristic sequence F¹N-th feature map of (1), F_n ²Representing a second temporal signature sequence F²N-th feature map of (1), F_n ³Representing a third temporal signature sequence F³The nth feature map of (1);

step 2.2.2, three time characteristic sequences F¹,F²,F³Respectively sending the time characteristic sequence { F) to the cavity convolution modules in the time multi-scale stage, and outputting the time characteristic sequence { F) through the cavity convolution layers with different expansion rates, namely the first time cavity convolution layer, the second time cavity convolution layer and the third time cavity convolution layer^i,1,F^i,2,F^i,31,2,3, where { F | i ═ 1,2,3}, where^i,1I | -1, 2,3} represents a time signature sequence obtained by the i-th time signature sequence passing through the first time hole convolution layer, { F^i,2I | -1, 2,3} represents a time signature sequence obtained by subjecting the ith time signature sequence to the second time hole convolution layer, { F^i,3I is 1,2,3, which represents the time characteristic sequence obtained by the ith time characteristic sequence passing through the third time hole convolution layer;

time feature series F^i,1,F^i,2,F^i,3Sending 1,2 and 3 to an attention mechanism module for feature fusion: first, the time feature sequence { F^i,1,F^i,2,F^i,3Sending the information I to a global pooling layer, and obtaining time characteristics through a first full-connection layer and a second full-connection layer with activation functionsSequence { F^i,1,F^i,2,F^i,3Time weighting factor { alpha } of | i ═ 1,2,3}^i,1,α^i,2,α^i,3I ═ 1,2,3}, the temporal signature sequence { F^i,1,F^i,2,F^i,3I | ═ 1,2,3} and the temporal weighting factor { α^i,1,α^i,2,α^i,3Correspondingly multiplying | i ═ 1,2,3} to obtain three output time characteristic sequences F of the cavity convolution module_d ¹,F_d ²,F_d ³(ii) a Wherein, F_d ¹，F_d ²，F_d ³Respectively representing a first time scale characteristic sequence, a second time scale characteristic sequence and a third time scale characteristic sequence of the electroencephalogram signal obtained through a time multi-scale stage.

In a specific implementation, the activation function of the first fully-connected layer is a sigmoid function, and the activation function of the second fully-connected layer is a softmax function.

Step 2.2.3, outputting the time characteristic sequence F_d ¹,F_d ²,F_d ³Sending the data into a multi-scale neural network module at a spatial multi-scale stage, and outputting a first spatial feature sequence

Second spatial feature sequence

Third spatial signature sequence

Wherein the content of the first and second substances,

representing a first sequence of spatial features G¹The n' th feature map in (1),

representing a second spatial signature sequence G²The n' th feature map in (1),

representing a third spatial signature sequence G³The n' th feature map in (1);

step 2.2.4, three spatial feature sequences G¹,G²,G³Respectively sending into the cavity convolution modules in spatial multi-scale stages, and outputting a spatial characteristic sequence { G ] through the cavity convolution layers with different expansion rates, namely a first spatial cavity convolution layer, a second spatial cavity convolution layer and a third spatial cavity convolution layer^i,1,G^i,2,G^i,31,2,3, where G^i,1I | -1, 2,3} represents a spatial signature sequence obtained by subjecting the ith spatial signature sequence to the first spatial hole convolution layer, { G { (G) }^i,2I | ═ 1,2,3} represents a spatial feature sequence obtained by subjecting the ith spatial feature sequence to the second spatial hole convolution layer, { Gi^,3I is 1,2,3, which represents the spatial feature sequence obtained by the ith spatial feature sequence through the third spatial hole convolution layer, i is 1,2, 3;

will space the characteristic sequence G^i,1,G^i,2,G^i,3Sending 1,2 and 3 to an attention mechanism module for feature fusion: first, the spatial feature sequence G^i,1,G^i,2,G^i,3Sending the | i ═ 1,2 and 3} into a global pooling layer, and obtaining a spatial feature sequence { G ] through a third full-connection layer and a fourth full-connection layer with activation functions^i,1,G^i,2,G^i,3Spatial weighting factor { β } of | i ═ 1,2,3}^i,1,β^i,2,β^i,31,2,3, and converting the spatial feature sequence { G | i ═ G |^i,1,G^i,2,G^i,31,2,3 and spatial weighting factor { beta |^i,1,β^i,2,β^i,3Correspondingly multiplying | i ═ 1,2,3} to obtain three output space characteristic sequences G of the cavity convolution module_d ¹,G_d ²,G_d ³(ii) a Wherein G is_d ¹，G_d ²，G_d ³Respectively representing a first spatial scale characteristic sequence, a second spatial scale characteristic sequence and a third spatial scale characteristic sequence of the electroencephalogram signal obtained through a spatial multi-scale stage.

In a specific implementation, the activation function of the third fully-connected layer is a sigmoid function, and the activation function of the fourth fully-connected layer is a softmax function.

Step 2.3, outputting three output space characteristic sequences G_d ¹,G_d ²,G_d ³The probability of the corresponding category of each electroencephalogram signal sample is obtained through the convolution layer, the global pooling layer and the full-connection layer of the classification module;

and 3, adopting focal length as a loss function of the classification model:

wherein n is the number of samples, y_iIs the true label of the sample, y'_iAlpha is used for adjusting the unbalance of positive and negative samples, and gamma is used for adjusting the unbalance of difficult and easy samples. In this embodiment, α is 0.5 and γ is 2.

Updating each weight in the classification model by using an optimization algorithm at a fixed learning rate to enable a loss function to tend to be stable, so as to obtain a trained classification model;

in a specific implementation, the gradient descent method adopted is the Adam method.

And 4, classifying any electroencephalogram signal sample by using the trained classification model to obtain the probability value of the corresponding class, and performing binarization classification on the probability value according to the set threshold value to obtain the final classification result.

In order to illustrate the performance of the scheme, the classification performance of the electroencephalogram signals reported in recent periodical literature is compared with the performance obtained by the scheme, and the compared index adopts an evaluation index commonly used for classification of the electroencephalogram signals, namely sensitivity and false alarm rate, wherein the sensitivity is the ratio of a positive class to all positive classes which are correctly predicted, and the false alarm rate is the average number of times of predicting the negative class into the positive class per hour.

In the specific implementation, four methods reported in journal literature are used for comparison, as shown in table 1: truong et al extract the time-frequency representation of an electroencephalogram signal by using short-time Fourier transform (STFT), classify the electroencephalogram signal by using a Convolutional Neural Network (CNN), and obtain the performances of average sensitivity of 81.2% and average false alarm rate of 0.160/hour; khan et al adopt Wavelet Transform (WT) to obtain wavelet coefficient of EEG signal, classify with convolutional neural network, have achieved the average sensitivity 87.8%, the performance of the average rate of false alarm 0.147/hour; ozcan et al manually designs the characteristics of the electroencephalogram signals and classifies the characteristics by using 3D-CNN, so that the performances of 85.7% of average sensitivity and 0.096/hour of average false alarm rate are obtained; zhang et al extract the features of the EEG signal through the Common Space Pattern (CSP), classify by CNN, have achieved the performance of average sensitivity 92.2%, average false alarm rate 0.120/hour.

TABLE 1 prediction of Performance of different methods on CHB-MIT dataset

	Average sensitivity (%)	Average false alarm rate (/ hour)
			STFT-CNN	81.2	0.160
WT-CNN	87.8	0.147
			3DCNN	85.7	0.096
CSP-CNN	92.2	0.120
			Method of the invention	93.3	0.007

The data sources of the comparison method are all public CHB-MIT data sets, and Table 1 shows the comparison result of the method and the comparison method, the method achieves the average sensitivity of 93.3 percent and the average false alarm rate of 0.007 per hour, and the performance of the method is superior to that of all the comparison methods.

Claims (1)

1. An electroencephalogram signal classification method based on a multi-scale neural network and cavity convolution is characterized by comprising the following steps:

step 1, acquiring an electroencephalogram signal data set with labeled categories, and preprocessing original electroencephalogram signals in the data set, wherein the preprocessing comprises channel selection and slice segmentation;

step 1.1, selecting electroencephalogram signals on C channels shared by original electroencephalogram signals from a data set, dividing the electroencephalogram signals into N sections of electroencephalogram signal segments with time length t by using a sliding window method, and recording the N sections of electroencephalogram signal segments as a training sample set X ═ X₁,X₂,...,X_n,...,X_N}；X_nRepresents the nth EEG signal sample, and X_n＝{X_n ¹,X_n ²,…,X_n ^c,…,X_n ^C}，X_n ^cRepresenting the electroencephalogram signal data of the c-th channel in the n-th electroencephalogram signal sample, and X_n ^c＝{x₁ ^c,x₂ ^c,...,x_r ^c,...,x_t×s ^c}；x_r ^cRepresenting the r-th data point of the c-th channel in the n-th electroencephalogram signal sample, and s represents the sampling rate of the electroencephalogram signal in the data set; n is 1,2,. cndot.n; c1, 2,. C; the ratio of r to 1,2,...,t×s；

step 2, establishing a classification model based on a multi-scale neural network and a cavity convolution, wherein the classification model comprises the following steps: a characteristic extraction stage consisting of a time multi-scale stage and a space multi-scale stage, and a classification stage consisting of a classification module; the time multi-scale stage and the space multi-scale stage are both composed of a multi-scale neural network module and a cavity convolution module;

step 2.1, initializing weight values: initializing parameters of the convolution layer of the classification model;

2.2, the multi-scale neural network module consists of three parallel network branches, each branch comprises two convolution layers and two pooling layers, and convolution kernels of different branches have different sizes so as to extract features of different scales; the cavity convolution module comprises three parallel network branches and an attention mechanism module for feature fusion, and each branch comprises cavity convolution layers with different expansion rates;

step 2.2.1, inputting the training sample set X into a time multi-scale stage, and outputting a first time characteristic sequence through a multi-scale neural network module of the time multi-scale stage

Second time signature sequence

Third temporal signature sequence

Wherein, F_n ¹Representing a first time characteristic sequence F¹N-th feature map of (1), F_n ²Representing a second temporal signature sequence F²N-th feature map of (1), F_n ³Representing a third temporal signature sequence F³The nth feature map of (1);

step 2.2.2, three time characteristic sequences F¹,F²,F³Respectively sent into a cavity convolution module of a time multi-scale stage,and passing through the void convolutional layers with different expansion rates, namely a first time void convolutional layer, a second time void convolutional layer and a third time void convolutional layer, thereby outputting a time characteristic sequence { F^i,1,F^i,2,F^i,31,2,3, where { F | i ═ 1,2,3}, where^i,1I | -1, 2,3} represents a time signature sequence obtained by the i-th time signature sequence passing through the first time hole convolution layer, { F^i,2I | -1, 2,3} represents a time signature sequence obtained by subjecting the ith time signature sequence to the second time hole convolution layer, { F^i,3I is 1,2,3, which represents the time characteristic sequence obtained by the ith time characteristic sequence passing through the third time hole convolution layer;

time feature series F^i,1,F^i,2,F^i,3Sending 1,2 and 3 to an attention mechanism module for feature fusion: first, the time feature sequence { F^i,1,F^i,2,F^i,3Sending the | i ═ 1,2 and 3} into the global pooling layer, and then obtaining a time characteristic sequence { F ] through the first full-connection layer and the second full-connection layer^i,1,F^i,2,F^i,3Time weighting factor { alpha } of | i ═ 1,2,3}^i,1,α^i,2,α^i,3I ═ 1,2,3}, the temporal signature sequence { F^i,1,F^i,2,F^i,3I | ═ 1,2,3} and the temporal weighting factor { α^i,1,α^i,2,α^i,3Correspondingly multiplying | i ═ 1,2,3} to obtain three output time characteristic sequences F of the cavity convolution module_d ¹,F_d ²,F_d ³(ii) a Wherein, F_d ¹，F_d ²，F_d ³Respectively representing a first time scale characteristic sequence, a second time scale characteristic sequence and a third time scale characteristic sequence of the electroencephalogram signal obtained through a time multi-scale stage;

step 2.2.3, outputting the time characteristic sequence F_d ¹,F_d ²,F_d ³Sending the data into a multi-scale neural network module at a spatial multi-scale stage, and outputting a first spatial feature sequence

Second spatial feature sequence

Third spatial signature sequence

Wherein the content of the first and second substances,

representing a first sequence of spatial features G¹The n' th feature map in (1),

representing a second spatial signature sequence G²The n' th feature map in (1),

representing a third spatial signature sequence G³The n' th feature map in (1);

step 2.2.4, three spatial feature sequences G¹,G²,G³Respectively sending into the cavity convolution modules in spatial multi-scale stages, and outputting a spatial characteristic sequence { G ] through the cavity convolution layers with different expansion rates, namely a first spatial cavity convolution layer, a second spatial cavity convolution layer and a third spatial cavity convolution layer^i,1,G^i,2,G^i,31,2,3, where G^i,1I | -1, 2,3} represents a spatial signature sequence obtained by subjecting the ith spatial signature sequence to the first spatial hole convolution layer, { G { (G) }^i,2I | -1, 2,3} represents a spatial signature sequence obtained by subjecting the ith spatial signature sequence to the second spatial hole convolution layer, { G { (G) }^i,3I is 1,2,3, which represents the spatial feature sequence obtained by the ith spatial feature sequence through the third spatial hole convolution layer;

will space the characteristic sequence G^i,1,G^i,2,G^i,3Sending 1,2 and 3 to an attention mechanism module for feature fusion: first, the spatial feature sequence { G }^i,1,G^i,2,G^i,3Sending the | i ═ 1,2,3} into the global pooling layer, and then passing throughThe third full connection layer and the fourth full connection layer obtain a spatial feature sequence { G }^i,1,G^i,2,G^i,3Spatial weighting factor { β } of | i ═ 1,2,3}^i,1,β^i,2,β^i,31,2,3, and converting the spatial feature sequence { G | i ═ G |^i,1,G^i,2,G^i,31,2,3 and spatial weighting factor { beta |^i,1,β^i,2,β^i,3Correspondingly multiplying | i ═ 1,2,3} to obtain three output space characteristic sequences G of the cavity convolution module_d ¹,G_d ²,G_d ³(ii) a Wherein G is_d ¹，G_d ²，G_d ³Respectively representing a first spatial scale characteristic sequence, a second spatial scale characteristic sequence and a third spatial scale characteristic sequence of the electroencephalogram signal obtained through a spatial multi-scale stage;

step 2.3, outputting three output space characteristic sequences G_d ¹,G_d ²,G_d ³The probability of the corresponding category of each electroencephalogram signal sample is obtained through the convolution layer, the global pooling layer and the full-connection layer of the classification module;

step 3, adopting focal length as a loss function of the classification model, and updating each weight in the classification model by using an optimization algorithm at a fixed learning rate to enable the loss function to tend to be stable, so as to obtain a trained classification model;

and 4, classifying any electroencephalogram signal sample by using the trained classification model to obtain a probability value of a corresponding class, and performing binarization classification on the probability value according to a set threshold value to obtain a final classification result.

Images