CN113158964A - Sleep staging method based on residual learning and multi-granularity feature fusion - Google Patents
Sleep staging method based on residual learning and multi-granularity feature fusion Download PDFInfo
- Publication number
- CN113158964A CN113158964A CN202110494434.1A CN202110494434A CN113158964A CN 113158964 A CN113158964 A CN 113158964A CN 202110494434 A CN202110494434 A CN 202110494434A CN 113158964 A CN113158964 A CN 113158964A
- Authority
- CN
- China
- Prior art keywords
- granularity
- hidden
- data
- fusion
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000004927 fusion Effects 0.000 title claims abstract description 68
- 238000000034 method Methods 0.000 title claims abstract description 61
- 238000000605 extraction Methods 0.000 claims abstract description 25
- 239000011159 matrix material Substances 0.000 claims abstract description 23
- 238000007781 pre-processing Methods 0.000 claims abstract description 12
- 230000008569 process Effects 0.000 claims description 23
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 19
- 235000019580 granularity Nutrition 0.000 claims description 18
- 238000011176 pooling Methods 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 15
- 238000005070 sampling Methods 0.000 claims description 13
- 230000008667 sleep stage Effects 0.000 claims description 9
- 230000009467 reduction Effects 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 7
- 230000008447 perception Effects 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 6
- 210000002569 neuron Anatomy 0.000 claims description 5
- 101100421134 Schizosaccharomyces pombe (strain 972 / ATCC 24843) sle1 gene Proteins 0.000 claims description 4
- 239000000284 extract Substances 0.000 claims description 4
- 230000003213 activating effect Effects 0.000 claims description 2
- 238000012549 training Methods 0.000 abstract description 12
- 238000002474 experimental method Methods 0.000 abstract description 11
- 238000012545 processing Methods 0.000 abstract description 6
- 238000003909 pattern recognition Methods 0.000 abstract description 2
- 108091006146 Channels Proteins 0.000 description 35
- 238000012360 testing method Methods 0.000 description 7
- 238000004422 calculation algorithm Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 6
- 238000000354 decomposition reaction Methods 0.000 description 5
- 238000006243 chemical reaction Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 4
- 238000011156 evaluation Methods 0.000 description 4
- 238000002679 ablation Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000003745 diagnosis Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 238000012512 characterization method Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000004070 electrodeposition Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 230000007177 brain activity Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004195 computer-aided diagnosis Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000037053 non-rapid eye movement Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000004461 rapid eye movement Effects 0.000 description 1
- 230000003860 sleep quality Effects 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 230000002618 waking effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/02—Preprocessing
- G06F2218/04—Denoising
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/08—Feature extraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/12—Classification; Matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/15—Biometric patterns based on physiological signals, e.g. heartbeat, blood flow
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention provides a sleep staging method based on residual learning and multi-granularity feature fusion, and belongs to the field of signal processing and pattern recognition. Firstly, preprocessing an original sleep electroencephalogram signal to obtain a plurality of sleep electroencephalogram signal data samples, wherein each data sample comprises data of N channels. And then, HHT is respectively carried out on the N channel data of each data sample to obtain time-frequency matrix characteristics of the N electroencephalogram data. And finally, sending the time-frequency matrix characteristics of all samples into a sleep staging network based on residual learning and multi-granularity characteristic fusion to finish multi-granularity characteristic extraction and classification tasks of the sleep electroencephalogram signals. The method can efficiently fuse different granularity characteristics on the self-adaptive extracted sleep electroencephalogram time-frequency characteristics, and experiments prove that the network training speed provided by the invention is high, the dependence on expert priori knowledge is less, and the classification performance and efficiency of the existing automatic sleep staging method are effectively improved.
Description
Technical Field
The invention relates to the field of signal processing and pattern recognition, in particular to a sleep electroencephalogram signal processing and multi-granularity feature extraction and fusion classification method.
Background
The sleep staging is an important step in the process of sleep quality grading or sleep-related disease diagnosis, and professional doctors need to observe sleep data of patients all night and classify and evaluate sleep stages, which is undoubtedly a diagnosis work which is time-consuming and labor-consuming. With the continuous development of artificial intelligence, the research on improving the working efficiency of doctors by using the computer-aided diagnosis and treatment technology is also in depth. For tasks with heavy workload such as sleep staging, the development and application of an automatic sleep electroencephalogram signal analysis method are necessary to reduce the workload of doctors. An electroencephalogram (EEG) is a kind of bioelectrical signal data that records brain activity and can reflect electrophysiological activity of human brain nerve cells. At present, the study on electroencephalogram signals is one of the hot and leading directions in academia, and mainly comprises the aspects of signal acquisition, signal preprocessing, feature extraction and feature classification. However, the traditional electroencephalogram feature extraction and feature selection based on manual design have great influence on the recognition result of signals, so that the automatic feature extraction and feature classification are concerned in electroencephalogram research.
In some existing automatic sleep staging research methods, the classification means of sleep data are mainly divided into two types: the classification method comprises the steps of directly utilizing a deep learning network to learn preprocessed time series sample data and classify the sample data, utilizing a signal processing algorithm to extract a primary feature from the preprocessed data, and utilizing the deep learning network to further learn high-order features and then classify the data. Since the sleep sample data is a timing signal, in the first classification means, the characteristics learned by the network often reflect the timing information of the signal to a greater extent. However, the significant characteristics of the electroencephalogram signals in different sleep stages are frequency domain information and amplitude information of the signals, so that the classification effect of the method is often not accurate enough. Furthermore, researchers have proposed a second classification means for the above problem, that is, extracting time-frequency domain features from the preprocessed data, and then learning high-order features by using a deep learning network. The method can better reflect the frequency domain and amplitude information of the signal, but more parameter values need to be set in the time-frequency feature extraction process, and the influence of the parameter values on the final result is also great. The parameter setting step usually requires the guidance of a professional doctor or multiple experimental analyses, resulting in poor robustness of the network model.
In order to solve the above problems, the present invention uses Hilbert-Huang Transform (HHT) to adaptively decompose the sleep sample data. The method is a self-adaptive signal processing method for extracting signal time-frequency information, which is proposed by Huang et al. Firstly, an Empirical Mode Decomposition (EMD) algorithm is used for decomposing a signal into a plurality of Intrinsic Mode Functions (IMFs) and a Decomposition margin, and the IMFs are further converted and integrated into a time-frequency information matrix through Hilbert conversion. Compared with short-time Fourier transform and wavelet transform, the method completely analyzes and processes according to the characteristics of the sleep electroencephalogram signals, parameter setting is not needed to be performed manually, and the influence of parameter setting on feature extraction and feature classification is reduced to a great extent. The necessity of applying the method to perform time-frequency feature conversion in the invention has two points: on one hand, compared with time-series sleep sample data, the time-frequency matrix characteristics can better simulate the spectrum and amplitude analysis strategy of a doctor focusing on signals in the sleep stage process; on the other hand, compared with the time-series sleep sample data, the transformed time-frequency matrix characteristics are more fit with the use scene of the convolutional neural network.
In addition, in the study of the automatic sleep staging algorithm of the convolutional neural network related to sleep staging, most networks extract local feature expression by using a local learning strategy, and the local feature is fused by deepening the layer number of the network and the global feature expression is carried out. On one hand, the methods do not fully mine the association between the global information and the local information, on the other hand, the difficulty of network learning is high, and the effectiveness and the robustness of the information with large local variance are lacked. In order to solve the problems, the multi-granularity feature representation in the data is efficiently extracted by constructing a deep fusion network based on residual learning and multi-granularity feature fusion, and the global information and the local information of the sleep data are fully represented by feature fusion, so that the sleep staging effect is improved. As in the task of re-identification of pedestrians, Wang et al combines global and local features via a multi-granular network architecture to improve resolution and identification accuracy. According to the invention, a multi-branch deep network module is set up to be used for extracting multi-granularity characteristics of different frequency domain intervals. The output result of any branch in the module is finally divided into a plurality of sub-areas in the horizontal direction for representing the identification information of different frequency domain intervals, and the local identification information is cooperatively supplemented to the global identification information, so that the multi-granularity characteristic learning strategy is realized. Finally, the recognition performance of the network is effectively improved by fusing different channel characteristics of the sleep electroencephalogram data.
Disclosure of Invention
In order to solve the problems of data parameter setting and how to efficiently fuse the multi-granularity characteristics of sleep sample data of a plurality of channels, the invention provides a sleep staging method based on residual learning and multi-granularity characteristic fusion. After preprocessing operation, the method firstly utilizes HHT to extract time-frequency characteristics to obtain a time-frequency matrix with characterization information. Furthermore, the multi-granularity feature extraction and fusion part respectively extracts and fuses the multi-granularity features of the time-frequency matrixes of the channel data. In the multi-granularity feature extraction and fusion module, firstly, a three-branch residual learning network is used for learning and inputting global information and local information of a time-frequency matrix, and then an attention sensing fusion module is used for fusing feature information with different granularities in a single channel. Finally, in the fusion classification part, feature information among different channels is fused by using an attention perception fusion module. Finally, the method is proved by experiments that the classification precision and the classification performance of the sleep staging task can be effectively improved.
The main ideas for realizing the method are as follows: firstly, performing data preprocessing steps, namely filtering and segmenting operations on original data; further, performing time-frequency domain transformation on the segmented sleep electroencephalogram sample data by using HHT (Hilbert-Huang transform), and extracting time-frequency domain characteristics; then, in the multi-granularity feature extraction and fusion process, a multi-granularity feature extraction and fusion module is used for extracting and fusing different granularity features of the single-channel time-frequency features; and finally, fusing the characteristics of a plurality of channels in the sleep electroencephalogram data for classification in the fusion classification process.
The sleep staging method based on residual learning and multi-granularity feature fusion comprises the following steps:
step one, preprocessing operation of an original sleep electroencephalogram signal. Firstly, performing band-pass filtering on original data with a sampling rate of T and containing N electroencephalogram channels to remove noise; next, the electroencephalogram signal is divided into P pieces of sample data in total without overlapping using a sliding time window having a length of 30 s. Each sample data Si,jAre data comprising N channels, each channel having a data length of 30T, where i ∈ [1,2]Indicates the serial number of the current sample data, j belongs to [1,2]Representing the channel of the current sample data.
And secondly, extracting time-frequency characteristics of the sample data. And performing Hilbert-Huang transform on each sample data obtained in the step one by one according to the channel sequence to extract the time-frequency characteristics of the data. In the step, the time-frequency feature resolution of the sample is set to be M manually, and the other parameters are calculated by algorithm self-adaption, so that the dimension of a matrix of the sample data after time-frequency transformation is N M.
And step three, extracting and fusing multi-granularity features. And (3) the time-frequency matrix is used as the input of the network and is sent into the deep network, and the multi-granularity characteristic of the time-frequency characteristic of each data channel is learned and fused. The time-frequency characteristic data of different channels use the same network architecture, but do not share network parameters. And the network used by each channel data is used for learning and fusing different granularity characteristics of the channel time-frequency characteristics as the significance characteristics of the channel data.
And step four, multi-channel feature fusion and classification. And step three, the significance characteristics of the sleep electroencephalogram data of the N channels can be obtained, the N channels are weighted and fused by the attention sensing fusion module, and then the classification result is obtained by using a classifier.
Compared with the prior art, the method provided by the invention has the following advantages:
the existing automatic sleep staging method is usually based on the characteristics of manual design, the classification performance has strong dependence on parameter value setting and strong dependence on priori knowledge, and sleep electroencephalogram time sequence signals are not suitable for input of a residual convolution neural network. In addition, the general sleep staging method based on deep learning is mostly based on the idea of local learning, and global information and local information are not comprehensively combined. The invention well overcomes the defects through a self-adaptive time-frequency conversion method and a multi-granularity fusion deep network.
Has the advantages that:
the invention creatively combines HHT with multi-granularity feature extraction and feature fusion deep network, utilizes self-adaptive time-frequency conversion and multi-granularity feature learning to simulate the diagnosis idea of distinguishing time-frequency information in data in the sleep stage process of a doctor, reduces the manual design features, simultaneously enables the extracted features to have more representativeness and identifiability, and further improves the classification precision.
Drawings
FIG. 1 is a flow chart of a method of the present invention;
FIG. 2 is a diagram of the overall network framework of the present invention;
FIG. 3 is a schematic diagram of a multi-granular feature extraction and fusion module according to the present invention;
FIG. 4 is a schematic view of an attention-aware fusion module according to the present invention;
fig. 5 is a schematic diagram comparing the results of the classification method according to the present invention with the results of manual classification by a specialist.
Detailed Description
The invention is further described with reference to the following figures and detailed description.
A sleep staging method based on residual learning and multi-granularity feature fusion is disclosed, wherein a flow chart is shown in figure 1, and an overall network framework diagram is shown in figure 2.
Step 1, data preprocessing.
Assume that there is a set of training data TrainData and a set of test data TestData for the method framework. The amount of data used for training contained in TrainData is DtrainThe amount of data for the test contained in the TestData is Dtest. Each piece of data is data with sampling frequency T containing N electroencephalogram channel signals.
And performing band-pass filtering on each piece of electroencephalogram signal data to remove noise in the signal, wherein the filtering range is set to be 0.5-32 Hz. For data with a higher sampling frequency, a down-sampling operation is also adopted to reduce the sampling frequency of the data to 100 Hz. Finally, the filtered data is subjected to non-overlapping segmentation operation with the time window length of 30 seconds, so that sub-segment sample data with the number of P can be obtained. In the sliding segmentation process, since there is no overlapping part between every two adjacent subsections, the sliding window segmentation process of each piece of data can be expressed as
For TrainData, the TrainData can be obtained after preprocessing
For the TestData, the sub-segment sample data can be obtained after preprocessing
The sub-segment sample data is N × T (30 × T).
In the practice of the present invention, the experimental data used involves two data sets. The first related data set is a public data set SLEEP-EDF, SLEEP dual-channel electroencephalogram data are selected for experiments, the electrode positions are Fpz-Cz and Pz-Oz respectively, and the sampling frequency is 100 Hz; the other related data set is a real data set acquired in a hospital, dual-channel sleep electroencephalogram data are selected for experiments, the electrode positions are respectively located at F3-A2 and F4-A1, and the sampling frequency is 1000 Hz.
And 2, extracting time-frequency characteristics.
Defining any segmented sample data S ═ S containing N electroencephalogram data channels1,S2,...,SN]. For each channel data SjExtracting corresponding time-frequency matrix characteristic F by using HHT algorithmjWhere j is an element [1,2]Representing the channel of the current sample data.
HHT transformation of single channel signal data requires the following processing:
(1) single channel signal S using EMD algorithmjThe n intrinsic mode functions IMF and a residual component R are adaptively decomposed. During decomposition, each IMF needs to satisfy the following two conditions: first, the number of extrema and the number of zeros of the signal components must differ by less than or equal to 1; second, the local average of the upper envelope defined by the signal component maximum and the lower envelope defined by the signal component minimum is 0. The decomposition process can be expressed as
Wherein S isj(t) represents the signal-channel signal to be decomposed, IMFu(t) represents the intrinsic mode function IMF, and R (t) represents the decomposition margin of the signal.
(2) For each IMF, the instantaneous frequency and instantaneous amplitude of the IMF are obtained using the Hilbert transform. The instantaneous amplitude is defined herein as au(t) and defining the instantaneous frequency as ωu(t), the calculation of the two variables is as follows:
ωu(t)＝d[arctan(cu(t)/cu(t))]/dt
wherein,
is the u-th IMF signal to perform Hilbert transformAnd (5) changing the result. The Hilbert transform process can be expressed as Hcu(t)]Reconstructed signal zu(t) using instantaneous frequency ωu(t) implementation.
Finally, the timing signal Sj(t) is expressed as the formula H (ω, t) for time, frequency and amplitude by the hilbert-yellow transform:
in the present invention, the square of the amplitude H is used2(omega, t) describes the energy value of the electroencephalogram signal, and the calculated result forms a time-frequency matrix characteristic F ═ F1,F2,...,FN]And artificially setting the time-frequency resolution of the features to be M. The time-frequency characteristics F of the sample data of a single channeljThe size is M × M, and the feature size of the entire sample data is N × M.
And 3, extracting and fusing multi-granularity features.
As can be seen from fig. 2, the sample data F after the above processing is ═ F1,F2,...,FN]Sending the data as input data into a structure containing N network branches to perform time-frequency matrix F aiming at different channelsjThe learning and integration of sleep multi-granular features. The structure and operation steps of each branch network are as follows:
(1) firstly, a single-channel time-frequency feature matrix F with the size of M is usedjAs input, the hidden layer output characteristic hidden _1 is obtained through a1 × 1 convolution layer, a ReLU activation layer and a maximum pooling layerj. This operation is used to expand the number of channels, improve the non-linear learning capability of the network and reduce featuresThe size of the graph.
(2) Then, the hidden layer characteristic hidden _1 output in (1) is usedjAs input to the multi-granularity feature extraction and fusion module of fig. 2. The network diagram of the module is shown in fig. 3, and the module is composed of a multi-granularity residual error learning module and a multi-granularity feature fusion module. The multi-granularity residual learning part mainly takes a residual block ResBlock as a basic unit of the network, and is divided into four branch networks after passing through a residual block ResBlock1 of a main network. The part learns the input data hidden _1 using the branch networksjThe characteristics of different granularity can represent the global and local information of the sleep electroencephalogram time-frequency characteristics. The network branch1 includes two residual blocks (ResBlock2, ResBlock4) and a pooling layer Maxpool 2. The network branch is used for learning input data hidden _1jThe residual block ResBlock2 is set to step size 2 for reducing the dimensionality of the data, resulting in the output result of the branch, hidden _ branch1j. The network branch2 is further divided into two-way network structures after passing through two residual blocks (ResBlock3, ResBlock 4). After one network passes through the pooling layer Maxpool3, the hidden layer output hidden _ branch2 of the network is obtainedjAfter the other network passes through the pooling layer Maxpool4, the obtained output is divided into two fine-grained characteristics with equal size along the horizontal direction to be used as hidden layer output hidden _ branch2_ seg1 of the other networkjAnd hidden _ branch2_ seg2j. The network branch3 shares a similar network structure as the network branch2, which is divided into two paths after passing through two residual blocks (ResBlock3, ResBlock 4). Hidden layer output hidden _ branch3 obtained after one network passes through pooling layer Maxpool3jAfter the other network passes through the pooling layer Maxpool5, the obtained output is divided into three fine-grained characteristics with equal size along the horizontal direction to be used as hidden layer output hidden _ branch3_ seg1 of the other networkj、hidden_branch3_seg2jAnd hidden _ branch3_ seg3j. It is noted that the residual block ResBlock3 is set to step size 1 in branch2 and branch3, and certain acceptance fields are reserved for subsequent fetching of local information. In addition, as can be seen from fig. 3, the multi-granularity residual learning part of the present invention further includes a branch structure including a1 × 1 convolutional layer and a pooling layer Maxpool1, the branch structure is used for performing dimension reduction expression on hidden layer features after passing through a main residual block ResBlock1, and an output result a of the branch structure is added to output results of a total of 8 hidden layers of the three branch networks to be a final output hidden of the multi-granularity residual learning partj,kWhere j is an element [1,2]Represents the channel in which the current output data is located, k ∈ [1,2]Sequence numbers representing the above-mentioned 8 final outputs, respectively, e.g. highj,1＝hidden_branch1j+ A. The main role of this summation operation is to supplement the shallow significance information lost in the multi-granular feature learning process.
(3) Further, 8 high-order multi-granularity characterization features hidden output in the step (2) are usedj,kAttention awareness fusion is performed as an input to the multi-granular feature fusion portion in fig. 3. A schematic diagram of this operation is shown in fig. 3 and 4. As can be seen from FIG. 3, the operation comprises two steps, namely convolution dimensionality reduction and attention perception fusion. In the convolution dimension reduction part, 8 hiddenes are respectively subjected to convolution by 1-by-1j,kAnd (4) performing dimension reduction in the channel dimension direction, and activating the network neurons by using a ReLU activation layer after the convolutional layer while adding a nonlinear factor to the network. The 8 outputs after dimensionality reduction will serve as inputs to the attention sensing fusion module shown in FIG. 4. The module firstly splices 8 input data to obtain a feature map Concat, and then feature matrixes Concat different granularitiesk8 granularity feature channels of the feature map Concat are formed. Then, as can be seen from fig. 4, the module is divided into two paths. The first path is used for learning the data weight of 8 input data, so as to distribute the attention of network convergence. Reducing the size of the feature matrix to 1 on the granularity channel dimension through global average pooling by the network to obtain the directional features GAP representing different granularity featuresk，k∈[1,2,...,8]The granularity characteristic channel where the directivity characteristic is located is expressed as follows:
where S × T represents the size of a feature matrix of a certain granularity, Concatk(u, v) represents the value of each element on the kth granular feature matrix.
Further, the first path of the attention-sensing fusion module is provided with two linear layers for learning the directivity characteristics GAP on different granularity characteristic channelskInformation weight value ofkSetting the activation functions of the two linear layers as a ReLU function and a Sigmoid function respectively, wherein the specific expression of the step is as follows:
weightk＝σ(W2δ(W1GAPk))
wherein, W1And W2Network parameters of two linear layers are respectively expressed, delta (·) and sigma (·) respectively express a ReLU function and a Sigmoid function, and k is equal to [1,2]And representing the granularity characteristic channel where the directivity characteristic and the learned weight value are located.
(4) Finally, it can be seen from FIG. 4 that the second pass of the attention-aware fusion module is an identity mapping operation. Combining two networks of the module, namely, the first learned weight of the characteristic channels with different granularitieskAnd the original input data ConcatkMultiplication allows for the assignment of attention weights to the input data. The invention weighs the scaled data based on attention weightkSumming along granularity channel dimensions to obtain final attention-aware Fusion output Fusionj，j∈[1,2,...,N]。FusionjAnd the final output result of certain channel data in the original N-channel time-frequency characteristics after multi-granularity characteristic extraction and fusion is represented, and all time-frequency information of the channel can be represented.
And 4, fusing and classifying the multichannel characteristics.
From the step 3, we can obtain N single-channel multi-granularity Fusion feature Fusionj. We fuse the characteristics of the N channel datajAs an input, the attention-aware fusion module mentioned in step three (3) is used for fusion between multi-channel features (during this process, the parameters k e [1, 2.,. N ] of the attention-aware fusion module]) And drawing the features obtained after fusion into a one-dimensional vector with the length of outdim. At the end of the whole model framework, the invention sets a linear classification layer with the neuron number of 5 according to the international standard AASM sleep stage standard, and classifies the extracted features into a sleep stage 5, namely a waking stage (W), a non-rapid eye movement stage (NREM, which comprises three types of N1, N2 and N3) and a rapid eye movement stage (REM). In addition, the number and class of sleep stages are not uniform criteria, and the number of classification neurons of the invention can be modified according to actual needs and specific situations.
Training optimization network
In the experimental process, the training set and the test set are divided according to a one-out-of-one verification method, namely, the sleep data of one subject is selected as the test set data, and the sleep data of all the other subjects are combined into the training set data. In the verification process of the model performance, the data of ten randomly selected subjects are respectively used as test data to carry out ten times of cross verification, and the stability of the model is verified.
In the training and optimizing process of the network, the high-order characteristics of single-channel data output by the N multi-granularity branch networks and the characteristics after multi-channel attention perception fusion are respectively constrained by using a plurality of cross entropy Loss functions to correct and adjust the weight and deviation in the network, and the Loss function Loss isceThe definition is as follows:
wherein p isiIs a network output prediction, yiThis indicates whether the prediction result is the same as the label, 1 for the same and 0 for the different. In addition, in the experiment, a Momentum optimizer is used for iteratively updating the network parameters, the Momentum value is set to be 0.4, and the learning rate is set to be 0.001.
The training of the network uses batch data (minipatch) to carry out optimization training, all sample data in a training set or a test set is divided into K minipatches, and each minipatch contains 1/K data of all the sample data. The network calculates the gradient of the sample in each minimatch in the training process, and updates the gradient parameter after averaging. And when the network finishes the cycle training of all K minibands, defining the network to finish a complete epoch. In the present invention, the minimatch data size is set to 512, and the network is set to train 100 epochs.
The network parameter settings for the entire framework are shown in table 1. In the experiment process, the time-frequency resolution M of the HHT is set to be 30.
Table 1 network framework parameter set-up for the method of the invention
Network performance evaluation
The present invention uses evaluation indicators common in several machine learning domains involved in model performance evaluation, including classification Accuracy (Accuracy), coarcapa value (Kappa), F1 values. These evaluation indices were calculated from True Positive (TP), True Negative (TN), False Positive (FP) and False Negative (FN).
Classification accuracy refers to the ratio of the number of correctly predicted samples to the total number of predicted samples.
Kappa is a form of correlation coefficient used to measure the consistency level of the classification estimates of the data set.
The F1 value is used for comprehensively considering two indexes of accuracy and recall value and comprehensively reflecting the indexes of the whole model.
Wherein the two indexes of accuracy and recall value are respectively defined as
Results of the experiment
The effectiveness of the modules of the invention was verified by ablation experiments using SLEEP-EDF data, the results of which are plotted in table 2. The branch1, branch2, and branch3 represent three multi-granularity feature extraction branch networks in the multi-granularity feature extraction module, respectively, and the segrge 2 and segrge 3 represent network models for performing two-branch and three-branch multi-granularity feature fusion using the attention-sensing fusion module proposed in the present invention, respectively.
As shown in table 2, the branch3 network can achieve better classification effect due to richer granularity information in the experimental results of the three branch networks. In the whole ablation experiment result, the average classification accuracy of the three-branch fusion network using the attention-aware fusion strategy is the highest and reaches 85.40%. Secondly, a two-branch multi-granularity fusion network using an attention-aware fusion strategy achieves an average accuracy of 84.93%. The result shows that the multi-granularity feature extraction and fusion module can effectively extract effective features in the input data to complete the sleep staging task, and the classification performance of the network can be further improved by using the attention perception fusion strategy.
TABLE 2 Cross-fold validation results of ablation experiments by the method of the present invention
Table 3 shows the results of the inventive process compared to other prior art model processes. Compared with other methods, the method can obtain better classification precision on the premise of experiments with the same data set and the same number of subjects. In addition, the method verifies the classification effect of the model on real data collected by a hospital. The result shows that the model of the invention can also keep stable and efficient classification performance on a real data set.
TABLE 3 comparison of the Process of the invention with other Processes
In conclusion, the method provided by the invention integrates and classifies the global information and the local information of the sleep electroencephalogram signal by excavating deep multi-granularity characteristics of the sleep electroencephalogram time-frequency information, thereby effectively improving the classification effect of the automatic sleep stage task. The method can obtain ideal experimental results on both the public data set and the real data set acquired by the hospital, and further proves the effectiveness and robustness of the model.
Claims (3)
1. A sleep staging method based on residual learning and multi-granularity feature fusion is characterized by comprising the following steps:
step one, preprocessing an original sleep electroencephalogram signal:
performing band-pass filtering on an original sleep electroencephalogram signal containing N channels to remove noise, and finally obtaining an N sleep-leading electroencephalogram signal with a sampling rate of T; further, dividing the preprocessed signal into P segments of sub-signals S of N channels without overlapping by adopting a sliding time window with the length of 30 secondsi,jWhere i ∈ [1, 2.,. P. ], P]Indicates the serial number of the current sample data, j belongs to [1,2]Representing channels of current sample data, and a signal for each channelEach number contains 30 × T sampling points;
step two, HHT extracts time-frequency characteristics:
for any segmented sample data S containing N sleep electroencephalogram data channelsi＝[Si1,Si2,...,SiN]Wherein S isiN＝[Si,1 ... Si,N]，Si,NI-th data representing an N-th pilot signal. Extracting corresponding time-frequency matrix characteristics from each channel data by using HHT; manually setting the time-frequency resolution of Hilbert-Huang transform to M, and obtaining a time-frequency matrix F ═ F with the characteristic size of any sample data being N × M1,F2,...,FN]Wherein the time-frequency characteristic F of each EEG channeljSize M x M;
step three, multi-granularity feature extraction and fusion:
time-frequency matrix F of each channel by using multi-granularity feature extraction methodjLearning and integrating sleep multi-granularity characteristics to finally obtain output Fusion representing all time-frequency information of each channelj，j∈[1,2,...,N](ii) a The multi-granularity feature extraction method comprises three stages, namely data preprocessing, multi-granularity feature extraction on each single channel by using a multi-granularity residual learning module, and obtaining final attention perception Fusion output Fusion of each single channel by using a multi-granularity feature Fusion modulej；
Step four, multi-channel feature fusion and classification:
utilizing the multi-granularity feature Fusion module in the step three to perform multi-granularity Fusion feature Fusion on N channelsjAnd performing fusion, namely drawing the fused features into a one-dimensional vector which is used as the input of a linear classification layer to perform sleep stage classification.
2. The sleep staging method based on residual learning and multi-granularity feature fusion as claimed in claim 1, wherein the preprocessing of step one includes band-pass filtering and down-sampling, specifically including:
1) performing band-pass filtering on the original electroencephalogram signals at 0.5-32 Hz;
2) and if the sampling frequency of the original electroencephalogram signal is higher than 100Hz, using down-sampling operation to reduce the sampling frequency of the filtered data to 100 Hz.
3. The sleep staging method based on residual learning and multi-granularity feature fusion as claimed in claim 1, wherein the multi-granularity feature extraction method of step three includes the following steps:
1) data preprocessing: for any M x M single-channel time-frequency feature matrix, firstly, a1 x 1 convolution layer, a ReLU activation layer and a maximum pooling layer are respectively passed through to obtain hidden layer output feature hidden _1jThe operation is used for expanding the number of channels, improving the nonlinearity of the deep network and reducing the size of the feature map;
2) hidden layer characteristic hidden _1 output in (1) by utilizing multi-granularity residual error learning modulejAnd performing multi-granularity feature extraction, wherein the multi-granularity residual error learning module takes a residual block ResBlock as a basic unit of a network, and the working process of the multi-granularity residual error learning module is as follows: single channel hidden layer feature hidden _1jAfter a backbone network composed of a residual block ResBlock1 is input, the backbone network is divided into four branch networks, and input data hidden _1 is learned by the four branch networksjThe first branch1 sequentially comprises a residual block ResBlock2, a residual block ResBlock4 and a pooling layer Maxpool2, and is used for learning and outputting input data hidden _1jGlobal information hidden _ branch1jWhere ResBlock2 uses a residual block with step size 2 to reduce the dimensionality of the data; the second branch2 is further divided into two-way network structure after sequentially passing through a residual block ResBlock3 and a residual block ResBlock4, wherein one way network passes through a pooling layer Maxpool3 to obtain a hidden layer output hidden _ branch2 of the way networkjAfter the other network passes through the pooling layer Maxpool4, the obtained output is divided into two fine-grained characteristics with equal size along the horizontal direction to be used as hidden layer output hidden _ branch2_ seg1 of the other networkjAnd hidden _ branch2_ seg2j(ii) a The third branch3 passes throughThe residual block ResBlock3 and the residual block ResBlock4 are divided into two paths, wherein one path of network passes through the hidden layer output hidden _ branch3 obtained by the pooling layer Maxpool3jAfter the other network passes through the pooling layer Maxpool5, the obtained output is divided into three fine-grained characteristics with equal size along the horizontal direction to be used as hidden layer output hidden _ branch3_ seg1 of the other networkj、hidden_branch3_seg2jAnd hidden _ branch3_ seg3j(ii) a Wherein, the residual block ResBlock3 in the branch2 and branch3 is set to step size 1, which is used to reserve a certain acceptance domain for the subsequent extraction of local information; the fourth branch structure sequentially comprises a1 × 1 convolution layer and a maximum pooling layer and is used for carrying out dimensionality reduction expression on hidden layer characteristics passing through a main residual block ResBlock1, and finally, 8 output results of the first three branches are added with output results of the fourth branch respectively to obtain 8 output hiddenj,kWhere j is an element [1,2]，k∈[1,2,...,8]；
3) Obtaining final attention perception Fusion output Fusion of each single channel by utilizing multi-granularity feature Fusion modulejThe working process of the multi-granularity feature fusion module is as follows:
first, to 8-way output hiddenj,kPerforming convolution dimensionality reduction, namely using 1-by-1 convolution to respectively perform convolution on 8 hiddenesj,kPerforming dimension reduction in the channel dimension direction, and activating network neurons by using a ReLU activation layer after the convolutional layer while adding a nonlinear factor to the network;
then, an attention-sensing fusion module is used for completing 8-path hidden through convolution dimensionality reductionj,kThe specific process comprises the following steps: 8 hidden elements after convolution dimensionality reductionj,kPerforming splicing to obtain a feature map, Concat, which is the result of splicing 8 features with different granularities, ConcatkData referring to the kth channel in Concat, k ∈ [1, 2., 8 ∈]Then, the size of the feature matrix is reduced to 1 on the granularity channel dimension through global average pooling to obtain the directional features GAP representing different granularity featureskIt is specifically expressed as follows:
where S × T represents the size of the input feature matrix of a certain channel, Concatk(u, v) represents the value of each element on the k-th channel feature matrix;
two linear layers are set next for learning the directivity characteristics GAP on different granularity characteristic channelskInformation weight value ofkThe activation functions of the two linear layers are respectively a ReLU function and a Sigmoid function, and the information weight value formula is expressed as follows:
weightk＝σ(W2δ(W1GAPk))
wherein, W1And W2Respectively representing the weight values of two linear layers, wherein delta (·) and sigma (·) respectively represent a ReLU function and a Sigmoid function;
finally, to ConcatkThe attention weight is distributed to obtain the data Weighted after the attention weight is scaledkAnd summing along the dimension of the granularity channel to obtain the final attention perception Fusion output Fusion of each single channelj：
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110494434.1A CN113158964B (en) | 2021-05-07 | 2021-05-07 | Sleep stage method based on residual error learning and multi-granularity feature fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110494434.1A CN113158964B (en) | 2021-05-07 | 2021-05-07 | Sleep stage method based on residual error learning and multi-granularity feature fusion |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113158964A true CN113158964A (en) | 2021-07-23 |
| CN113158964B CN113158964B (en) | 2024-05-28 |
Family
ID=76873610
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110494434.1A Active CN113158964B (en) | 2021-05-07 | 2021-05-07 | Sleep stage method based on residual error learning and multi-granularity feature fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113158964B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113723340A (en) * | 2021-09-08 | 2021-11-30 | 湖北理工学院 | Multi-scale attention depth nonlinear factorization method |
| CN115349874A (en) * | 2022-07-12 | 2022-11-18 | 西安电子科技大学 | Multi-granularity information-based rapid sequence visual presentation electroencephalogram signal classification method |
| CN115429293A (en) * | 2022-11-04 | 2022-12-06 | 之江实验室 | Sleep type classification method and device based on impulse neural network |
| CN115607170A (en) * | 2022-11-18 | 2023-01-17 | 中国科学技术大学 | Lightweight sleep staging method based on single-channel electroencephalogram signal and application |
| CN117033916A (en) * | 2023-07-10 | 2023-11-10 | 国网四川省电力公司营销服务中心 | Power theft detection method based on neural network |
| CN117814756A (en) * | 2024-01-10 | 2024-04-05 | 湖北大学 | Feature processing method and system based on deep learning detection model |
| CN118436317A (en) * | 2024-07-08 | 2024-08-06 | 山东锋士信息技术有限公司 | Sleep stage classification method and system based on multi-granularity feature fusion |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109846477A (en) * | 2019-01-29 | 2019-06-07 | 北京工业大学 | A kind of brain electricity classification method based on frequency band attention residual error network |
| CN110693493A (en) * | 2019-10-12 | 2020-01-17 | 北京工业大学 | Epilepsy electroencephalogram prediction method based on convolution and recurrent neural network combined time multiscale |
| CN111325155A (en) * | 2020-02-21 | 2020-06-23 | 重庆邮电大学 | Video motion recognition method based on residual difference type 3D CNN and multi-mode feature fusion strategy |
| AU2020102907A4 (en) * | 2020-10-21 | 2020-12-17 | Ahamad, Danish MR | Novel automated machine learning algorithms based system for sleep staging features analysis |
| CN112115849A (en) * | 2020-09-16 | 2020-12-22 | 中国石油大学(华东) | Video scene identification method based on multi-granularity video information and attention mechanism |
| CN112641451A (en) * | 2020-12-18 | 2021-04-13 | 北方工业大学 | Multi-scale residual error network sleep staging method and system based on single-channel electroencephalogram signal |
-
2021
- 2021-05-07 CN CN202110494434.1A patent/CN113158964B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109846477A (en) * | 2019-01-29 | 2019-06-07 | 北京工业大学 | A kind of brain electricity classification method based on frequency band attention residual error network |
| CN110693493A (en) * | 2019-10-12 | 2020-01-17 | 北京工业大学 | Epilepsy electroencephalogram prediction method based on convolution and recurrent neural network combined time multiscale |
| CN111325155A (en) * | 2020-02-21 | 2020-06-23 | 重庆邮电大学 | Video motion recognition method based on residual difference type 3D CNN and multi-mode feature fusion strategy |
| CN112115849A (en) * | 2020-09-16 | 2020-12-22 | 中国石油大学(华东) | Video scene identification method based on multi-granularity video information and attention mechanism |
| AU2020102907A4 (en) * | 2020-10-21 | 2020-12-17 | Ahamad, Danish MR | Novel automated machine learning algorithms based system for sleep staging features analysis |
| CN112641451A (en) * | 2020-12-18 | 2021-04-13 | 北方工业大学 | Multi-scale residual error network sleep staging method and system based on single-channel electroencephalogram signal |
Non-Patent Citations (2)
| Title |
|---|
| 吕甜甜;王心醉;俞乾;贾朋飞;陈骁;吴成雄;: "基于多参数特征融合的自动睡眠分期方法", 计算机应用, no. 2, 20 December 2017 (2017-12-20) * |
| 袁韶祖;王雷全;吴春雷;: "基于多粒度视频信息和注意力机制的视频场景识别", 计算机系统应用, no. 05, 15 May 2020 (2020-05-15) * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113723340A (en) * | 2021-09-08 | 2021-11-30 | 湖北理工学院 | Multi-scale attention depth nonlinear factorization method |
| CN113723340B (en) * | 2021-09-08 | 2023-05-30 | 湖北理工学院 | Depth nonlinear factorization method for multi-scale attention |
| CN115349874A (en) * | 2022-07-12 | 2022-11-18 | 西安电子科技大学 | Multi-granularity information-based rapid sequence visual presentation electroencephalogram signal classification method |
| CN115429293A (en) * | 2022-11-04 | 2022-12-06 | 之江实验室 | Sleep type classification method and device based on impulse neural network |
| CN115429293B (en) * | 2022-11-04 | 2023-04-07 | 之江实验室 | Sleep type classification method and device based on impulse neural network |
| CN115607170A (en) * | 2022-11-18 | 2023-01-17 | 中国科学技术大学 | Lightweight sleep staging method based on single-channel electroencephalogram signal and application |
| CN115607170B (en) * | 2022-11-18 | 2023-04-25 | 中国科学技术大学 | Lightweight sleep staging method based on single-channel electroencephalogram signals and application |
| CN117033916A (en) * | 2023-07-10 | 2023-11-10 | 国网四川省电力公司营销服务中心 | Power theft detection method based on neural network |
| CN117814756A (en) * | 2024-01-10 | 2024-04-05 | 湖北大学 | Feature processing method and system based on deep learning detection model |
| CN118436317A (en) * | 2024-07-08 | 2024-08-06 | 山东锋士信息技术有限公司 | Sleep stage classification method and system based on multi-granularity feature fusion |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113158964B (en) | 2024-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113158964A (en) | Sleep staging method based on residual learning and multi-granularity feature fusion | |
| Aslan et al. | Automatic Detection of Schizophrenia by Applying Deep Learning over Spectrogram Images of EEG Signals. | |
| CN111012336B (en) | Parallel convolutional network motor imagery electroencephalogram classification method based on spatio-temporal feature fusion | |
| CN108491077B (en) | Surface electromyographic signal gesture recognition method based on multi-stream divide-and-conquer convolutional neural network | |
| CN114052735B (en) | Deep field self-adaption-based electroencephalogram emotion recognition method and system | |
| CN110353702A (en) | A kind of emotion identification method and system based on shallow-layer convolutional neural networks | |
| CN110163180A (en) | Mental imagery eeg data classification method and system | |
| CN109299751B (en) | EMD data enhancement-based SSVEP electroencephalogram classification method of convolutional neural model | |
| CN113128552B (en) | Electroencephalogram emotion recognition method based on depth separable causal graph convolution network | |
| CN104771163A (en) | Electroencephalogram feature extraction method based on CSP and R-CSP algorithms | |
| CN109598222B (en) | EEMD data enhancement-based wavelet neural network motor imagery electroencephalogram classification method | |
| CN101221554A (en) | Brain wave characteristic extraction method based on wavelet translation and BP neural network | |
| CN112465069B (en) | Electroencephalogram emotion classification method based on multi-scale convolution kernel CNN | |
| CN108280414A (en) | A kind of recognition methods of the Mental imagery EEG signals based on energy feature | |
| CN111931656B (en) | User independent motor imagery classification model training method based on transfer learning | |
| CN110522412A (en) | Method based on multiple dimensioned brain function network class EEG signals | |
| CN113180659A (en) | Electroencephalogram emotion recognition system based on three-dimensional features and cavity full convolution network | |
| CN117195099A (en) | Electroencephalogram signal emotion recognition algorithm integrating multi-scale features | |
| Ranjani et al. | Classifying the autism and epilepsy disorder based on EEG signal using deep convolutional neural network (DCNN) | |
| CN116211319A (en) | Resting state multichannel electroencephalogram signal identification method based on graph attention network and sparse coding | |
| CN115919330A (en) | EEG Emotional State Classification Method Based on Multi-level SE Attention and Graph Convolution | |
| CN114305452B (en) | Cross-task cognitive load identification method based on electroencephalogram and field adaptation | |
| CN111603158A (en) | Fatigue driving warning method and system based on electrophysiological signal artificial intelligence analysis | |
| CN115251909A (en) | Electroencephalogram signal hearing assessment method and device based on space-time convolutional neural network | |
| CN112259228B (en) | Depression screening method by dynamic attention network non-negative matrix factorization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |