CN110069958B - Electroencephalogram signal rapid identification method of dense deep convolutional neural network - Google Patents
Electroencephalogram signal rapid identification method of dense deep convolutional neural network Download PDFInfo
- Publication number
- CN110069958B CN110069958B CN201810057413.1A CN201810057413A CN110069958B CN 110069958 B CN110069958 B CN 110069958B CN 201810057413 A CN201810057413 A CN 201810057413A CN 110069958 B CN110069958 B CN 110069958B
- Authority
- CN
- China
- Prior art keywords
- layer
- convolution
- output
- input
- pooling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 40
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000013528 artificial neural network Methods 0.000 claims abstract description 10
- 238000011176 pooling Methods 0.000 claims description 39
- 230000006870 function Effects 0.000 claims description 28
- 230000004913 activation Effects 0.000 claims description 20
- 238000005070 sampling Methods 0.000 claims description 11
- 238000010586 diagram Methods 0.000 claims description 10
- 238000009795 derivation Methods 0.000 claims description 5
- 230000008859 change Effects 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 3
- 238000012952 Resampling Methods 0.000 claims description 2
- 230000001902 propagating effect Effects 0.000 claims description 2
- 238000012360 testing method Methods 0.000 abstract description 18
- 238000004422 calculation algorithm Methods 0.000 abstract description 9
- 239000000284 extract Substances 0.000 abstract description 5
- 238000012549 training Methods 0.000 description 23
- 238000012795 verification Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 6
- 238000013461 design Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000002829 reductive effect Effects 0.000 description 4
- 210000004556 brain Anatomy 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 230000000638 stimulation Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000000763 evoking effect Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 210000002569 neuron Anatomy 0.000 description 2
- 230000036961 partial effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000007177 brain activity Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 230000004424 eye movement Effects 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 210000004761 scalp Anatomy 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/12—Classification; Matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/03—Recognition of patterns in medical or anatomical images
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method for quickly identifying electroencephalogram (EEG) signals by an intensive deep convolutional neural network, which is designed by combining the characteristics of time and space characteristics of motor imagery electroencephalogram signals and using a characteristic connection method in the convolutional neural network. The convolutional neural network designed by the invention can simultaneously extract time and space characteristics, and also can interconnect the outputs of different convolutional layers, thereby reducing the number of weights and achieving the purposes of overfitting resistance and characteristic reuse. Firstly, inputting filtered and resampled original data into a dense deep convolution neural network, then updating parameters of each layer of the network through a back propagation and random gradient descent algorithm, finally testing the network, inputting test data into a trained network, and analyzing an output result. Compared with the Shallow ConvNet method proposed in 2017, the signal identification accuracy and kappa value of the method are improved by 5% and 0.066.
Description
Technical Field
The invention relates to rapid identification of an original electroencephalogram signal, and is suitable for design, mode classification and deep learning of a convolutional neural network of the electroencephalogram signal, belonging to the technical field of signal processing and mode identification.
Background
The brain-computer interface (BCI) technology can establish connection between the human brain and external equipment, and achieves the purpose of communicating and controlling with the external environment independent of human muscles. The main processing procedures of the BCI technology include recording brain activity, Electroencephalogram (EEG) signal processing, signal recognition, and then controlling an external device according to the recognition result. At present, the types of electroencephalogram signals suitable for BCI are various, such as P300, steady-state visual evoked potential, motor imagery and the like. The P300 is an electroencephalogram signal induced by an occasional small-probability flicker signal, the stimulation of a visual evoked potential is a flicker picture with a fixed frequency, the motor imagery signal only needs a subject to imagine to execute the action of a certain part and does not need to be executed really, the motor imagery electroencephalogram signal has the characteristics of easiness in acquisition, no need of external stimulation, capability of realizing asynchronous communication and the like, and therefore the motor imagery electroencephalogram signal becomes one of the most applied EEG signal types at present.
For feature extraction of motor imagery electroencephalogram signals, Common Spatial Pattern (CSP) methods are common, but the feature extraction effect of the algorithm depends on the frequency bandwidth range specified by the algorithm. On the basis, the Filter Bank Common Spatial Pattern algorithm (FBCSP) provided by the method designs a Filter Bank, enlarges the frequency range, uses the CSP algorithm for each Filter, and selects available features from the output of the Filter Bank, so that the algorithm obtains a classification effect with the accuracy of 68% on a BCIIV datasets 2a data set. However, the FBCSP algorithm still needs prior knowledge to design the range of each frequency bandwidth, and needs to classify after manually extracting features, however, brain signals are very complex, many collected signals have no clear meaning yet, and information loss is caused by manually extracting features.
In recent years, with the rise of deep learning, a Convolutional Neural Network (CNN) has attracted wide attention of researchers, and has achieved certain application effects in many fields such as images, voices, and videos, and therefore, researchers have begun to use the convolutional neural network to automatically extract and classify characteristics of motor imagery electroencephalograms. The weight sharing network structure of the convolutional neural network can reduce the number of weights, reduce the complexity of a network model and extract time and space characteristics. In the training process, the neural network updates the parameters of each convolution kernel by means of a back propagation algorithm, changes different layers into proper feature extractors, can avoid using the manually designed feature extractors, and therefore extracts more features and achieves the effect of improving the classification accuracy. Different from a two-dimensional static picture, the motor imagery electroencephalogram signal is a dynamic time sequence acquired from a three-dimensional brain scalp, and the signal to noise ratio of the motor imagery electroencephalogram signal is low, so that the motor imagery electroencephalogram signal is easily interfered by noise irrelevant to an event, such as disturbance of an electrode, light stimulation, eye movement of a subject and the like, and the training of a convolutional neural network from the original motor imagery electroencephalogram signal becomes difficult, so that the structure of the network needs to be adjusted according to the characteristics of an EEG signal when the convolutional neural network is designed. In 2017, the probability that the input original motor imagery signals belong to each category can be obtained through operations such as time convolution, space convolution, average pooling and the like by taking the original motor imagery electroencephalogram signals as input, and the method is an end-to-end automatic identification method. The end-to-end convolutional neural network does not need any prior knowledge, obtains characteristics from learning of the original motor imagery electroencephalogram signal, and directly obtains a final classification result. The Shallow ConvNet structure is used on the BCIIV2a data set, the accuracy rate reaches 72% which is 5% higher than that of the traditional FBCSP method, and the recognition accuracy rate of the motor imagery electroencephalogram signals can be obviously improved by using the convolutional neural network. However, the accuracy of the model is still not high, and because the used convolutional layer has only two layers, and the direct deepening of the model can cause severe overfitting, deeper features cannot be extracted. The invention researches how to combine the characteristics of the motor imagery electroencephalogram signals and adjust the connection mode and the hyper-parameters of the model, avoids aggravating overfitting while deepening the depth of the model, and improves the identification accuracy of the motor imagery electroencephalogram signals.
Disclosure of Invention
The invention provides a dense deep convolutional neural network on the basis of Shallow ConvNet. The accuracy of the method is 77% obtained by testing on the BCIIV2a data set, and compared with the accuracy of 72% obtained by a Shallow ConvNet network on the BCIIV2a data set, the accuracy of the method is still 5% higher, so that the accuracy of the method for recognizing the motor imagery electroencephalogram signals is obviously improved. Compared with Shallow ConvNet, the dense connection method provided by the invention connects the input and output characteristic diagrams of the middle convolutional layers as the input of the next layer, so that the characteristic diagrams generated by the two convolutional layers can be directly transmitted to the next layer, and the characteristics of the middle layer are fully utilized under the condition of not increasing the number of parameters, so that the model provided by the invention has higher accuracy.
The invention designs an intensive depth convolution neural network to realize end-to-end identification of an original motor imagery electroencephalogram signal, which comprises the following steps:
carrying out third-order band-pass 0-40hz filtering on an electroencephalogram signal, and filtering noise or other irrelevant components during signal acquisition;
resampling the filtered signals, wherein the data length of the input convolutional neural network needs to be kept consistent, and the data volume under the same time length needs to be ensured to be the same, so that the data with different sampling frequencies need to be sampled to the same frequency;
thirdly, intercepting the events with fixed length from the preprocessed data and acquiring corresponding labels;
the dense deep convolutional neural network is designed, and different from a conventional convolutional neural network in continuous convolution and pooling operations, the dense deep convolutional neural network is designed, and input and output characteristic graphs of two convolutional layers in the middle are connected and then used as input of the next layer;
fifthly, training the dense deep convolutional neural network. Calculating the error between the predicted value and the label by using a square error function as a loss function, updating the parameters of each layer of the network through a back propagation and random gradient descent algorithm, and stopping training when the accuracy rate converges to a certain value or the accuracy rate declines;
sixthly, testing the dense deep convolutional neural network, inputting test data and a label and analyzing an output result, wherein parameters of the network are not changed any more.
The specific steps of the step four are as follows:
data input layer: the format of the input data is a four-dimensional array n multiplied by m multiplied by 990 multiplied by 22, and the meaning of each dimension of the array is sample number multiplied by feature image number multiplied by sampling point number multiplied by channel number;
time convolution layer: carrying out convolution operation on data in a time dimension, wherein the size of a convolution kernel is 11 multiplied by 1, and generating 25 characteristic graphs;
③ space convolution layer: in order to reduce the data dimension, performing spatial convolution on the data, mapping all channels into a feature map, wherein the size of a convolution kernel is 22 multiplied by 1, entering an activation function after convolution, and the number of the feature maps is still 25;
a pooling layer: the input is the output of the upper layer activation function, the size of the pooling kernel is 3 multiplied by 1, and the step length is 3;
a characteristic connecting layer: the pooled data enters two continuous convolution pooling layers, the size of a first convolution kernel is 1 multiplied by 1, the number of the first convolution kernel is 200, the size of a second convolution kernel is 11 multiplied by 1, the number of the second convolution kernel is 50, and 50 output feature maps obtained after the second convolution layer is activated are connected with 25 input feature maps of the first convolution layer, so that 75 feature maps are output;
a pooling layer: the operation is performed simultaneously;
the last two layers of the network are a full connection layer and an output layer, the full connection layer expands the output of the upper layer after being pooled into one-dimensional data, the labels are of four types, so that the output layer has four neurons, and the output result is a probability value that the input data belongs to each type.
It should be noted that after each convolution operation of the network, a batch regularization operation is performed on data once and then an activation function is input, and Ioffe and the like prove that the number of iterations can be obviously reduced by using the batch regularization operation in the deep neural network, and meanwhile, the calculation time required by each iteration is reduced.
The step fife specifically comprises the following steps:
firstly, separating 20% of training set of BCIIV2a data set as verification set, and remaining 80% of training set, and keeping the testing set unchanged;
and secondly, training for the first time, namely, training iteratively on a training set at first, and then testing on a verification set. And stopping training when the test accuracy rate on the verification set does not change any more or the accuracy rate is reduced, recording the accuracy rate which is best represented on the verification set at the moment, and storing the model. Through multiple experiments, the model accuracy rate is not changed after the iteration times of the data set reach 1000 times, so that the maximum iteration time of the BCIIV2a data set is set to be 1000;
and thirdly, training for the second time, mixing the training set and the verification set together to be used as a new training set, and testing on the verification set. Continuing training on the new training set, and testing on the original verification set until the accuracy on the verification set is higher than the recorded value and does not change any more, or stopping training after 1000 iterations are reached;
and fourthly, storing the model after the second training is finished, inputting the test data and the label, and recording the test result.
The motor imagery-oriented electroencephalogram signal rapid identification method provided by the invention has the advantages that:
the method is an end-to-end method, does not need time-frequency analysis on signals, does not need prior knowledge to manually select characteristics, and therefore information loss is avoided;
compared with a Shallow ConvNet model, the dense convolutional neural network provided by the invention has more layers and can extract deeper features;
the invention provides a characteristic connection layer, connects the input and output of the middle convolution layer and transmits the input and output to the next layer, thereby not only effectively solving the problem of gradient disappearance, but also supporting characteristic reuse without increasing the number of parameters, and effectively inhibiting overfitting when the data volume is small;
the model generalization capability is strong, and the method can be applied to similar motor imagery electroencephalogram signal data sets only by changing the parameters of the input and output layers and finely adjusting some hyper-parameters of other layers according to different data sets.
Drawings
Fig. 1 is a diagram of a dense deep convolutional neural network structure according to the present invention.
FIG. 2 is a flow chart of the data processing, training and testing process of the present invention.
FIG. 3 is a classification result confusion matrix for Shallow ConvNet on BCIIV2 a.
Fig. 4 is a classification result confusion matrix of the network of the present invention on BCIIV2 a.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and specific embodiments. The method mainly utilizes the weight sharing and local receptive field thought of the convolutional neural network to connect the channels of the intermediate layer output characteristic diagram to improve the identification accuracy. The same convolution kernel is adopted when each neuron of the convolution neural network performs convolution on different feature maps, so that the weight parameter quantity is greatly reduced, and after the convolution operation, the input and the output of the convolution are connected, so that the feature is repeatedly utilized, and only few new feature maps are generated after the convolution, thereby achieving the purpose of reducing the redundancy. The dense deep convolution neural network disclosed by the invention carries out back propagation on errors by using a random gradient descent method, adjusts the weight of a convolution kernel, and finally obtains the probability that input data belongs to each category through full connection and a linear classification layer.
The invention discloses a convolutional neural network feature extraction and classification method, which specifically comprises the following steps:
the method includes acquiring data. The data of the present invention are derived from data set BCI composition IV Dataset2a provided by berlin BCI research group 2008. The data set comprises 9 groups, wherein the data set is respectively collected from 9 healthy subjects and divided into two parts, namely training data and testing data. For each subject, four types of motor imagery of left hand, right hand, feet and tongue are performed;
preprocessing data, wherein the sampling frequency is 100Hz, a 3-order Butterworth filter in a Scipy library filter toolbox is used, and a 0-40Hz band-pass filter is arranged to filter high-frequency signals and partial noise;
thirdly, designing a dense deep convolution neural network, wherein the network structure can refer to fig. 1. The first layer is the input layer, the input data is 990 × 22, 22 represents 22 channels, and 990 is the number of samples. The second layer is a time convolution layer, and convolution operation is carried out on data in a time dimension, wherein the size of a convolution kernel is 11 multiplied by 1, and 25 characteristic maps are generated. The third layer is a space convolution layer, the size of a convolution kernel is 22 multiplied by 1, and 25 characteristic graphs are obtained after the convolution kernel passes through a Relu activation function. The fourth layer is a pooling layer, the pooling range is 3 × 1, the pooling mode adopts maximum pooling, and the step length is 3. Then, a characteristic connection layer is formed, namely a 1 × 1 convolutional layer, the number of convolution kernels is 200, then an 11 × 1 convolutional layer, the number of convolution kernels is 50, and the characteristic connection layer enters the pooling layer. The last two layers of the network are full connection and linear classification layers, the data labels have four types, so that four output units are arranged after full connection, and the output result is the probability value that the input data belongs to each type;
training an intensive deep convolution neural network;
the network of the invention adopts a square error cost function as judgment, and the formula is as follows:
wherein N represents the number of samples, c represents the number of sample classes,
indicating that the nth sample corresponds to the kth dimension of the label,
indicating that the nth sample corresponds to the kth output of the network output.
The first layer is a data input layer, the input data format is fixed, and parameters needing training do not exist.
The second layer is a time convolution layer, each time a batch of samples is input, the input image is convolved with 25 convolution kernels, which have a size of 11 × 1, a step size of 1 and are initialized randomly, to obtain 25 feature maps, the layer has no bias and activation functions, and the formula is as follows:
wherein,
is the jth feature map of the ith layer, MjIn order to input a set of feature maps,
is the convolution kernel used for the connection between the input ith feature map and the output jth feature map.
The third layer is a space convolution layer, the input is the output after convolution of the upper layer, the size of a convolution kernel is 22 multiplied by 1, the number of input and output characteristic graphs is 25, but the layer has an activation function and has no offset term, and the formula is as follows:
wherein
Is the jth feature map of the ith layer, MjIn order to input a set of feature maps,
for the convolution kernel selected for this layer, f is the activation function Relu, i.e., f (x) max (0, x).
The fourth layer is a pooling layer with a size of 3 × 1, a step size of 3, 25 feature maps as input, and pooling does not change the number of feature maps, so that 25 output feature maps are obtained, and the formula is as follows:
wherein,
is the jth characteristic diagram of the l-1 layer,
for the ith layer bias, f is the activation function of the pooling layer, where there is no activation function, so f ═ f (x). Where down () represents a down-sampling function, here the maximum of 3 adjacent pixel values, so the down-sampling function is max ().
And the fifth layer is a feature connection layer which comprises two convolutional layers, wherein the input of the first convolutional layer is 25 pooled feature maps, convolution kernels with the size of 1 multiplied by 1 are adopted, the step length is 1 for convolution, 200 feature maps are generated, and then the feature maps are activated by a relu function and enter the next convolutional layer. The size of the second convolution kernel is 11 × 1, the step size is 1, 200 feature maps are input, 50 feature maps are output, after relu is activated, the 50 output feature maps of the second convolution are connected with the 25 feature maps of the first convolution input to serve as a new output feature map, and the formula is as follows:
xl＝Hl([x0,x1]) (5)
wherein Hl(. represents a characteristic join operation, x0Input of the first convolution for the feature connection layer, x1The output of the second convolution of the feature connection layer.
The sixth layer is a pooling layer, the input is a feature map after feature connection, the pooling size is 3 × 1, the step length is 3, and the operation is the same as the fourth layer.
The seventh layer is a full connection layer which expands and fully connects the characteristic diagram of the upper layer and converts the characteristic diagram into one-dimensional data.
And finally, outputting a required classification result for the output layer through a sigmoid activation function.
Calculating and propagating errors of the layers;
for the last output layer, the error between the activation value generated by the network and the actual value can be directly calculated, and the formula is as follows:
wherein the n islThe layer represents an output layer of the video stream,
weight weighting, h, representing that the output layer has not been activated by a functionw,b(x) Indicating the output result, y indicates the standard output,
represents nlThe ith output of the layer is then output,
the derivation is indicated.
N for l1-1,n1-2,n1-3, ·,2 layer errors, the general formula:
wherein wl+1Is the weight of layer l +1, δ(l+1)Error, sign calculated for layer l +1
Denotes the multiplication of each element, f' (u)l) Representing the output u for that layerlAnd (6) derivation.
If the first layer is a convolutional layer, the lower layer is a pooling layer, and one pixel of the pooling layer corresponds to a 3 × 1 pixel of an output graph of the convolutional layer, a size mismatch phenomenon occurs, so that the pooling layer needs to be upsampled, and the formula is as follows:
where up (-) denotes an upsample operation,
represents the l +1 th layer weight, δ(l+1)Error calculated for layer l + 1.
If the l-th layer is a downsampled layer and the l + 1-th layer is a convolutional layer, the formula is:
where conv2 is the convolution implementation function and rot180 denotes flipping the convolution kernel by 180 degrees.
Sixthly, calculating a partial derivative which is finally needed, updating a weight parameter and using a formula as follows;
wherein
The weight value of the old is represented,
for the new weight, η is the learning rate.
The old offset is represented by the value of the old offset,
is the new bias.
For convolutional layers, the weight update formula is:
wherein
Is that
When making convolution, with kijEach patch for convolution, (u, v) is the patch center, and the value of the (u, v) position in the output feature map is determined by the patch of the (u, v) position in the input feature map and the convolution kernel kijThe resulting values are convolved.
For the pooling layer, the weight updating formula is as follows:
wherein
And (4) a network is tested, test data and real labels are added, the output result is compared with the real labels, and a confusion matrix of the output result is obtained to analyze the model.
The effects of the present invention can be further illustrated by experimental results. The test data of the experiment used the official data BCIIV2a of the BCI race. The data set had a total of nine subjects, each with a training set and a test set. There are four types of labels for data, which are the motor imagery of the left hand, right hand, feet, and tongue, respectively. Each class of label corresponds to 72 samples of data, so that 288 samples exist in the training set and the testing set of one subject. Figures 3 and 4 are confusion matrices sorted using the Shallow ConvNet and method of the present invention. The accuracy of the ShallowConvNet was calculated to be 72% and the kappa value 0.632 based on FIG. 3, and the accuracy of the present invention was calculated to be 77% and the kappa value 0.698 based on FIG. 4. Compared with the Shallow ConvNet, the dense depth convolution neural network provided by the invention has the advantages that the recognition accuracy is higher by 5%, the kappa value is higher by 0.066, and the classification effect of the invention on the motor imagery electroencephalogram signals is obviously improved.
Claims (3)
1. A motor imagery electroencephalogram signal rapid identification method of a dense deep convolutional neural network is characterized by comprising the following steps:
A) inputting a motor imagery electroencephalogram signal into a densely connected deep convolutional neural network, wherein the densely connected deep convolutional neural network comprises:
as an input layer of the first layer, input data is 990 × 22 data of 22 channels 990 sampling points,
a time convolution layer, which is a second layer, performs a convolution operation of the data in a time dimension with a convolution kernel size of 11 x 1, generates 25 feature maps,
the space convolution layer as the third layer has convolution kernel size of 22 × 1, and is used for obtaining 25 characteristic maps after passing through Relu activation function,
the first pooling layer as the fourth layer had a pooling range of 3X 1, the pooling manner adopted maximum pooling with a step size of 3,
the output of the featured connection layer after the first pooling layer goes to the second pooling layer,
the second layer of the pool is a second layer of the pool,
a fully-connected layer after the second pooling layer,
the linear classification layer behind the full connection layer, the data label has four categories, four output units are arranged after full connection, the output result is the probability value of the input data belonging to each category,
wherein:
the time convolution layer inputs one batch of samples at a time, convolves the input image with 25 convolution kernels of size 11 × 1, step size 1 and random initialization, resulting in 25 feature maps, the layer being free of bias and activation functions, as follows:
wherein,
is the jth feature map of the ith layer, MjIn order to input a set of feature maps,
is a convolution kernel for the connection between the input ith feature map and the output jth feature map,
the input of the space convolution layer is the output of the time convolution layer, the size of the convolution kernel is 22 multiplied by 1, the number of input and output characteristic graphs is 25, the layer has an activation function and has no offset term, and the formula is as follows:
wherein
Is the first layerj feature maps, MjIn order to input a set of feature maps,
the convolution kernel is selected for this layer, and f is the activation function Relu, i.e. f (x) max (0, x),
the input of the first pooling layer is 25 characteristic graphs, the pooling does not change the number of the characteristic graphs, and 25 output characteristic graphs are obtained, and the formula is as follows:
wherein,
is the jth characteristic diagram of the l-1 layer,
for layer bias, f is the activation function of the pooling layer, where there is no activation function, so f ═ f (x), where down (·) denotes a down-sampling function, where the neighboring 3 pixel values are taken to be the largest one, so the down-sampling function is max (·),
the featured connection layer includes two convolution layers, wherein: the input of the first convolutional layer is 25 feature maps after pooling of the first pooling layer, convolution kernels with the size of 1 x 1 and the step length of 1 are adopted for convolution to generate 200 feature maps, and then the feature maps are activated through a relu function to enter a second convolutional layer; the convolution kernel size of the second convolution layer is 11 × 1, the step size is 1, the input of the convolution kernel size is 200 feature maps output by the first convolution layer, the output of the convolution kernel size is 50 feature maps, the 50 output feature maps of the second convolution are activated by relu and connected with the 25 feature maps input by the first convolution to serve as new output feature maps, and the formula is as follows:
xl＝Hl([x0,x1]) (5)
wherein Hl(. represents a splicing operation, x)0For the first convolutionInput, x1Is the output of the second convolution and,
the input of the second pooling layer is the new output characteristic map after the splicing, the pooling size is 3 x 1, the step length is 3,
the full connection layer expands and fully connects the characteristic diagram output by the second pooling layer, converts the characteristic diagram into one-dimensional data,
the linear classification layer outputs probability values of input data belonging to various categories through a sigmoid activation function,
B) calculating and propagating errors for each layer, including:
calculating the error between the activation value and the actual value generated by each layer of the network directly for the final output layer, wherein the formula is as follows:
wherein the n islThe layer represents an output layer of the video stream,
weights, h, representing the output layer without the activation functionw,b(x) Indicating the output result, y indicates the standard output,
the ith output of the output layer is represented,
it is indicated that the derivation is performed,
n for l1-1,n1-2,n1-3, …,2 layer error, general formula:
wherein wl+1Is the weight of layer l +1, δl+1For the error calculated for layer l +1, the notation, indicates each elementMultiplication, f' (u)l) Representing the output u for that layerlThe derivation is carried out by the derivation,
when the first layer is a convolutional layer, the lower layer of the layer is a pooling layer, one pixel of the pooling layer corresponds to one pixel (3 × 1) of an output image of the convolutional layer, and in order to eliminate the size mismatch phenomenon, the pooling layer is up-sampled by the formula:
where up (-) denotes an upsample operation,
represents the l +1 th layer weight, δ(l+1)The error calculated for the l +1 th layer,
when the l-th layer is a down-sampling layer, and the l + 1-th layer is a convolutional layer, the error formula is:
where conv2 is the convolution implementation function, rot180 denotes flipping the convolution kernel 180 degrees,
C) updating the weight parameter by using the formula:
wherein
The weight value of the old is represented,
as a new weight, η is the learning rate,
the old offset is represented by the value of the old offset,
in order to be the new offset,
wherein:
for convolutional layers, the weight update formula is:
wherein
Is that
When making convolution, with kijEach patch for convolution, (u, v) is the patch center, and the value of the (u, v) position in the output feature map is determined by the patch of the (u, v) position in the input feature map and the convolution kernel kijThe value obtained by the convolution is used,
for the pooling layer, the weight updating formula is as follows:
wherein
2. The method for rapidly identifying motor imagery electroencephalogram signals of the dense deep convolutional neural network of claim 1, wherein the following operations are performed before the step A: carrying out third-order band-pass 0-40hz filtering on the originally acquired motor imagery electroencephalogram signals to acquire signal components of larger frequency bands related to the motor imagery electroencephalogram signals and the motor imagery;
resampling the filtered signal components, and sampling data with different sampling frequencies to the same frequency so as to keep the data length of the input convolutional neural network consistent and ensure that the data volume under the same time length is the same;
and intercepting the events with fixed length from the preprocessed data, and acquiring corresponding labels of the events as motor imagery electroencephalogram signals input into the dense connection type deep convolution neural network.
3. The motor imagery electroencephalogram signal rapid identification method of the dense deep convolutional neural network of claim 1, wherein:
the four categories of data tags are the motor imagery of the left hand, right hand, feet, and tongue, respectively.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810057413.1A CN110069958B (en) | 2018-01-22 | 2018-01-22 | Electroencephalogram signal rapid identification method of dense deep convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810057413.1A CN110069958B (en) | 2018-01-22 | 2018-01-22 | Electroencephalogram signal rapid identification method of dense deep convolutional neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110069958A CN110069958A (en) | 2019-07-30 |
| CN110069958B true CN110069958B (en) | 2022-02-01 |
Family
ID=67364510
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810057413.1A Active CN110069958B (en) | 2018-01-22 | 2018-01-22 | Electroencephalogram signal rapid identification method of dense deep convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110069958B (en) |
Families Citing this family (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110458066B (en) * | 2019-07-31 | 2022-11-18 | 同济大学 | Age group classification method based on resting electroencephalogram data |
| CN110543831A (en) * | 2019-08-13 | 2019-12-06 | 同济大学 | brain print identification method based on convolutional neural network |
| CN110555468A (en) * | 2019-08-15 | 2019-12-10 | 武汉科技大学 | Electroencephalogram signal identification method and system combining recursion graph and CNN |
| CN110796175A (en) * | 2019-09-30 | 2020-02-14 | 武汉大学 | Electroencephalogram data online classification method based on light convolutional neural network |
| CN110765920B (en) * | 2019-10-18 | 2023-03-24 | 西安电子科技大学 | Motor imagery classification method based on convolutional neural network |
| CN111012336B (en) * | 2019-12-06 | 2022-08-23 | 重庆邮电大学 | Parallel convolutional network motor imagery electroencephalogram classification method based on spatio-temporal feature fusion |
| CN111368884B (en) * | 2020-02-22 | 2023-04-07 | 杭州电子科技大学 | Motor imagery electroencephalogram feature extraction method based on matrix variable Gaussian model |
| CN111339975B (en) * | 2020-03-03 | 2023-04-21 | 华东理工大学 | Target detection, identification and tracking method based on central scale prediction and twin neural network |
| CN111428648B (en) * | 2020-03-26 | 2023-03-28 | 五邑大学 | Electroencephalogram signal generation network, method and storage medium |
| CN111709267B (en) * | 2020-03-27 | 2022-03-29 | 吉林大学 | Electroencephalogram signal emotion recognition method of deep convolutional neural network |
| CN111407269A (en) * | 2020-03-30 | 2020-07-14 | 华南理工大学 | EEG signal emotion recognition method based on reinforcement learning |
| CN111638249B (en) * | 2020-05-31 | 2022-05-17 | 天津大学 | Water content measuring method based on deep learning and application of water content measuring method in oil well exploitation |
| CN111783857A (en) * | 2020-06-18 | 2020-10-16 | 内蒙古工业大学 | Motor imagery brain-computer interface based on nonlinear network information graph |
| CN111882036B (en) * | 2020-07-22 | 2023-10-31 | 广州大学 | Convolutional neural network training method, electroencephalogram signal identification method, device and medium |
| CN112528819B (en) * | 2020-12-05 | 2023-01-20 | 西安电子科技大学 | P300 electroencephalogram signal classification method based on convolutional neural network |
| CN112633365B (en) * | 2020-12-21 | 2024-03-19 | 西安理工大学 | Mirror convolution neural network model and motor imagery electroencephalogram recognition algorithm |
| CN112890828A (en) * | 2021-01-14 | 2021-06-04 | 重庆兆琨智医科技有限公司 | Electroencephalogram signal identification method and system for densely connecting gating network |
| CN112818876B (en) * | 2021-02-04 | 2022-09-20 | 成都理工大学 | Electromagnetic signal extraction and processing method based on deep convolutional neural network |
| CN113057653B (en) * | 2021-03-19 | 2022-11-04 | 浙江科技学院 | Channel mixed convolution neural network-based motor electroencephalogram signal classification method |
| CN113642528B (en) * | 2021-09-14 | 2022-12-09 | 西安交通大学 | Hand movement intention classification method based on convolutional neural network |
| CN113791691B (en) * | 2021-09-18 | 2022-05-20 | 中国科学院自动化研究所 | Electroencephalogram signal band positioning method and device |
| CN114004257A (en) * | 2021-11-02 | 2022-02-01 | 南京邮电大学 | Myoelectric gesture recognition method based on lightweight convolutional neural network |
| CN114652326B (en) * | 2022-01-30 | 2024-06-14 | 天津大学 | Real-time brain fatigue monitoring device based on deep learning and data processing method |
| CN114781441B (en) * | 2022-04-06 | 2024-01-26 | 电子科技大学 | EEG motor imagery classification method and multi-space convolution neural network model |
| CN114818803A (en) * | 2022-04-25 | 2022-07-29 | 上海韶脑传感技术有限公司 | Neuron optimization-based motor imagery electroencephalogram modeling method for unilateral limb patient |
| CN115337026B (en) * | 2022-10-19 | 2023-03-10 | 之江实验室 | Convolutional neural network-based EEG signal feature retrieval method and device |
| CN116630697B (en) * | 2023-05-17 | 2024-04-05 | 安徽大学 | Image classification method based on biased selection pooling |
| CN117763399B (en) * | 2024-02-21 | 2024-05-14 | 电子科技大学 | Neural network classification method for self-adaptive variable-length signal input |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105068644A (en) * | 2015-07-24 | 2015-11-18 | 山东大学 | Method for detecting P300 electroencephalogram based on convolutional neural network |
| CN106821681A (en) * | 2017-02-27 | 2017-06-13 | 浙江工业大学 | A kind of upper limbs ectoskeleton control method and system based on Mental imagery |
| CN107506774A (en) * | 2017-10-09 | 2017-12-22 | 深圳市唯特视科技有限公司 | A kind of segmentation layered perception neural networks method based on local attention mask |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10417529B2 (en) * | 2015-09-15 | 2019-09-17 | Samsung Electronics Co., Ltd. | Learning combinations of homogenous feature arrangements |
| US20170132511A1 (en) * | 2015-11-10 | 2017-05-11 | Facebook, Inc. | Systems and methods for utilizing compressed convolutional neural networks to perform media content processing |
-
2018
- 2018-01-22 CN CN201810057413.1A patent/CN110069958B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105068644A (en) * | 2015-07-24 | 2015-11-18 | 山东大学 | Method for detecting P300 electroencephalogram based on convolutional neural network |
| CN106821681A (en) * | 2017-02-27 | 2017-06-13 | 浙江工业大学 | A kind of upper limbs ectoskeleton control method and system based on Mental imagery |
| CN107506774A (en) * | 2017-10-09 | 2017-12-22 | 深圳市唯特视科技有限公司 | A kind of segmentation layered perception neural networks method based on local attention mask |
Non-Patent Citations (2)
| Title |
|---|
| Deep learning with convolutional neural networks for decoding and visualization of EEG pathology;R. T. Schirrmeister等;《2017 IEEE Signal Processing in Medicine and Biology Symposium 》;20171202;第1-7页 * |
| Densely connected convolutional networks;Gao Huang等;《2017 IEEE Conference on Computer Vision and Pattern Recognition》;20170726;第2261-2269页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110069958A (en) | 2019-07-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110069958B (en) | Electroencephalogram signal rapid identification method of dense deep convolutional neural network | |
| Shoieb et al. | Computer-aided model for skin diagnosis using deep learning | |
| CN108491077B (en) | Surface electromyographic signal gesture recognition method based on multi-stream divide-and-conquer convolutional neural network | |
| CN111062250B (en) | Multi-subject motor imagery electroencephalogram signal identification method based on deep feature learning | |
| CN110353675B (en) | Electroencephalogram signal emotion recognition method and device based on picture generation | |
| CN112120694B (en) | Motor imagery electroencephalogram signal classification method based on neural network | |
| Kumar et al. | Breast cancer classification of image using convolutional neural network | |
| CN108959895B (en) | Electroencephalogram EEG (electroencephalogram) identity recognition method based on convolutional neural network | |
| CN110991406B (en) | RSVP electroencephalogram characteristic-based small target detection method and system | |
| CN110059565A (en) | A kind of P300 EEG signal identification method based on improvement convolutional neural networks | |
| CN114224342B (en) | Multichannel electroencephalogram signal emotion recognition method based on space-time fusion feature network | |
| CN105068644A (en) | Method for detecting P300 electroencephalogram based on convolutional neural network | |
| CN108960182A (en) | A kind of P300 event related potential classifying identification method based on deep learning | |
| CN112022153B (en) | Electroencephalogram signal detection method based on convolutional neural network | |
| CN107424145A (en) | The dividing method of nuclear magnetic resonance image based on three-dimensional full convolutional neural networks | |
| CN108921141B (en) | Electroencephalogram EEG (electroencephalogram) feature extraction method based on depth self-coding neural network | |
| CN113627401A (en) | Myoelectric gesture recognition method of feature pyramid network fused with double-attention machine system | |
| CN112465069B (en) | Electroencephalogram emotion classification method based on multi-scale convolution kernel CNN | |
| CN112450947B (en) | Dynamic brain network analysis method for emotional arousal degree | |
| CN107330393A (en) | A kind of neonatal pain expression recognition method based on video analysis | |
| CN108960299A (en) | A kind of recognition methods of multiclass Mental imagery EEG signals | |
| CN111428601B (en) | P300 signal identification method, device and storage medium based on MS-CNN | |
| WO2021212715A1 (en) | Schizophrenia classification and identification method, operation control apparatus, and medical equipment | |
| CN109325410B (en) | Electroencephalogram EEG (electroencephalogram) feature extraction method based on convolutional neural network | |
| CN115919330A (en) | EEG Emotional State Classification Method Based on Multi-level SE Attention and Graph Convolution |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |