CN113269058A - Movement imagery identification method based on GAN model and PLV network - Google Patents
Movement imagery identification method based on GAN model and PLV network Download PDFInfo
- Publication number
- CN113269058A CN113269058A CN202110503058.8A CN202110503058A CN113269058A CN 113269058 A CN113269058 A CN 113269058A CN 202110503058 A CN202110503058 A CN 202110503058A CN 113269058 A CN113269058 A CN 113269058A
- Authority
- CN
- China
- Prior art keywords
- plv
- matrix
- network
- gan
- state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013256 Gubra-Amylin NASH model Methods 0.000 title claims abstract description 17
- 238000000034 method Methods 0.000 title claims abstract description 15
- 230000033001 locomotion Effects 0.000 title claims abstract description 11
- 239000011159 matrix material Substances 0.000 claims abstract description 44
- 238000012545 processing Methods 0.000 claims abstract description 9
- 238000012549 training Methods 0.000 claims abstract description 8
- 238000012952 Resampling Methods 0.000 claims abstract description 6
- 238000001914 filtration Methods 0.000 claims abstract description 4
- 238000007781 pre-processing Methods 0.000 claims abstract description 4
- 230000011218 segmentation Effects 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 2
- 210000004556 brain Anatomy 0.000 abstract description 7
- 230000000284 resting effect Effects 0.000 abstract description 4
- 238000011160 research Methods 0.000 abstract description 3
- 241000282414 Homo sapiens Species 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 2
- 230000010365 information processing Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000003710 cerebral cortex Anatomy 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/02—Preprocessing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/12—Classification; Matching
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention provides a movement imagery identification method based on a GAN model and a PLV network, which comprises the following steps: (1) preprocessing an original signal, including resampling and segmenting; (2) filtering each section of electroencephalogram signal in a band-pass mode, and performing sliding window processing; (3) calculating PLV for each sliding window lower brain electrical signal, then calculating the average value of the PLV of each sliding window signal, and taking the average value as the PLV in the state; (4) calculating PLV in a resting state; (5) calculating a differential PLV characteristic matrix; (6) the PLV feature matrix with the label sample is used as a constraint condition, and the GAN is utilized to convert the random generation matrix obtained by the generator into a generated PLV feature vector, so that the expansion of the sample feature set is realized; (7) training a classifier based on the expanded sample feature set, constructing a motor imagery identification model based on a GAN model and a PLV network, and outputting an identification result. The method realizes effective expansion of the sample feature set through the GAN, overcomes the phenomenon of over-training fitting caused by insufficient sample feature set, improves the state recognition capability of the motor imagery, and lays a foundation for further research.
Description
Technical Field
The invention relates to the crossing field of signal and information processing and neurobiology, in particular to a motor imagery identification method based on a GAN model and a PLV network.
Background
The brain, one of the most complex systems in nature, is the physiological basis for the human brain to perform information processing and cognitive expression. Taking the interaction of cerebral cortex into consideration, noninvasive acquisition of structural connection of human brain by using mathematical modeling is still a research hotspot content of constructing a human brain structural network at present, which is beneficial to enhancing understanding of brain-task relationship. Therefore, the research on the brain network is helpful for deepening understanding of the working mechanism of the brain of human beings in a task state, and promoting the development and progress of the fields of brain-computer interfaces, cognitive processes, clinical application and the like.
In recent years, although new feature extraction methods are continuously emerging to bring more choices for motor imagery identification, the problem of universality of data among different testees is caused by (1) individual difference; (2) the data acquisition equipment and the acquisition time cost are high, the acquisition cost problem is caused, and the loss of the motor imagery label sample is easily caused under the combined action of the factors. Particularly with the advent of deep learning networks, the urgency for solving the problem is further increased. Compared with the traditional feature extraction method, the deep learning has the characteristic of a deep network structure, and the data volume of the training samples puts higher requirements on the deep learning, otherwise, the phenomenon of overfitting is easy to occur.
Therefore, the method fully utilizes the pseudo constraint conditions of a small amount of labeled feature matrixes, achieves expansion of the feature matrixes on the basis of the GAN model, and is expected to solve the overfitting phenomenon of network training and effectively solve the problem of sample completeness so as to ensure the classification and identification effects. Through competition and game processes of an internal generation model and a discrimination model of the GAN network, parameter optimization and performance improvement of the whole network are realized. Considering the design flexibility of the GAN model, in order to improve the completeness of data, a small number of labeled feature matrixes are used for optimizing and fine-tuning the learning network so as to ensure the completeness and sufficiency of network model training, and finally, the GAN model is designed to be suitable for a motor imagery data expansion algorithm framework.
Disclosure of Invention
In order to solve the problems, the invention provides a motor imagery state identification method combining a sample generation algorithm based on a GAN model and a differential PLV network model. The basic technical scheme of the invention is as follows:
1. preprocessing an original signal, including resampling and segmenting;
2. filtering each section of electroencephalogram signal in a band-pass mode, and performing sliding window processing;
3. calculating PLV of each sliding window signal of the electroencephalogram signal in each state respectively, then calculating the average value of the PLV of each sliding window signal, and taking the average value as the PLV in the state;
4. performing sliding window processing on 2s of electroencephalogram data before each motor imagery begins, calculating PLV (pulse-to-volume) for each window data, then calculating the average value of each sliding window PLV, and taking the average value as the PLV in a resting state;
5. calculating a differential PLV characteristic matrix;
6. converting a random generation matrix acquired by a generator into a generated PLV feature vector by using a generated countermeasure network (GAN) with the PLV feature matrix with the label sample as a constraint condition, so as to realize the expansion of the sample feature set;
7. training a classifier based on the expanded sample feature set, constructing a motor imagery identification model based on a GAN model and a PLV network, and outputting an identification result.
The method has the advantages that when the motor imagery state is identified, the generation of the confrontation network is utilized to realize the expansion of the feature set sample, so that the problem of insufficient sample sets is effectively solved, and the motor imagery state identification capability is improved.
Drawings
FIG. 1 is a flow chart of a movement imagery identification method based on a GAN model and a PLV network
FIG. 2 PLV network schematic of a real sample (BrainNet based tool box display)
FIG. 3A PLV network schematic of a generated sample (based on the BrainNet tool box display)
Detailed Description
Fig. 1 shows a flow chart of a method for identifying a motor imagery based on a GAN model and a PLV network, and the following describes in detail a specific embodiment of the present invention with reference to the drawings.
(1) Preprocessing the original N-channel electroencephalogram signals, including resampling and segmentation processing: reading the electroencephalogram data of the motor imagery, and performing down-sampling on the electroencephalogram data; then, carrying out segmentation processing on the data after resampling according to the label of the imagination motion state, and setting the duration time of an electroencephalogram signal s (t) of each imagination motion state as 3 s;
(2) carrying out (15-35Hz) band-pass filtering on each section of electroencephalogram signal s (t), carrying out sliding window processing, wherein the window length is 1s, the overlapping rate is 10%, and accordingly segmenting the electroencephalogram signal s (t), each section of signal is si(t)
(3) Calculate each si(t) Phase Locked Value (PLV) MatrixPLVAnd calculating the average value Matrix of the Matrix sequencemeanPLV；
(4) Taking the electroencephalogram data 2s before the beginning of the imagination movement as the electroencephalogram data of the resting state, repeating the steps (2) and (3), and calculating the average value PLV Matrix of the resting staterest-meanPLV，
(5) Calculating a differential PLV characteristic Matrix according to the PLV matrixes calculated in the steps (3) and (4)diffPLVAs shown in equation (1):
MatrixdiffPLV＝MatrixmeanPLV-Matrixrest-meanPLVformula (1)
(6) The method comprises the following specific steps of taking a PLV feature matrix with a label sample as a constraint condition, converting a random generation matrix acquired by a generator into a generated PLV feature vector by using a generation countermeasure network (GAN), and expanding a sample feature set, wherein the specific steps are as follows:
(6.1) randomly generating an N × N pseudo feature Matrix of N samples with a generator using GANpseudoPLV；
(6.2) matching the pseudo feature MatrixpseudoPLVAnd an NxN feature Matrix of N samples with labelsdiffPLVMixing MatrixpseudoPLVAnd MatrixdiffPLVAs two kinds of samples, transfuseEntering a discriminator to realize discrimination of the pseudo feature matrix;
(6.3) if the error between the generated PLV characteristic matrix and the real PLV characteristic matrix reaches a certain threshold value, the optimization of the generator and the discriminator of the GAN is finished, and the next step is carried out; otherwise, optimizing and updating the network parameters of the generator and the discriminator by using a back propagation algorithm, and re-entering the step (6.1);
(6.4) repeating the step (6.1) by taking the sample feature matrix in each state as a constraint condition until all the pseudo feature matrices in the motor imagery state are generated;
(6.5) realizing capacity expansion of the characteristic Matrix with the label in the specific state by utilizing the pseudo characteristic Matrix in the specific state, and constructing a new characteristic Matrix [ MatrixpseudoPLV；MatrixdiffPLV]；
(7) Training a classifier based on the expanded sample characteristic matrix, constructing a motor imagery identification model based on a GAN model and a PLV network, and outputting an identification result.
Claims (3)
1. A motor imagery identification method based on a GAN model and a PLV network comprises the following steps:
(1) preprocessing the original N-channel electroencephalogram signals, including resampling and segmentation processing: reading the electroencephalogram data of the motor imagery, and performing down-sampling on the electroencephalogram data; then, carrying out segmentation processing on the data after resampling according to the label of the imagination motion state, and setting the duration time of an electroencephalogram signal s (t) of each imagination motion state as 3 s;
(2) band-pass filtering each section of electroencephalogram signals, and performing sliding window processing, wherein the window length is 1s, the overlapping rate is 10%, and accordingly each section of electroencephalogram signals with the duration of 3s is divided into 21 sections;
(3) smoothly windowing the electroencephalogram signals in each imagination motion state, and calculating the average value of a Phase Locked Value (PLV) matrix;
(4) solving a differential PLV matrix in each imaginary movement state, and obtaining the differential PLV matrix by solving a difference value with the PLV characteristic matrix in the rest state;
(5) the PLV feature matrix with the label sample is used as a constraint condition, and a generation countermeasure network (GAN) is utilized to convert the random generation matrix obtained by the generator into a generated PLV feature vector, so that the expansion of the sample feature set is realized;
(6) training a classifier based on the expanded sample feature set, constructing a motor imagery identification model based on a GAN model and a PLV network, and outputting an identification result.
2. The GAN model and PLV network-based motor imagery identification method of claim 1, wherein step (4) is characterized by computing a differential PLV feature matrix.
3. The GAN model and PLV network based motor imagery identification method of claim 1, wherein steps (5) and (6) are subdivided into the following steps:
(3.1) randomly generating an N × N pseudo feature matrix of N samples using a generator of GAN;
(3.2) inputting the randomly generated pseudo feature matrix and the N multiplied by N feature matrix of N samples with the same state label into a discriminator as two types of samples to realize discrimination of the pseudo samples;
(3.3) if the error between the distinguishing label and the real label reaches a certain threshold value, indicating that the optimization of parameters of a generator and a discriminator of the GAN is finished, and entering (3.4); otherwise, updating and optimizing the network parameters of the generator and the discriminator by using a back propagation algorithm, and then repeating the step (3.1);
(3.4) constraining the sample feature matrixes of other state labels, and entering the step (3.1) until the pseudo feature matrixes of all the states are obtained;
and (3.5) taking the pseudo feature matrix in the specific state as the extension of the PLV feature matrix in the state, constructing a new feature matrix, and realizing the identification of the motor imagery state.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110503058.8A CN113269058A (en) | 2021-05-08 | 2021-05-08 | Movement imagery identification method based on GAN model and PLV network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110503058.8A CN113269058A (en) | 2021-05-08 | 2021-05-08 | Movement imagery identification method based on GAN model and PLV network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113269058A true CN113269058A (en) | 2021-08-17 |
Family
ID=77230218
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110503058.8A Pending CN113269058A (en) | 2021-05-08 | 2021-05-08 | Movement imagery identification method based on GAN model and PLV network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113269058A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104510468A (en) * | 2014-12-30 | 2015-04-15 | 中国科学院深圳先进技术研究院 | Character extraction method and device of electroencephalogram |
| CN110531861A (en) * | 2019-09-06 | 2019-12-03 | 腾讯科技（深圳）有限公司 | The treating method and apparatus and storage medium of Mental imagery EEG signals |
| CN112001306A (en) * | 2020-08-21 | 2020-11-27 | 西安交通大学 | Electroencephalogram signal decoding method for generating neural network based on deep convolution countermeasure |
| CN112545532A (en) * | 2020-11-26 | 2021-03-26 | 中国人民解放军战略支援部队信息工程大学 | Data enhancement method and system for classification and identification of electroencephalogram signals |
-
2021
- 2021-05-08 CN CN202110503058.8A patent/CN113269058A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104510468A (en) * | 2014-12-30 | 2015-04-15 | 中国科学院深圳先进技术研究院 | Character extraction method and device of electroencephalogram |
| CN110531861A (en) * | 2019-09-06 | 2019-12-03 | 腾讯科技（深圳）有限公司 | The treating method and apparatus and storage medium of Mental imagery EEG signals |
| CN112001306A (en) * | 2020-08-21 | 2020-11-27 | 西安交通大学 | Electroencephalogram signal decoding method for generating neural network based on deep convolution countermeasure |
| CN112545532A (en) * | 2020-11-26 | 2021-03-26 | 中国人民解放军战略支援部队信息工程大学 | Data enhancement method and system for classification and identification of electroencephalogram signals |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109919031B (en) | Human behavior recognition method based on deep neural network | |
| Li et al. | SiamVGG: Visual tracking using deeper siamese networks | |
| Du et al. | Skeleton based action recognition with convolutional neural network | |
| CN111523601B (en) | Potential emotion recognition method based on knowledge guidance and generation of countermeasure learning | |
| CN113065526B (en) | Electroencephalogram signal classification method based on improved depth residual error grouping convolution network | |
| CN109493342B (en) | Skin disease picture lesion type classification method based on deep learning | |
| CN111956208B (en) | ECG signal classification method based on ultra-lightweight convolutional neural network | |
| CN110782427B (en) | Magnetic resonance brain tumor automatic segmentation method based on separable cavity convolution | |
| CN111202512A (en) | Electrocardiogram classification method and device based on wavelet transformation and DCNN | |
| CN114613013A (en) | End-to-end human behavior recognition method and model based on skeleton nodes | |
| CN116186593B (en) | Electrocardiosignal detection method based on separable convolution and attention mechanism | |
| CN112257741A (en) | Method for detecting generative anti-false picture based on complex neural network | |
| CN112836602A (en) | Behavior recognition method, device, equipment and medium based on space-time feature fusion | |
| CN115238796A (en) | Motor imagery electroencephalogram signal classification method based on parallel DAMSCN-LSTM | |
| CN112818958B (en) | Action recognition method, device and storage medium | |
| CN113627391A (en) | Cross-mode electroencephalogram signal identification method considering individual difference | |
| CN112826507A (en) | Brain function network evolution modeling method for sensorineural deafness | |
| CN113269058A (en) | Movement imagery identification method based on GAN model and PLV network | |
| CN110992320A (en) | Medical image segmentation network based on double interleaving | |
| CN115708687A (en) | Stimulated image reconstruction method and system based on artificial neural network | |
| CN116421200A (en) | Brain electricity emotion analysis method of multi-task mixed model based on parallel training | |
| CN113033359B (en) | Self-supervision-based pre-training and facial paralysis grading modeling and grading method and system | |
| Liu et al. | Spatial-temporal convolutional attention for mapping functional brain networks | |
| CN112614539A (en) | Motor imagery detection method based on TEO-MIC algorithm | |
| CN112270359A (en) | One-dimensional sequence ascending clustering method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20210817 |
|
| WD01 | Invention patent application deemed withdrawn after publication |