CN111523520A - Method for analyzing electroencephalogram signals of brain patients with motor imagery stroke by using cycleGAN - Google Patents

Method for analyzing electroencephalogram signals of brain patients with motor imagery stroke by using cycleGAN Download PDF

Info

Publication number
CN111523520A
CN111523520A CN202010527458.8A CN202010527458A CN111523520A CN 111523520 A CN111523520 A CN 111523520A CN 202010527458 A CN202010527458 A CN 202010527458A CN 111523520 A CN111523520 A CN 111523520A
Authority
CN
China
Prior art keywords
spectrogram
cyclegan
domain
patient
stroke
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010527458.8A
Other languages
Chinese (zh)
Inventor
徐舫舟
荣芬奇
冯超
张凯丽
张杨
苗芸菁
孙亚南
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qilu University of Technology
Qilu Hospital of Shandong University
Original Assignee
Qilu University of Technology
Qilu Hospital of Shandong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qilu University of Technology, Qilu Hospital of Shandong University filed Critical Qilu University of Technology
Priority to CN202010527458.8A priority Critical patent/CN111523520A/en
Publication of CN111523520A publication Critical patent/CN111523520A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/02—Preprocessing
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/08—Feature extraction

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Pathology (AREA)
  • Evolutionary Computation (AREA)
  • Animal Behavior & Ethology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Surgery (AREA)
  • Medical Informatics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Computational Linguistics (AREA)
  • Psychiatry (AREA)
  • Signal Processing (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Psychology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Physiology (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)

Abstract

The invention discloses a method for analyzing electroencephalogram signals of a patient with stroke containing motor imagery by using cycleGAN, belonging to the field of brain-computer interfaces, aiming at solving the technical problem of how to expand EEG data volume of the patient with stroke by using a cycleGAN network to make the research of an MI-BCI system progress on DL, and adopting the technical scheme that: the method comprises the following specific steps: acquisition and pre-processing of EEG signals: collecting EEG signals of a stroke patient and a healthy person, and preprocessing the EEG signals; generating a spectrogram: converting one-dimensional EEG signals of a stroke patient and a healthy person into a two-dimensional spectrogram through EEG2 Image; training and generating artificial frequency spectrogram: the spectrogram of a healthy person and the spectrogram of a cerebral apoplexy patient are simultaneously input into a cycleGAN neural network for training, and the spectrogram of the cerebral apoplexy patient is used as a comparison sample, so that the spectrogram of the healthy person learns the characteristics of the spectrogram of the cerebral apoplexy patient to generate an artificial spectrogram based on the cerebral apoplexy patient.

Description

Method for analyzing electroencephalogram signals of brain patients with motor imagery stroke by using cycleGAN
Technical Field
The invention relates to the field of Brain-Computer Interface (BCI), in particular to a method for analyzing electroencephalogram signals of a patient with stroke including motor imagery by using cycleGAN.
Background
Stroke, also known as stroke, is an acute cerebrovascular disease, is a common disease seriously threatening human beings, and has the characteristics of high morbidity, high disability rate, high mortality and the like. About 1500 million people worldwide cause central paralysis of limbs due to cardiovascular diseases such as cerebral apoplexy and the like every year. The fatality rate of cardiovascular disease patients in China shows an increasing trend, wherein 1300 ten thousands of patients with stroke are in the fatality rate. The stroke patients have poor daily life capability, and heavy mental and economic burden is brought to families and society. At present, Chinese rehabilitation technology has a very large gap, and increasingly patients with cerebral apoplexy urgently want to receive rehabilitation therapy, aiming at patients with cerebral apoplexy with dyskinesia, damaged Motor cortex parts can be repeatedly stimulated through a Motor Image (MI) therapy based on a BCI system, damaged peripheral Motor nerve cells are reactivated, and the Motor ability of the patients is recovered. The motor imagery therapy based on the BCI system can be used as an auxiliary treatment means in the rehabilitation process, effectively improves the motor function of patients and relieves the shortage of rehabilitation technicians in China.
BCI is a cross technology relating to multiple disciplines such as neuroscience, signal detection, signal processing, pattern recognition and the like, and is a system which can communicate with the external environment and external equipment completely and autonomously without depending on peripheral nerves and muscles. The fields that BCI systems mainly relate to at present are: disease identification, generation of visual images, stroke rehabilitation, military fields, and the like. The BCI system replaces human limbs or language organs to realize communication between a human body and the outside and control the external equipment by collecting information sent by the brain and directly converting the information into a command capable of driving the external equipment. The electroencephalogram signal acquisition mainly comprises an invasive type and a non-invasive type. Among them, invasive is to directly implant a collecting chip into a cerebral gray layer to obtain a cortical electroencephalogram (ECoG) at the time of neurosurgery. The invasive type can generate an electroencephalogram signal with high signal-to-noise ratio, but the implantation process needs an operation and has certain risks. The second non-invasive method is to place detecting electrodes on the scalp to obtain electroencephalogram (EEG), which is simple and easy to operate but has interference of signals such as electrooculogram.
The acquired EEG signal has poor stability due to unstable states of the stroke patient; the patient is easy to fatigue in the experimental process, which can cause the shortage of effective EEG data based on the cerebral apoplexy patient, so that the expansion of analysis data is a great challenge for the development of the brain-computer interface at the present stage.
Disclosure of Invention
The technical task of the invention is to provide a method for analyzing an electroencephalogram signal of a patient with stroke including motor imagery by using cycleGAN, so as to solve the problem of how to expand the EEG data volume of the patient with stroke by using a cycleGAN network and make the research of an MI-BCI system progress on DL.
The technical task of the invention is realized in the following way, a method for analyzing the EEG signals of a patient with stroke containing motor imagery by using cycleGAN, which combines the cycleGAN network to artificially generate EEG data of the patient with stroke so as to enlarge the EEG data volume of the patient with stroke; the method comprises the following specific steps:
acquisition and pre-processing of EEG signals: collecting EEG signals of a stroke patient and a healthy person, and preprocessing the EEG signals;
generating a spectrogram: converting one-dimensional EEG signals of a stroke patient and a healthy person into a two-dimensional spectrogram through EEG2 Image;
training and generating artificial frequency spectrogram: simultaneously inputting a spectrogram of a healthy person and a spectrogram of a stroke patient into a cycleGAN neural network for training, and taking the spectrogram of the stroke patient as a comparison sample, so that the spectrogram of the healthy person learns the characteristics of the spectrogram of the stroke patient to generate an artificial spectrogram based on the stroke patient, thereby achieving the purposes of artificially generating EEG data of the stroke patient and expanding the EEG data volume based on the stroke patient; wherein, the training condition is observed and the training progress is checked through the TensorBoard in the training process.
Preferably, the EEG signal is acquired non-invasively using an electrode cap of 64 electrodes;
the pretreatment is as follows:
carrying out eye-removing and artifact-removing treatment on the collected EEG signal data by using Curry 8;
processing the EEG signal by adopting a down-sampling mode; the down-sampling is to take out partial sampling points at equal intervals in the original sampling sequence as a preprocessed data set.
Preferably, the generated spectrogram specifically includes:
extracting the μ and β bands of each EEG signal;
calculating the sum of the squares of the absolute values of the mu and beta bands;
the absolute values of the μ and β bands are combined into a matrix and input into the EEG2Image to obtain a spectrogram.
Preferably, the structure of the CycleGAN neural network comprises two generators and two discriminators, the purpose of the structure of the CycleGAN neural network is to map a source distribution S to a target distribution T with G; the CycleGAN neural network has two cycles, one of which is S → g (S)) → P (g (S)), and the formula for the loss of resistance of the cycle is as follows:
LGAN(G,Dt,S,T)＝Et～pdata(t)[logDT(t)]+Es～pdata(s)[log(1-DT(G(s)))]；
wherein G represents a generator, the goal of G is to generate G (S), let G (S) resemble the samples in the target distribution T; dtRepresenting a discriminator, DtThe goal of (a) is to distinguish between G (S) generated and control samples in the S domain; t represents data of a cerebral apoplexy patient, namely control data; s represents data of a healthy person; e represents an expected value; when D is presentT(t)＝1，DTWhen (g (s)) is 0, then LGAN(G,DtS, T) ═ 0, the goal of the optimization is to make the generator losses smaller and smaller;
another cycle is T → p (T) → G (p (T)), and the formula for the cyclic antagonistic loss is as follows:
LGAN(P,Ds,T,S)＝Es～pdata(s)[logDS(t)]+Et～pdata(t)[log(1-DS(G(t)))]；
wherein, P represents a generator, the goal of P is to generate P (T) and let P (T) be similar to the sample in the target distribution S; dsRepresenting a discriminator, DsThe goal of (a) is to distinguish between P (T) produced by P and a control sample in the T domain; t represents data of a cerebral apoplexy patient, namely control data; s represents data of a healthy person; e represents an expected value; when D is presentT(t)＝1，Ds(G (t)) ═ 0, then LGAN(P,Ds,T,S)＝0；
The generator G and the generator P of the CycleGAN neural network respectively generate samples with similar S-domain and T-domain distributions, and can convert back, namely after the picture of the S-domain is converted into the T-domain space, the picture of the S-domain can be converted into the S-domain, and the pictures of all the S-domains are converted into the same picture in the T-domain space by an end-to-end module, so that the CycleGAN neural network is a cyclic countermeasure network, the cyclic consistency loss exists in the cyclic process and is called as cyclic countermeasure loss, and the formula is as follows:
Lcyc(G,P)＝Es～pdata(s)||[P(G(s))-s||1]+Et～pdata(t)||[P(G(t))-t||1]；
wherein, the input of G is s, which is used to generate a false t-map (fake t); the input of P is t, which is used to generate a false s-map (fake s); after s is taken as the input of G, fake t is generated, and then the fake t is input into P to obtain a fake s graph; theoretically, the false s-map should be comparable to the original input s-map;
in summary, the total loss of the CycleGAN neural network is:
L(G,P,DS,DT)＝LGAN(G,DT,S,T)+LGAN(P,DS,T,S)+λLcyc(G,P)；
wherein, λ represents a balance parameter, and the value is preferably 10.
More preferably, the generator is a network of encoders, converters and decoders, which network as a whole goes through a down-sampling and an up-sampling process, with the residual block in between, producing spectrograms from EEG2 images, processing the spectrograms into tfrecrds form; the working process of the generator is as follows:
inputting a spectrogram of a healthy person into a generator G, and inputting a spectrogram of a stroke patient into a generator P;
the spectrogram passes through an encoder, and features are extracted through a convolutional layer of the encoder;
the converter transforms the feature vector of the spectrogram from an S domain to a T domain or from the T domain to the S domain according to the features extracted by the convolutional layer of the encoder;
the decoder uses a deconvolution network to recover low-level features from the feature vectors, converting the low-level features into a spectrogram.
Preferably, the discriminator is a convolutional network composed of five convolutional layers, the first four convolutional layers extract the features of the spectrogram, and the last convolutional layer judges whether the spectrogram belongs to a T domain or an S domain;
the convolution network adopts a PatchGAN structure, the size of the PatchGAN structure is 70 multiplied by 70, namely, an image is equally divided into a plurality of Patch with fixed sizes, the truth of each Patch is respectively judged, a matrix is output, and then the average value is taken as the final output of the discriminator.
A system for analyzing an electroencephalogram signal of a patient with motor imagery stroke by using cycleGAN, the system comprises,
the EEG signal acquisition and preprocessing module is used for acquiring EEG signals of a stroke patient and a healthy person and preprocessing the EEG signals; wherein, the EEG signal adopts a non-invasive acquisition method, and electrode caps of 64 electrodes are utilized; the pretreatment comprises the steps of removing eye charge, removing artifacts and down-sampling;
the spectrogram generating module is used for converting one-dimensional EEG signals of a stroke patient and a healthy person into two-dimensional spectrograms through EEG2 Image; the working process of the spectrogram generating module is as follows:
extracting mu and beta frequency bands of each EEG signal;
secondly, calculating the square sum of absolute values of the mu frequency band and the beta frequency band;
combining absolute values of the mu frequency band and the beta frequency band into a matrix, and inputting the matrix into an EEG2Image to obtain a spectrogram;
the training module is used for inputting the spectrogram of a healthy person and the spectrogram of a stroke patient into a cycleGAN neural network simultaneously for training, and the spectrogram of the stroke patient is used as a comparison sample, so that the spectrogram of the healthy person learns the characteristics of the spectrogram of the stroke patient to generate an artificial spectrogram based on the stroke patient, and the aims of artificially generating EEG data of the stroke patient and expanding the EEG data volume based on the stroke patient are fulfilled; wherein, the training condition is observed and the training progress is checked through the TensorBoard in the training process.
Preferably, the structure of the CycleGAN neural network comprises two generators and two discriminators, the purpose of the structure of the CycleGAN neural network is to map a source distribution S to a target distribution T with G; the CycleGAN neural network has two cycles, one of which is S → g (S)) → P (g (S)), and the formula for the loss of resistance of the cycle is as follows:
LGAN(G,Dt,S,T)＝Et～pdata(t)[logDT(t)]+Es～pdata(s)[log(1-DT(G(s)))]；
wherein G represents a generator, the goal of G is to generate G (S), let G (S) and goalSample similarities in distribution T; dtRepresenting a discriminator, DtThe goal of (a) is to distinguish between G (S) generated and control samples in the S domain; t represents data of a cerebral apoplexy patient, namely control data; s represents data of a healthy person; e represents an expected value; when D is presentT(t)＝1，DTWhen (g (s)) is 0, then LGAN(G,DtS, T) ═ 0, the goal of the optimization is to make the generator losses smaller and smaller;
another cycle is T → p (T) → G (p (T)), and the formula for the cyclic antagonistic loss is as follows:
LGAN(P,Ds,T,S)＝Es～pdata(s)[logDS(t)]+Et～pdata(t)[log(1-DS(G(t)))]；
wherein, P represents a generator, the goal of P is to generate P (T) and let P (T) be similar to the sample in the target distribution S; dsRepresenting a discriminator, DsThe goal of (a) is to distinguish between P (T) produced by P and a control sample in the T domain; t represents data of a cerebral apoplexy patient, namely control data; s represents data of a healthy person; e represents an expected value; when D is presentT(t)＝1，Ds(G (t)) ═ 0, then LGAN(P,Ds,T,S)＝0；
The generator G and the generator P of the CycleGAN neural network respectively generate samples with similar S-domain and T-domain distributions, and can convert back, namely after the picture of the S-domain is converted into the T-domain space, the picture of the S-domain can be converted into the S-domain, and the pictures of all the S-domains are converted into the same picture in the T-domain space by an end-to-end module, so that the CycleGAN neural network is a cyclic countermeasure network, the cyclic consistency loss exists in the cyclic process and is called as cyclic countermeasure loss, and the formula is as follows:
Lcyc(G,P)＝Es～pdata(s)||[P(G(s))-s||1]+Et～pdata(t)[||P(G(t))-t||1]；
wherein, the input of G is s, which is used to generate a false t-map (fake t); the input of P is t, which is used to generate a false s-map (fake s); after s is taken as the input of G, fake t is generated, and then the fake t is input into P to obtain a fake s graph; theoretically, the false s-map should be comparable to the original input s-map;
in summary, the total loss of the CycleGAN neural network is:
L(G,P,DS,DT)＝LGAN(G,DT,S,T)+LGAN(P,DS,T,S)+λLcyc(G,P)；
wherein, λ represents a balance parameter, and the value is preferably 10;
wherein, the generator is a network composed of an encoder, a converter and a decoder, the network is wholly subjected to a down-sampling process and an up-sampling process, a residual block is arranged in the middle, a spectrogram is generated through EEG2Image, and the spectrogram is processed into a tfrecrds form;
the discriminator is a convolution network consisting of five convolution layers, the first four convolution layers extract the characteristics of the spectrogram, and the last convolution layer judges whether the spectrogram belongs to a T domain or an S domain; the convolution network adopts a PatchGAN structure, the size of the PatchGAN structure is 70 multiplied by 70, namely, an image is equally divided into a plurality of Patch with fixed sizes, the truth of each Patch is respectively judged, a matrix is output, and then the average value is taken as the final output of the discriminator.
An electronic device, comprising: a memory and at least one processor;
wherein the memory stores computer-executable instructions;
the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform a method of identifying motor imagery of brain electrical signals of a stroke patient as described by CycleGAN.
A computer readable storage medium, which stores computer executable instructions, and when a processor executes the computer, the method for recognizing motor imagery of brain electrical signals of a stroke patient by CycleGAN as described above is implemented.
The method for analyzing the electroencephalogram signals of the patient with the motor imagery stroke by utilizing the cycleGAN has the following advantages:
the invention relates to a generation countermeasure Network (GAN) in Deep Learning, which mainly utilizes a Cycle-Consistent countermeasure Network (CYCLEGAN) structure to identify a spectrogram of a cerebral apoplexy patient and generate an artificial spectrogram based on the cerebral apoplexy patient, namely utilizes the CYCLEGAN structure to identify the spectrogram of the cerebral apoplexy patient and learns the characteristics of the spectrogram to generate the artificial spectrogram, thereby achieving the purposes of artificially generating EEG data of the cerebral apoplexy patient and expanding EEG data volume based on the cerebral apoplexy patient and enabling the research of an MI-BCI system to progress on DL (Deep Learning);
the method converts the one-dimensional EEG signals into a two-dimensional spectrogram, and combines a cycleGAN network, so that one-to-one mapping between training data is not required to be established between a source domain and a target domain, migration is realized, EEG data of a stroke patient are generated manually finally, the EEG data volume of the stroke patient is enlarged, the deep learning network is favorably used for analyzing and processing the EEG signals, and the development of a deep learning technology on the analysis and processing of the one-dimensional EEG signals is promoted;
the method comprises the steps that (1) a spectrogram of a stroke patient is identified by utilizing a cycleGAN structure and an artificial spectrogram based on the stroke patient is generated, specifically, in the process that a generator and an identifier are mutually played, the spectrogram is used as input, the spectrogram of a healthy person learns the spectrogram characteristics of the stroke patient, so that the artificial spectrogram based on the stroke patient is generated, and the generated spectrogram can very visually see the similarity with an original spectrogram; the result shows that the method has better effect, can effectively expand the EEG data volume of the patient with stroke, is beneficial to analyzing and processing the EEG signal by using a deep learning network, and promotes the development of the deep learning technology on the analysis and processing of the one-dimensional EEG signal.
Drawings
The invention is further described below with reference to the accompanying drawings.
FIG. 1 is a flow chart of a method for analyzing an electroencephalogram signal of a patient with motor imagery stroke by using cycleGAN;
FIG. 2 is a schematic of the EEG2Image flow chart algorithm;
FIG. 3 is a schematic structural diagram of CycleGAN;
FIG. 4 is a schematic diagram of a generator network of the cycleGAN;
FIG. 5 is a schematic diagram of a network structure of a CycleGAN discriminator.
Detailed Description
The method for analyzing the electroencephalogram signals of a patient with stroke including motor imagery by using CycleGAN according to the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.
Example 1:
as shown in the attached figure 1, the method for analyzing the electroencephalogram signals of the stroke patient including motor imagery by using the CycleGAN is to combine the CycleGAN network to artificially generate the EEG data of the stroke patient so as to expand the EEG data volume of the stroke patient; the method comprises the following specific steps:
s1, acquiring and preprocessing EEG signals: collecting EEG signals of a stroke patient and a healthy person, and preprocessing the EEG signals;
the EEG signal used by the present invention employs a non-invasive acquisition method using a 64-electrode cap. The hair needs to be kept dry and tidy before EEG signals are collected; after the electrode cap is worn, the conductive paste needs to be injected, and the experiment can be carried out after the resistance is kept stable. EEG data were collected in a hospital and 70 experiments were performed on each volunteer, including 60 imaginary right and left hand grasping movements and 10 imaginary nail grasping tasks (patients imagine affected limbs to grasp nails), i.e. the data format was 70X 64X 10000.
Wherein, the pretreatment is as follows:
s101, carrying out eye-removing and artifact-removing processing on the collected EEG signal data by using Curry 8;
s102, processing the EEG signal by adopting a down-sampling mode; the down-sampling is to take out partial sampling points at equal intervals in an original sampling sequence to be used as a preprocessed data set;
for example: 10000 original sampling points, set sampling interval of 10, that is, 1000 points are sampled from 10000 points of each channel as a preprocessed data set, and the format of the data set is 70 × 64 × 1000.
S2, generating a spectrogram: converting one-dimensional EEG signals of a stroke patient and a healthy person into a two-dimensional spectrogram through EEG2 Image; as shown in fig. 2, the following is detailed:
s201, extracting mu and beta frequency bands of each EEG signal; wherein, the preferable value range of mu is 8-13Hz, and the preferable value range of beta is 18-26 Hz;
s202, calculating the square sum of absolute values of mu and beta frequency bands, and converting the data format into 70 x 64;
s203, combining the absolute values of the mu and beta bands into a matrix, wherein the data format is 70 x 128, and inputting the data into an EEG2Image to obtain a spectrogram.
S3, training and generating an artificial spectrogram: simultaneously inputting a spectrogram of a healthy person and a spectrogram of a stroke patient into a cycleGAN neural network for training, and taking the spectrogram of the stroke patient as a comparison sample, so that the spectrogram of the healthy person learns the characteristics of the spectrogram of the stroke patient to generate an artificial spectrogram based on the stroke patient, thereby achieving the purposes of artificially generating EEG data of the stroke patient and expanding the EEG data volume based on the stroke patient; wherein, the training condition is observed and the training progress is checked through the TensorBoard in the training process.
As shown in fig. 3, the structure of the CycleGAN neural network includes two generators and two discriminators, and the purpose of the structure of the CycleGAN neural network is to map a source distribution S to a target distribution T by G; the CycleGAN neural network has two cycles, one of which is S → g (S)) → P (g (S)), and the formula for the loss of resistance of the cycle is as follows:
LGAN(G,Dt,S,T)＝Et～pdata(t)[logDT(t)]+Es～pdata(s)[log(1-DT(G(s)))]；
wherein G represents a generator, the goal of G is to generate G (S), let G (S) resemble the samples in the target distribution T; dtRepresenting a discriminator, DtThe goal of (a) is to distinguish between G (S) generated and control samples in the S domain; t represents data of a cerebral apoplexy patient, namely control data; s represents data of a healthy person; e represents an expected value; when D is presentT(t)＝1，DTWhen (g (s)) is 0, then LGAN(G,DtS, T) is 0, the optimization goal isThe loss of the generator is reduced;
another cycle is T → p (T) → G (p (T)), and the formula for the cyclic antagonistic loss is as follows:
LGAN(P,Ds,T,S)＝Es～pdata(s)[logDS(t)]+Et～pdata(t)[log(1-DS(G(t)))]；
wherein, P represents a generator, the goal of P is to generate P (T) and let P (T) be similar to the sample in the target distribution S; dsRepresenting a discriminator, DsThe goal of (a) is to distinguish between P (T) produced by P and a control sample in the T domain; t represents data of a cerebral apoplexy patient, namely control data; s represents data of a healthy person; e represents an expected value; when D is presentT(t)＝1，Ds(G (t)) ═ 0, then LGAN(P,Ds,T,S)＝0；
The generator G and the generator P of the CycleGAN neural network respectively generate samples with similar S-domain and T-domain distributions, and can convert back, namely after the picture of the S-domain is converted into the T-domain space, the picture of the S-domain can be converted into the S-domain, and the pictures of all the S-domains are converted into the same picture in the T-domain space by an end-to-end module, so that the CycleGAN neural network is a cyclic countermeasure network, the cyclic consistency loss exists in the cyclic process and is called as cyclic countermeasure loss, and the formula is as follows:
Lcyc(G,P)＝Es～pdata(s)[||P(G(s))-s||1]+Et～pdata(t)[||P(G(t))-t||1]；
wherein, the input of G is s, which is used to generate a false t-map (fake t); the input of P is t, which is used to generate a false s-map (fake s); after s is taken as the input of G, fake t is generated, and then the fake t is input into P to obtain a fake s graph; theoretically, the false s-map should be comparable to the original input s-map;
in summary, the total loss of the CycleGAN neural network is:
L(G,P,DS,DT)＝LGAN(G,DT,S,T)+LGAN(P,DS,T,S)+λLcyc(G,P)；
wherein, λ represents a balance parameter, and the value is preferably 10.
As shown in fig. 4, the generator is a network of encoders, converters and decoders, which goes through a down-sampling and an up-sampling process as a whole, and in the middle is a residual block, which produces a spectrogram by EEG2Image, processing the spectrogram into tfrecrds form; the working process of the generator is as follows:
s301, inputting a spectrogram of a healthy person into a generator G, and inputting a spectrogram of a stroke patient into a generator P;
s302, extracting characteristics of the spectrogram through a convolutional layer of an encoder through the encoder;
s303, the converter converts the feature vector of the spectrogram from an S domain to a T domain or from the T domain to the S domain according to the features extracted from the convolutional layer of the encoder;
s304, the decoder restores low-level features from the feature vectors by using a deconvolution network, and converts the low-level features into a spectrogram.
For example: the format input to the generator is 256 × 256 × 3, so 9 residual blocks are used in the network, each Reset module is a neural network layer composed of two convolutional layers, and the goal of preserving the original image characteristics during conversion can be achieved; the method specifically comprises the following steps:
firstly, inputting an image into an encoder, wherein the data format is 256 multiplied by 3;
secondly, extracting features from the input image by using a convolution network to obtain an output of 64 multiplied by 256;
thirdly, the network structure in the converter is mainly used for combining different similar features of the image, then determining how to convert the feature vector of the image from the S domain to the feature vector of the T domain based on the features, and finally obtaining the output of 64 multiplied by 256;
and fourthly, entering a decoder, recovering low-level features from the feature vector by using a deconvolution network, and finally converting the low-level features into a 256 × 256 × 3 image.
As shown in fig. 5, the discriminator is a convolutional network composed of five convolutional layers, the first four convolutional layers extract the features of the spectrogram, and the last convolutional layer determines whether the spectrogram belongs to the T domain or the S domain; the convolution network adopts a PatchGAN structure, the size of the PatchGAN structure is 70 multiplied by 70, namely, an image is equally divided into a plurality of Patch with fixed sizes, the truth of each Patch is respectively judged, a matrix is output, and then the average value is taken as the final output of the discriminator.
In the training process, because the least square method loss is more stable in the training process, the least square method loss is used for replacing the original logarithmic loss function, and therefore a high-quality spectrogram is easier to obtain. Second λ LcycThe value of lambda in (G, P) in the training process is 10, an Adam optimizer is used, and the value of batch _ size is 1; the learning rate in the first 100 epochs is 0.0002, and in the following training process, the learning rate is continuously reduced to 0 finally.
In the game process of the generator and the discriminator, the spectrogram of the healthy person learns the characteristics of the spectrogram of the stroke patient, so that more artificial spectrograms based on the stroke patient are generated to expand a stroke data set, the artificial spectrograms based on the stroke patient are finally generated through a CycleGAN neural network structure, the spectrograms of the healthy person are all converted into the spectrograms of the stroke patient, the aim of artificially generating EEG data of the stroke patient is fulfilled, and the aim of expanding the EEG data volume based on the stroke patient is fulfilled. In the training process, the training result can be judged by observing the loss of the generator and the discriminator; after the spectrogram is generated, the similarity between the original sample and the artificial data can be visually displayed. Compared with other generation models, the model is simpler in proving the usability of generated data and does not need to be substituted into other algorithms or formulas for proving, so that the EEG data generation model for the stroke patient, provided by the invention, applies the GAN to the MI-BCI field and obtains a satisfactory generation result.
Example 2:
as shown in fig. 6, the system for analyzing the electroencephalogram signals of the patient with motor imagery stroke by using CycleGAN of the present invention comprises,
the EEG signal acquisition and preprocessing module is used for acquiring EEG signals of stroke patients and healthy people, has a data format of 70 x 64 x 10000, and preprocesses the EEG signals; wherein, the EEG signal adopts a non-invasive acquisition method, and electrode caps of 64 electrodes are utilized; the pretreatment comprises the steps of removing eye charge, removing artifacts and down-sampling;
the spectrogram generating module is used for converting one-dimensional EEG signals of a stroke patient and a healthy person into two-dimensional spectrograms through EEG2 Image; as shown in fig. 2, the working process of the spectrogram generating module is as follows:
extracting a mu frequency band (8-13Hz) and a beta frequency band (18-26Hz) of each EEG signal;
calculating the square sum of absolute values of the mu frequency band and the beta frequency band, and converting the data format into 70 multiplied by 64;
combining absolute values of the mu frequency band and the beta frequency band into a matrix, inputting the matrix into an EEG2Image to obtain a spectrogram, wherein the data format is 70 multiplied by 128;
the training module is used for inputting the spectrogram of a healthy person and the spectrogram of a stroke patient into a cycleGAN neural network simultaneously for training, and the spectrogram of the stroke patient is used as a comparison sample, so that the spectrogram of the healthy person learns the characteristics of the spectrogram of the stroke patient to generate an artificial spectrogram based on the stroke patient, and the aims of artificially generating EEG data of the stroke patient and expanding the EEG data volume based on the stroke patient are fulfilled; wherein, the training condition is observed and the training progress is checked through the TensorBoard in the training process.
As shown in fig. 3, the structure of the CycleGAN neural network includes two generators and two discriminators, and the purpose of the structure of the CycleGAN neural network is to map a source distribution S to a target distribution T with G; the CycleGAN neural network has two cycles, one of which is S → g (S)) → P (g (S)), and the formula for the loss of resistance of the cycle is as follows:
LGAN(G,Dt,S,T)＝Et～pdata(t)[logDT(t)]+Es～pdata(s)[log(1-DT(G(s)))]；
wherein G represents a generator, the goal of G is to generate G (S), let G (S) resemble the samples in the target distribution T; dtRepresenting a discriminator, DtThe goal of (a) is to distinguish control samples in the G (S) and S domains of G generation(ii) a t represents data of a cerebral apoplexy patient, namely control data; s represents data of a healthy person; e represents an expected value; when D is presentT(t)＝1，DTWhen (g (s)) is 0, then LGAN(G,DtS, T) ═ 0, the goal of the optimization is to make the generator losses smaller and smaller;
another cycle is T → p (T) → G (p (T)), and the formula for the cyclic antagonistic loss is as follows:
LGAN(P,Ds,T,S)＝Es～pdata(s)[logDS(t)]+Et～pdata(t)[log(1-DS(G(t)))]；
wherein, P represents a generator, the goal of P is to generate P (T) and let P (T) be similar to the sample in the target distribution S; dsRepresenting a discriminator, DsThe goal of (a) is to distinguish between P (T) produced by P and a control sample in the T domain; t represents data of a cerebral apoplexy patient, namely control data; s represents data of a healthy person; e represents an expected value; when D is presentT(t)＝1，Ds(G (t)) ═ 0, then LGAN(P,Ds,T,S)＝0；
The generator G and the generator P of the CycleGAN neural network respectively generate samples with similar S-domain and T-domain distributions, and can convert back, namely after the picture of the S-domain is converted into the T-domain space, the picture of the S-domain can be converted into the S-domain, and the pictures of all the S-domains are converted into the same picture in the T-domain space by an end-to-end module, so that the CycleGAN neural network is a cyclic countermeasure network, the cyclic consistency loss exists in the cyclic process and is called as cyclic countermeasure loss, and the formula is as follows:
Lcyc(G,P)＝Es～pdata(s)[||P(G(s))-s||1]+Et～pdata(t)[||P(G(t))-t||1]；
wherein, the input of G is s, which is used to generate a false t-map (fake t); the input of P is t, which is used to generate a false s-map (fake s); after s is taken as the input of G, fake t is generated, and then the fake t is input into P to obtain a fake s graph; theoretically, the false s-map should be comparable to the original input s-map;
in summary, the total loss of the CycleGAN neural network is:
L(G,P,DS,DT)＝LGAN(G,DT,S,T)+LGAN(P,DS,T,S)+λLcyc(G,P)；
wherein, λ represents a balance parameter, and the value is preferably 10;
as shown in fig. 4, the generator is a network of encoders, converters and decoders, which goes through a down-sampling and an up-sampling process as a whole, and in the middle is a residual block, which produces a spectrogram by EEG2Image, processing the spectrogram into tfrecrds form;
as shown in fig. 5, the discriminator is a convolutional network composed of five convolutional layers, the first four convolutional layers extract the features of the spectrogram, and the last convolutional layer determines whether the spectrogram belongs to the T domain or the S domain; the convolution network adopts a PatchGAN structure, the size of the PatchGAN structure is 70 multiplied by 70, namely, an image is equally divided into a plurality of Patch with fixed sizes, the truth of each Patch is respectively judged, a matrix is output, and then the average value is taken as the final output of the discriminator.
Example 3:
an embodiment of the present invention further provides an electronic device, including: a memory and a processor;
wherein the memory stores computer-executable instructions;
the one processor executes the computer-executable instructions stored in the memory, so that the one processor executes the method for analyzing the electroencephalogram signals of the brain patients including motor imagery stroke by using CycleGAN as in embodiment 1.
Example 4:
the embodiment of the invention also provides a computer-readable storage medium, wherein a plurality of instructions are stored, and the instructions are loaded by a processor, so that the processor executes the method for analyzing the electroencephalogram signals of the patient with the motor imagery stroke by using CycleGAN in the embodiment 1 of the invention. Specifically, a system or an apparatus equipped with a storage medium on which software program codes that realize the functions of any of the above-described embodiments are stored may be provided, and a computer (or a CPU or MPU) of the system or the apparatus is caused to read out and execute the program codes stored in the storage medium.
In this case, the program code itself read from the storage medium can realize the functions of any of the above-described embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.
Examples of the storage medium for supplying the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (e.g., CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD + RW), a magnetic tape, a nonvolatile memory card, and a ROM. Alternatively, the program code may be downloaded from a server computer via a communications network.
Further, it should be clear that the functions of any one of the above-described embodiments may be implemented not only by executing the program code read out by the computer, but also by causing an operating system or the like operating on the computer to perform a part or all of the actual operations based on instructions of the program code.
Further, it is to be understood that the program code read out from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion unit connected to the computer, and then causes a CPU or the like mounted on the expansion board or the expansion unit to perform part or all of the actual operations based on instructions of the program code, thereby realizing the functions of any of the above-described embodiments.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A method for analyzing EEG signals of a patient with stroke containing motor imagery by using cycleGAN is characterized in that EEG data of the patient with stroke are generated artificially by combining a cycleGAN network so as to expand EEG data volume of the patient with stroke; the method comprises the following specific steps:
acquisition and pre-processing of EEG signals: collecting EEG signals of a stroke patient and a healthy person, and preprocessing the EEG signals;
generating a spectrogram: converting one-dimensional EEG signals of a stroke patient and a healthy person into a two-dimensional spectrogram through EEG2 Image;
training and generating artificial frequency spectrogram: the spectrogram of a healthy person and the spectrogram of a cerebral apoplexy patient are simultaneously input into a cycleGAN neural network for training, and the spectrogram of the cerebral apoplexy patient is used as a comparison sample, so that the spectrogram of the healthy person learns the characteristics of the spectrogram of the cerebral apoplexy patient to generate an artificial spectrogram based on the cerebral apoplexy patient.
2. The method for analyzing EEG signals of patients with stroke including motor imagery by using cycleGAN as claimed in claim 1, wherein said EEG signals are acquired non-invasively by using an electrode cap with 64 electrodes;
the pretreatment is as follows:
carrying out eye-removing and artifact-removing treatment on the collected EEG signal data by using Curry 8;
processing the EEG signal by adopting a down-sampling mode; the down-sampling is to take out partial sampling points at equal intervals in the original sampling sequence as a preprocessed data set.
3. The method for analyzing electroencephalogram signals of patients with stroke involving motor imagery by using CycleGAN as claimed in claim 1, wherein the generated spectrogram specifically comprises:
extracting the μ and β bands of each EEG signal;
calculating the sum of the squares of the absolute values of the mu and beta bands;
the absolute values of the μ and β bands are combined into a matrix and input into the EEG2Image to obtain a spectrogram.
4. The method for analyzing the EEG signals of a patient with stroke including motor imagery by using the CycleGAN according to claim 1, 2 or 3, wherein the structure of the CycleGAN neural network comprises two generators and two discriminators, the purpose of the structure of the CycleGAN neural network is to map a source distribution S to a target distribution T by using G; the CycleGAN neural network has two cycles, one of which is S → g (S)) → P (g (S)), and the formula for the loss of resistance of the cycle is as follows:
LGAN(G,Dt,S,T)＝Et～pdata(t)[logDT(t)]+Es～pdata(s)[log(1-DT(G(s)))]；
wherein G represents a generator, the goal of G is to generate G (S), let G (S) resemble the samples in the target distribution T; dtRepresenting a discriminator, DtThe goal of (a) is to distinguish between G (S) generated and control samples in the S domain; t represents data of a cerebral apoplexy patient, namely control data; s represents data of a healthy person; e represents an expected value; when D is presentT(t)＝1，DTWhen (g (s)) is 0, then LGAN(G,DtS, T) ═ 0, the goal of the optimization is to make the generator losses smaller and smaller;
another cycle is T → p (T) → G (p (T)), and the formula for the cyclic antagonistic loss is as follows:
LGAN(P,Ds,T,S)＝Es～pdata(s)[logDS(t)]+Et～pdata(t)[log(1-DS(G(t)))]；
wherein, P represents a generator, the goal of P is to generate P (T) and let P (T) be similar to the sample in the target distribution S; dsRepresenting a discriminator, DsThe goal of (a) is to distinguish between P (T) produced by P and a control sample in the T domain; t represents data of a cerebral apoplexy patient, namely control data; s represents data of a healthy person; e represents an expected value; when D is presentS(s)＝1，Ds(G (t)) ═ 0, then LGAN(P,Ds,T,S)＝0；
The generator G and the generator P of the CycleGAN neural network respectively generate samples with similar S-domain and T-domain distributions, and can convert back, namely after the picture of the S-domain is converted into the T-domain space, the picture of the S-domain can be converted into the S-domain, so that the CycleGAN neural network is a cyclic confrontation network, and cyclic consistency loss exists in the cyclic process, namely cyclic confrontation loss, and the formula is as follows:
Lcyc(G,P)＝Es～pdata(s)[||P(G(s))-s||1]+Et～pdata(t)[||P(G(t))-t||1]；
wherein, the input of G is s, which is used to generate a false t-map (fake t); the input of P is t, which is used for generating a false s-map (fakes); after s is taken as the input of G, fake t is generated, and then the fake t is input into P to obtain a fake s graph;
in summary, the total loss of the CycleGAN neural network is:
L(G,P,DS,DT)＝LGAN(G,DT,S,T)+LGAN(P,DS,T,S)+λLcyc(G,P)；
where λ represents the equilibrium parameter.
5. The method of claim 4, wherein the generator is a network of encoders, converters and decoders, the network being entirely subjected to a down-sampling and an up-sampling process, with the residual block in between, and the spectrogram is generated by EEG2Image, processed into tfrecrds form; the working process of the generator is as follows:
inputting a spectrogram of a healthy person into a generator G, and inputting a spectrogram of a stroke patient into a generator P;
the spectrogram passes through an encoder, and features are extracted through a convolutional layer of the encoder;
the converter transforms the feature vector of the spectrogram from an S domain to a T domain or from the T domain to the S domain according to the features extracted by the convolutional layer of the encoder;
the decoder uses a deconvolution network to recover low-level features from the feature vectors, converting the low-level features into a spectrogram.
6. The method for analyzing electroencephalogram signals of patients with stroke containing motor imagery by using CycleGAN as claimed in claim 4, wherein the discriminator is a convolutional network composed of five convolutional layers, the first four convolutional layers extract the characteristics of the spectrogram, and the last convolutional layer judges whether the spectrogram belongs to a T domain or an S domain;
the convolution network adopts a PatchGAN structure, the size of the PatchGAN structure is 70 multiplied by 70, namely, an image is equally divided into a plurality of Patch with fixed sizes, the truth of each Patch is respectively judged, a matrix is output, and then the average value is taken as the final output of the discriminator.
7. A system for analyzing an electroencephalogram signal of a patient with motor imagery stroke by using cycleGAN, which is characterized by comprising,
the EEG signal acquisition and preprocessing module is used for acquiring EEG signals of a stroke patient and a healthy person and preprocessing the EEG signals; wherein, the EEG signal adopts a non-invasive acquisition method, and electrode caps of 64 electrodes are utilized; the pretreatment comprises the steps of removing eye charge, removing artifacts and down-sampling;
the spectrogram generating module is used for converting one-dimensional EEG signals of a stroke patient and a healthy person into two-dimensional spectrograms through EEG2 Image; the working process of the spectrogram generating module is as follows:
extracting mu and beta frequency bands of each EEG signal;
secondly, calculating the square sum of absolute values of the mu frequency band and the beta frequency band;
combining absolute values of the mu frequency band and the beta frequency band into a matrix, and inputting the matrix into an EEG2Image to obtain a spectrogram;
the training module is used for inputting the spectrogram of the healthy person and the spectrogram of the cerebral apoplexy patient into the cycleGAN neural network at the same time for training, and the spectrogram of the cerebral apoplexy patient is used as a comparison sample, so that the spectrogram of the healthy person learns the characteristics of the spectrogram of the cerebral apoplexy patient to generate an artificial spectrogram based on the cerebral apoplexy patient.
8. The system for analyzing electroencephalograms including motor imagery stroke patients using CycleGAN as claimed in claim 7, wherein the structure of CycleGAN neural network comprises two generators and two discriminators, the purpose of the structure of CycleGAN neural network is to map source distribution S to target distribution T with G; the CycleGAN neural network has two cycles, one of which is S → g (S)) → P (g (S)), and the formula for the loss of resistance of the cycle is as follows:
LGAN(G,Dt,S,T)＝Et～pdata(t)[logDT(t)]+Es～pdata(s)[log(1-DT(G(s)))]；
wherein G represents a generator, the goal of G is to generate G (S), let G (S) resemble the samples in the target distribution T; dtRepresenting a discriminator, DtThe goal of (a) is to distinguish between G (S) generated and control samples in the S domain; t represents data of a cerebral apoplexy patient, namely control data; s represents data of a healthy person; e represents an expected value; when D is presentT(t)＝1，DTWhen (g (s)) is 0, then LGAN(G,DtS, T) ═ 0, the goal of the optimization is to make the generator losses smaller and smaller;
another cycle is T → p (T) → G (p (T)), and the formula for the cyclic antagonistic loss is as follows:
LGAN(P,Ds,T,S)＝Es～pdata(s)[logDS(t)]+Et～pdata(t)[log(1-DS(G(t)))]；
wherein, P represents a generator, the goal of P is to generate P (T) and let P (T) be similar to the sample in the target distribution S; dsRepresenting a discriminator, DsThe goal of (a) is to distinguish between P (T) produced by P and a control sample in the T domain; t represents data of a cerebral apoplexy patient, namely control data; s represents data of a healthy person; e represents an expected value; when D is presentT(t)＝1，Ds(G (t)) ═ 0, then LGAN(P,Ds,T,S)＝0；
The generator G and the generator P of the CycleGAN neural network respectively generate samples with similar S-domain and T-domain distributions, and can convert back, namely after the picture of the S-domain is converted into the T-domain space, the picture of the S-domain can be converted into the S-domain, so that the CycleGAN neural network is a cyclic confrontation network, and cyclic consistency loss exists in the cyclic process, namely cyclic confrontation loss, and the formula is as follows:
Lcyc(G,P)＝Es～pdata(s)[||P(G(s))-s||1]+Et～pdata(t)[||P(G(t))-t||1]；
wherein, the input of G is s, which is used to generate a false t-map (fake t); the input of P is t, which is used for generating a false s-map (fakes); after s is taken as the input of G, fake t is generated, and then the fake t is input into P to obtain a fake s graph;
in summary, the total loss of the CycleGAN neural network is:
L(G,P,DS,DT)＝LGAN(G,DT,S,T)+LGAN(P,DS,T,S)+λLcyc(G,P)；
wherein λ represents a balance parameter;
wherein, the generator is a network composed of an encoder, a converter and a decoder, the network is wholly subjected to a down-sampling process and an up-sampling process, a residual block is arranged in the middle, a spectrogram is generated through EEG2Image, and the spectrogram is processed into a tfrecrds form;
the discriminator is a convolution network consisting of five convolution layers, the first four convolution layers extract the characteristics of the spectrogram, and the last convolution layer judges whether the spectrogram belongs to a T domain or an S domain; the convolution network adopts a PatchGAN structure, the size of the PatchGAN structure is 70 multiplied by 70, namely, an image is equally divided into a plurality of Patch with fixed sizes, the truth of each Patch is respectively judged, a matrix is output, and then the average value is taken as the final output of the discriminator.
9. An electronic device, comprising: a memory and at least one processor;
wherein the memory stores computer-executable instructions;
the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the method of CycleGAN for identifying motor imagery in brain electrical signals of a stroke patient as claimed in any one of claims 1 to 6.
10. A computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, which when executed by a processor, implement the method for CycleGAN to identify motor imagery in brain electrical signals of stroke patients as claimed in claims 1 to 6.
CN202010527458.8A 2020-06-11 2020-06-11 Method for analyzing electroencephalogram signals of brain patients with motor imagery stroke by using cycleGAN Pending CN111523520A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010527458.8A CN111523520A (en) 2020-06-11 2020-06-11 Method for analyzing electroencephalogram signals of brain patients with motor imagery stroke by using cycleGAN

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010527458.8A CN111523520A (en) 2020-06-11 2020-06-11 Method for analyzing electroencephalogram signals of brain patients with motor imagery stroke by using cycleGAN

Publications (1)

Publication Number Publication Date
CN111523520A true CN111523520A (en) 2020-08-11

Family

ID=71909621

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010527458.8A Pending CN111523520A (en) 2020-06-11 2020-06-11 Method for analyzing electroencephalogram signals of brain patients with motor imagery stroke by using cycleGAN

Country Status (1)

Country Link
CN (1) CN111523520A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112450946A (en) * 2020-11-02 2021-03-09 杭州电子科技大学 Electroencephalogram artifact restoration method based on loop generation countermeasure network
CN113763241A (en) * 2020-11-19 2021-12-07 四川大学 Depth self-learning image super-resolution method based on similar image guidance

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108446667A (en) * 2018-04-04 2018-08-24 北京航空航天大学 Based on the facial expression recognizing method and device for generating confrontation network data enhancing
CN108564039A (en) * 2018-04-16 2018-09-21 北京工业大学 A kind of epileptic seizure prediction method generating confrontation network based on semi-supervised deep layer
CN109620223A (en) * 2018-12-07 2019-04-16 北京工业大学 A kind of rehabilitation of stroke patients system brain-computer interface key technology method
CN110163180A (en) * 2019-05-29 2019-08-23 长春思帕德科技有限公司 Mental imagery eeg data classification method and system
CN111000555A (en) * 2019-11-29 2020-04-14 中山大学 Training data generation method, automatic recognition model modeling method and automatic recognition method for epilepsia electroencephalogram signals
CN111012336A (en) * 2019-12-06 2020-04-17 重庆邮电大学 Parallel convolutional network motor imagery electroencephalogram classification method based on spatio-temporal feature fusion

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108446667A (en) * 2018-04-04 2018-08-24 北京航空航天大学 Based on the facial expression recognizing method and device for generating confrontation network data enhancing
CN108564039A (en) * 2018-04-16 2018-09-21 北京工业大学 A kind of epileptic seizure prediction method generating confrontation network based on semi-supervised deep layer
CN109620223A (en) * 2018-12-07 2019-04-16 北京工业大学 A kind of rehabilitation of stroke patients system brain-computer interface key technology method
CN110163180A (en) * 2019-05-29 2019-08-23 长春思帕德科技有限公司 Mental imagery eeg data classification method and system
CN111000555A (en) * 2019-11-29 2020-04-14 中山大学 Training data generation method, automatic recognition model modeling method and automatic recognition method for epilepsia electroencephalogram signals
CN111012336A (en) * 2019-12-06 2020-04-17 重庆邮电大学 Parallel convolutional network motor imagery electroencephalogram classification method based on spatio-temporal feature fusion

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
S.M. ABDELFATTACH ET.AL: "《Augmenting The Size of EEG datasets Using Generative Adversarial Networks》", 《2018 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN)》, 13 July 2018 (2018-07-13), pages 1 - 6 *
SIDDHARTH: "《Utilizing Deep Learning Towards Multi-modal Bio-sensing and Vision-based Affective Computing》", 《ARXIV:1905.07039V1 [CS.LG]》, 16 May 2019 (2019-05-16), pages 1 - 12 *
ZHU J Y: "《Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks》", 《ARXIV:1703.10593V1 [CS.CV]》, 30 March 2017 (2017-03-30), pages 1 - 18 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112450946A (en) * 2020-11-02 2021-03-09 杭州电子科技大学 Electroencephalogram artifact restoration method based on loop generation countermeasure network
CN113763241A (en) * 2020-11-19 2021-12-07 四川大学 Depth self-learning image super-resolution method based on similar image guidance

Similar Documents

Publication Publication Date Title
Zhao et al. A multi-branch 3D convolutional neural network for EEG-based motor imagery classification
Autthasan et al. MIN2Net: End-to-end multi-task learning for subject-independent motor imagery EEG classification
CN110876626B (en) Depression detection system based on optimal lead selection of multi-lead electroencephalogram
CN104091172B (en) A kind of feature extracting method of Mental imagery EEG signals
CN108363493B (en) User characteristic model establishing method and system based on brain-computer interface and storage medium
CN110555468A (en) Electroencephalogram signal identification method and system combining recursion graph and CNN
CN111329474A (en) Electroencephalogram identity recognition method and system based on deep learning and information updating method
CN114533086B (en) Motor imagery brain electrolysis code method based on airspace characteristic time-frequency transformation
CN111329469A (en) Arrhythmia prediction method
CN111523520A (en) Method for analyzing electroencephalogram signals of brain patients with motor imagery stroke by using cycleGAN
CN105266804B (en) A kind of brain-electrical signal processing method based on low-rank and sparse matrix decomposition
CN111695500A (en) Method and system for recognizing motor imagery task of stroke patient based on transfer learning
CN117520891A (en) Motor imagery electroencephalogram signal classification method and system
CN108523883A (en) A kind of continuous Mental imagery identifying system of left and right index finger based on actual act modeling
CN115562488A (en) Motor imagery brain-computer interface communication method, device, system, medium and equipment
CN113662560A (en) Method for detecting seizure-like discharge between attacks, storage medium and device
Ghonchi et al. Spatio-temporal deep learning for EEG-fNIRS brain computer interface
Xie et al. A brain controlled wheelchair based on common spatial pattern
Ai et al. Flexible coding scheme for robotic arm control driven by motor imagery decoding
CN116421200A (en) Brain electricity emotion analysis method of multi-task mixed model based on parallel training
CN116369950A (en) Target detection method based on electroencephalogram tracing and multi-feature extraction
CN109685031A (en) A kind of brain-computer interface midbrain signal characteristics classification method and system
CN113662561A (en) Electroencephalogram feature extraction method and device of sub-band cascade common space mode
CN112450946A (en) Electroencephalogram artifact restoration method based on loop generation countermeasure network
Wang et al. Generative adversarial networks for electroencephalogram signal analysis: A mini review

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200811

RJ01 Rejection of invention patent application after publication