CN113705464B - Domain adaptation method for solving characteristic migration problem in motor imagery brain-computer interface - Google Patents
Domain adaptation method for solving characteristic migration problem in motor imagery brain-computer interface Download PDFInfo
- Publication number
- CN113705464B CN113705464B CN202111003357.1A CN202111003357A CN113705464B CN 113705464 B CN113705464 B CN 113705464B CN 202111003357 A CN202111003357 A CN 202111003357A CN 113705464 B CN113705464 B CN 113705464B
- Authority
- CN
- China
- Prior art keywords
- data
- test
- value
- motor imagery
- experiment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000013508 migration Methods 0.000 title claims abstract description 24
- 230000005012 migration Effects 0.000 title claims abstract description 24
- 230000006978 adaptation Effects 0.000 title claims abstract description 14
- 238000012549 training Methods 0.000 claims abstract description 85
- 238000013507 mapping Methods 0.000 claims abstract description 35
- 210000004556 brain Anatomy 0.000 claims abstract description 11
- 238000012360 testing method Methods 0.000 claims description 148
- 238000002474 experimental method Methods 0.000 claims description 68
- 239000011159 matrix material Substances 0.000 claims description 48
- 238000001914 filtration Methods 0.000 claims description 28
- 238000012545 processing Methods 0.000 claims description 26
- 238000000605 extraction Methods 0.000 claims description 6
- 238000013480 data collection Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000009826 distribution Methods 0.000 abstract description 27
- 230000006870 function Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003183 myoelectrical effect Effects 0.000 description 1
- 230000035484 reaction time Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/02—Preprocessing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/12—Classification; Matching
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Dermatology (AREA)
- General Health & Medical Sciences (AREA)
- Neurology (AREA)
- Neurosurgery (AREA)
- Human Computer Interaction (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
A domain adaptation method for solving the characteristic migration problem in a motor imagery brain-computer interface belongs to the technical field of migration learning in the motor imagery brain-computer interface. The invention solves the problem that in the motor imagery brain-computer interface, the MI-BCI classification accuracy is low due to the deviation caused by the brain-electrical signal characteristic migration. According to the invention, the required mapping is obtained according to the sample distribution of the source domain and the target domain, the sample distribution of the target domain becomes a new sample distribution after mapping, the difference between the new sample distribution and the sample distribution of the source domain is minimum, the sample distribution of the target domain is changed through mapping, so that the consistency of the sample distribution of the target domain and the sample distribution of the training domain in the brain-computer interface is maintained, the problem of deviation caused by the characteristic migration of the brain electrical signal is solved, and the classification accuracy of the new sample distribution is effectively improved. The invention can be applied to the feature migration in the motor imagery brain-computer interface.
Description
Technical Field
The invention belongs to the technical field of migration learning in a motor imagery brain-computer interface, and particularly relates to a domain adaptation method for solving the problem of feature migration in the motor imagery brain-computer interface.
Background
The Brain-computer interface (Brain-Computer Interface, BCI) provides a direct connection mode between the human Brain and the electronic equipment, and realizes the information exchange between the human Brain and the external equipment. The brain-computer interfaces can be divided into different brain-computer interfaces according to the types of the brain-computer signals. Among them, brain-computer interfaces based on motor imagery electroencephalogram signals (Motor Imagery Electroencephalogram, MI-EEG) are widely used because they are active and related to motion. However, there is a problem of brain-electrical signal feature migration in the motor imagery brain-computer interface (Motor Imagery Brain-Computer Interface, MI-BCI), and the classification accuracy of the MI-BCI is reduced due to the deviation caused by the feature migration, so that the problem cannot be solved by the machine learning algorithm in the traditional brain-computer interface, and thus the problem becomes a main reason for limiting the development of the MI-BCI.
Disclosure of Invention
The invention aims to solve the problem that the MI-BCI classification accuracy is low due to the deviation caused by the characteristic migration of an electroencephalogram signal in a motor imagery brain-computer interface, and provides a domain adaptation method for solving the characteristic migration problem in the motor imagery brain-computer interface.
The technical scheme adopted by the invention for solving the technical problems is as follows: a domain adaptation method for solving a problem of feature migration in a motor imagery brain-computer interface, the method comprising in particular the steps of:
taking a person using a brain-computer interface system as a tested, performing M motor imagery experiments on the tested person altogether, and collecting brain electrical signal data of the tested person in the whole process from the beginning of the motor imagery experiments to the end of the motor imagery experiments in each motor imagery experiment process;
dividing the acquired electroencephalogram data into a training set and a testing set, and numbering the data in the training set and the testing set according to the corresponding experimental sequence;
step two, intercepting electroencephalogram data of a motor imagery period in the training set and the testing set;
step three, band-pass filtering is carried out on the intercepted electroencephalogram signal data, and electroencephalogram signal data after band-pass filtering is obtained;
step four, processing the training set data and the test set data by adopting a CSP feature extraction algorithm to obtain a spatial filter;
the method comprises the steps of performing spatial filtering on electroencephalogram signal data after bandpass filtering processing corresponding to each experiment in a training set and a testing set by using a spatial filter, respectively obtaining spatial filtering matrixes corresponding to acquired data of each experiment in the training set and the testing set, respectively processing the spatial filtering matrixes, obtaining characteristic matrixes of the acquired electroencephalogram signal data of each experiment, and performing normalization processing on the characteristic matrixes;
training the SVM function by utilizing a normalized feature matrix corresponding to the training set data to obtain a trained model;
step six, calculating a threshold value threshold by using a normalized feature matrix of the training set data, selecting a plurality of groups of test data from the test set data according to rules, and calculating an average feature value of each group of test data by using the normalized feature matrix corresponding to the test set data;
selecting a plurality of groups of test data from the test set data according to rules, and calculating the average characteristic value of each group of test data by using a normalized characteristic matrix corresponding to the test set data; the specific process is as follows:
the sequence number l is 1 to M 1 Sequentially take values, serial number j 0 At 1 to M 2 Sequentially take values of M 2 Representing the total experiment times corresponding to the test set data;
the sequence number l takes a value from 1, and when l=1, the sequence number j 0 Sequentially take pass 1,2, …, M 2 When the serial number j 0 When the value of (a) is smaller than the value of the sequence number l, the 1 st to the j-th in the test set 0 -1 experimental data is selected as a set of data; when j is 0 When the test set is greater than or equal to l, j is the j in the test set 0 -l times to j th 0 -1 experimental data is selected as a set of data; up to sequence number j 0 Sequentially traversing 1,2, …, M 2 After that, M is obtained together 2 Group data, respectively calculating the average characteristic value of each group of data;
similarly, the sequence number l is sequentially taken from 1 to M 1 For each value of the sequence number l, M is correspondingly obtained 2 Group data;
respectively combining the average characteristic value of each group of test data with the average characteristic value of all data of the training setMake difference toPerforming a step seven on the test data group with the 2-norm value of the difference result larger than the threshold value threshold;
step seven, after the mapping F corresponding to each group of test data is calculated respectively, calculating new test data corresponding to the experiment by using the calculated mapping F;
for a certain value of the sequence number l, updating corresponding data in the test set by using new test data calculated under the value to obtain updated test set data under the value;
similarly, updated test set data under each value of the sequence number l is obtained respectively;
step eight, respectively processing the new test data of each experiment obtained in the step seven by using a CSP feature extraction algorithm, and respectively obtaining a feature matrix of the new test data; the spatial filter adopted in the processing is the same as the spatial filter obtained in the step four;
classifying feature matrixes of the test set data updated under each value of the sequence number l by using the model trained in the fifth step, and comparing the classification result with labels of the test set data to obtain classification accuracy of the test set data updated under each value of the sequence number l;
selecting a serial number l corresponding to the highest classification accuracy, and taking the selected l value as the tested test data step length;
step nine, classifying the acquired tested actual electroencephalogram signal data according to the step length of the test data obtained in the step eight;
the specific process of the step nine is as follows:
step nine, firstly, acquiring actual electroencephalogram signal data to be tested to construct a training set A, wherein the training set A contains acquired M 3 Secondary data, wherein the time length of each data acquisition is the same;
step nine, carrying out the processing of the step three and the step four on the data in the training set A to obtain a normalized feature matrix of the data in the training set A;
training the SVM function by utilizing the normalized feature matrix of the data in the training set A to obtain a trained model;
step nine, calculating a new threshold value threshold' by using the normalized feature matrix obtained in the step nine two;
step nine, acquiring actual electroencephalogram signal data of a tested once in real time;
after calculating the characteristic value of the secondary acquisition data, making a difference between the characteristic value of the secondary acquisition data and the average characteristic value of the data in the training set A, and if the 2-norm value of the difference result is smaller than or equal to a threshold value threshold', classifying the acquired secondary acquisition data by using the model trained in the step nine two to obtain a classification result;
otherwise, if the 2-norm value of the difference result is greater than the threshold value threshold', calculating mapping by adopting the method of the step seven, obtaining mapped new data corresponding to the acquired data according to the mapping, and classifying the mapped new data by utilizing the model trained in the step nine two to obtain a classification result;
step nine, acquiring actual electroencephalogram signal data to be tested in real time, and calculating average characteristic values of the data acquired at the current time and the data acquired in the step nine; the average characteristic value is differenced from the average characteristic value of the data in the training set A, and if the 2-norm value of the differenced result is smaller than or equal to a new threshold value threshold', the data acquired at the present time are classified by using the model trained in the step nine two to obtain a classification result; otherwise, the 2-norm value of the difference result is larger than a new threshold value threshold', mapping is calculated by adopting the method of the step seven, new mapped data corresponding to the current acquired data are obtained according to mapping, and the new mapped data are classified by utilizing the model trained in the step nine two to obtain a classification result;
step nine and six, repeatedly executing the process of step nine and five, and when the number of times of data collection exceeds the test data step length l corresponding to the tested data 0 Previously, all calculated average characteristic values of data acquired from the step nine and four are obtained; in excess of the test data step length l 0 Thereafter, the most recently acquired l is calculated 0 Average eigenvalues of the secondary data;
until the motor imagery brain-computer interface is stopped.
The beneficial effects of the invention are as follows: the invention obtains the required mapping according to the sample distribution of the source domain (training set) and the target domain (testing set), the sample distribution of the target domain becomes a new sample distribution after mapping, the difference between the new sample distribution and the sample distribution of the source domain is minimum, the consistency of the sample distribution of the target domain and the training domain in the brain-computer interface is maintained by changing the sample distribution of the target domain through mapping, the problem of deviation caused by the characteristic migration of the brain electrical signal is solved, and the classification accuracy of the new sample distribution is effectively improved.
Drawings
FIG. 1 is a flow chart of an experimental paradigm for acquiring brain electrical signals;
fig. 2 is a flowchart of applying the migration learning method of KL divergence to a motor imagery brain-computer interface.
Detailed Description
Detailed description of the inventionin the first embodiment, this embodiment will be described with reference to fig. 2. The domain adaptation method for solving the problem of feature migration in a motor imagery brain-computer interface according to the embodiment specifically includes the following steps:
taking a person using a brain-computer interface system as a tested, performing M motor imagery experiments on the tested person altogether, and collecting brain electrical signal data of the tested person in the whole process from the beginning of the motor imagery experiments to the end of the motor imagery experiments in each motor imagery experiment process;
dividing the acquired electroencephalogram data into a training set and a testing set, respectively numbering the data in the training set and the testing set according to the corresponding experimental sequence, wherein the data in the training set and the testing set are numbered sequentially from 1, and each number corresponds to the electroencephalogram data acquired in one experimental process;
step two, intercepting electroencephalogram data of a motor imagery period in the training set and the testing set;
step three, band-pass filtering is carried out on the intercepted electroencephalogram signal data, and electroencephalogram signal data after band-pass filtering is obtained;
carrying out band-pass filtering on the data to filter high-frequency electro-oculogram and myoelectric interference, wherein the optimal filtering frequency ranges of different tested bands are different, and in the experiment of the invention, the cut-off frequency of the tested pass bands of A01 and A09 is 7-18Hz, and the cut-off frequency of the stop band is 5-20Hz; the passband cut-off frequency of the test A02 and A05 is 18-28Hz, and the stopband cut-off frequency is 15-33Hz; the rest tested pass band cut-off frequency is 7-30Hz, and the stop band cut-off frequency is 4-35Hz;
step four, processing the training set data and the test set data by adopting a CSP feature extraction algorithm to obtain a spatial filter;
the method comprises the steps of performing spatial filtering on electroencephalogram signal data after bandpass filtering processing corresponding to each experiment in a training set and a testing set by using a spatial filter, respectively obtaining spatial filtering matrixes corresponding to acquired data of each experiment in the training set and the testing set, respectively processing the spatial filtering matrixes, obtaining characteristic matrixes of the acquired electroencephalogram signal data of each experiment, and performing normalization processing on the characteristic matrixes;
training the SVM function by utilizing a normalized feature matrix corresponding to training set data to obtain a trained model;
the LIBSVM toolbox in MATLAB has integrated SVM functions;
step six, calculating a threshold value threshold by using a normalized feature matrix of the training set data, selecting a plurality of groups of test data from the test set data according to rules, and calculating an average feature value of each group of test data by using the normalized feature matrix corresponding to the test set data;
selecting a plurality of groups of test data from the test set data according to rules, and calculating the average characteristic value of each group of test data by using a normalized characteristic matrix corresponding to the test set data; the specific process is as follows:
the sequence number l is 1 to M 1 Sequentially take values, serial number j 0 At 1 to M 2 Sequentially take values of M 2 Representing the total experiment times corresponding to the test set data;
the sequence number l takes a value from 1, and when l=1, the sequence number j 0 Sequentially take pass 1,2, …, M 2 When the serial number j 0 When the value of (a) is smaller than the value of the sequence number lThe 1 st to the j th in the test set 0 -1 experimental data is selected as a set of data; when j is 0 When the test set is greater than or equal to l, j is the j in the test set 0 -l times to j th 0 -1 experimental data is selected as a set of data; up to sequence number j 0 Sequentially traversing 1,2, …, M 2 After that, M is obtained together 2 Group data, respectively calculating the average characteristic value of each group of data;
similarly, the sequence number l is sequentially taken from 1 to M 1 For each value of the sequence number l, M is correspondingly obtained 2 Group data;
respectively combining the average characteristic value of each group of test data with the average characteristic value of all data of the training setMaking a difference, and executing a step seven on the test data group with the 2-norm value of the difference result larger than the threshold value threshold;
calculation ofWherein (1)>For the average eigenvalue of a certain set of test data, if +.>Considering that the characteristic distribution difference between the group of test data and the training set data is larger, executing the seventh step by using the KL divergence domain adaptation algorithm, otherwise executing the eighth step;
step seven, after the mapping F corresponding to each group of test data is calculated respectively, calculating new test data corresponding to the experiment by using the calculated mapping F;
for a certain value of the sequence number l, updating corresponding data in the test set by using new test data calculated under the value to obtain updated test set data under the value;
similarly, updated test set data under each value of the sequence number l is obtained respectively;
step eight, respectively processing the new test data of each experiment obtained in the step seven by using a CSP feature extraction algorithm, and respectively obtaining a feature matrix of the new test data; the spatial filter adopted in the processing is the same as the spatial filter obtained in the step four;
classifying feature matrixes of the test set data updated under each value of the sequence number l by using the model trained in the fifth step, and comparing the classification result with labels of the test set data to obtain classification accuracy of the test set data updated under each value of the sequence number l;
selecting a serial number l corresponding to the highest classification accuracy, and taking the selected l value as the tested test data step length;
and step nine, classifying the acquired actual brain wave signal data according to the step length of the test data obtained in the step eight.
The migration learning is used as a method for solving the problem of the target field by using the existing knowledge, so that the problem of the characteristic migration of the brain-computer signal can be effectively solved, and the domain adaptation method is more suitable for a motor imagery brain-computer interface due to the high adaptability. The basic idea of the domain adaptation method is to build a mapping, and the sample distribution of the target domain becomes a new sample distribution after mapping, and the difference between the new sample distribution and the sample distribution of the source domain is minimum.
The invention uses KL divergence to measure the size of the sample distribution difference between the source domain and the target domain. The smaller the KL divergence value, the smaller the difference in the two sample distributions. According to the invention, the required mapping is obtained by minimizing KL divergence values of sample distribution of the source domain and the target domain, and the sample distribution of the target domain is changed through mapping, so that the consistency of the sample distribution of the target domain and the source domain in the brain-computer interface is maintained, and the problem of characteristic migration is solved.
The second embodiment is as follows: this embodiment will be described with reference to fig. 1. The first embodiment is different from the first embodiment in that, in the first step, a specific process of the motor imagery experiment is as follows:
the test is right against the computer screen, t=0 at the beginning of the experiment, and a fixed appears on the screenCross sign of (c) until t=t 1 An arrow prompt appears on the screen in seconds, the arrow prompt points to the left or points to the right, wherein the arrow prompt points to the left corresponding imagined left hand movement, the arrow prompt points to the right corresponding imagined right hand movement, and the arrow prompt lasts for t 0 Seconds until t=t 2 The cross symbol disappears in seconds, at t 1 ～t 2 During the period, the tested person always performs motor imagery;
at t=t 2 After seconds, enter a rest state for a rest time of t 0 After 'seconds', the current motor imagery experiment is completed; collecting the current motor imagery experiment from t=0 to t=t 2 +t 0 ' s electroencephalogram data;
the above motor imagery experiment process was repeated M times for the test co-ordinates.
And each time a motor imagery experiment is carried out, acquiring a tested brain electrical signal correspondingly, and acquiring 144 groups of experimental data for the tested brain electrical signal, wherein the number of times of imagining right hand movement is the same as the number of times of imagining left hand movement in the tested experiment, and the sampling rate of a data set is 250Hz. The first 34 groups of experimental data are used as training sets for training a model, the second 110 groups of experimental data are used as test sets, and the experimental degree of freedom is 2, namely, the experimental data only comprise left and right hand motor imagery. And the collected data is subjected to band-pass filtering of 0.5-100 Hz and power frequency notch filtering of 50Hz.
Other steps and parameters are the same as in the first embodiment.
And a third specific embodiment: this embodiment differs from the one or two embodiments in that t 1 Has a value of 2, t 0 Has a value of 1.25, t 2 Has a value of 6, t 0 ' has a value of 1.5.
Other steps and parameters are the same as in the first or second embodiment.
The specific embodiment IV is as follows: the difference between the present embodiment and one of the first to third embodiments is that the specific process of the second step is:
and intercepting the electroencephalogram data of the motor imagery period from the acquired electroencephalogram data by using a time window with the offset of 0.5 seconds and the window length of 3 seconds.
The collected experimental data are labeled differently according to the experimental contents of different time periods, wherein 768 labels represent the experimental start, 769 labels represent left-hand motion imagination prompts, 770 represent right-hand motion imagination prompts, and 32766 represents a new set of experimental start. A time window with a bias of 0.5s and a window length of 3s is set to fetch the data corresponding to tags 769 and 770.
In this embodiment, the time window with the window length of 3 seconds is adopted for interception, so that the data corresponding to the tested reaction time can be removed, and the calculated amount is reduced, and meanwhile, the electroencephalogram data in the motor imagery period can be efficiently extracted.
Other steps and parameters are the same as in one to three embodiments.
Fifth embodiment: in the fourth step, spatial filtering is performed on the electroencephalogram data after bandpass filtering processing corresponding to each experiment in the training set and the test set by using a spatial filter, so as to respectively obtain spatial filter matrixes corresponding to the acquired data of each experiment in the training set and the test set, and after the spatial filter matrixes are respectively processed, a feature matrix corresponding to the acquired electroencephalogram data of each experiment is obtained; the specific process is as follows:
for electroencephalogram signal data after bandpass filtering processing corresponding to an experiment, multiplying the data by a spatial filter, obtaining a multiplication result which is still a matrix, and solving a variance of the multiplication result to obtain a spatial filter matrix, namely solving a variance of data in the matrix obtained by multiplication;
extracting a first row, a second row and a rearmost two rows of the spatial filter matrix to serve as a characteristic matrix of electroencephalogram signal data acquired in the experiment;
and similarly, respectively obtaining the feature matrix of the electroencephalogram signal data acquired in each experiment in the training set and the testing set.
Other steps and parameters are the same as in one to four embodiments.
Specific embodiment six: in the sixth step, the threshold is calculated by using the normalized feature matrix of the training set data, and the calculation method is as follows:
selecting the kth to (k+l-1) -th experimental electroencephalogram data in the training set, and taking the selected data as a group of data, wherein the serial number l is 1-M 1 The values are sequentially taken, and the serial number k is 1 to M 1 The values between l+1 are sequentially taken, which are specifically as follows:
the sequence number l takes the value from 1, and when l=1, k takes the values of 1,2, … and M in sequence 1 -l+1, each time k takes a value, a corresponding set of data is obtained, i.e. when l=1, M is obtained altogether 1 -l+1 set of data, M 1 Representing the total experiment times corresponding to the training set data; similarly, sequentially taking l for 2-M 1 Obtaining each group of data;
for any group of data obtained, calculating the average eigenvalue of the group of data by using the normalized eigenvalue matrix corresponding to the group of data
Wherein C is lk For a data set imagining left-hand movement in the set of data, C rk For a data set of imagined right hand movement in the set of data,representing set C lk Total number of experiments corresponding to data in +.>Representing set C rk Total number of experiments corresponding to data in +.>Representing set C lk Characteristic values of normalized characteristic matrix of ith experimental data in (a),/th experimental data>Representing set C rk The characteristic value of the normalized characteristic matrix of the j-th experimental data;
after the data of each group are traversed, average characteristic values of the data of each group are obtained respectively;
calculating average characteristic value of all experimental data in training set
Wherein C is l Data set representing imagined left hand movement in training set, C r Data set representing imagined right hand movement in training set, L DL Representing set C l The total experiment times corresponding to the data in L DR Representing set C r The total number of experiments corresponding to the data in (c),representing set C l Normalized eigenvalue of eigenvalue matrix of ith experimental data in (i),. Sup.+ -. Of +>Representing set C r The eigenvalue of the normalized eigenvalue matrix of the j' th experimental data;
using the average eigenvalue sum of each set of dataCalculating a threshold value threshold:
wherein,representation->2-norm of>Representing the average eigenvalue of each group of data with +.>After the difference is made, the 2-norm is calculated according to the difference result corresponding to each group of data, and the calculated maximum 2-norm result is used as a threshold value threshold.
Other steps and parameters are the same as in one of the first to fifth embodiments.
Seventh embodiment: in the seventh step, after calculating the mapping F corresponding to each set of test data, new test data corresponding to the experiment is calculated by using the calculated mapping F; the specific process is as follows:
computing the ith in the training set 0 Covariance of the secondary experimental dataWherein (1)>Is the ith in training set 0 The secondary experimental data, tr (·) is the trace of the matrix, the mean covariance of the training set data +.>
Covariance of nth experimental data in a set of test dataWherein T is n For the nth experimental data in the set of test data, the mean covariance of the set of test data +.>The method comprises the following steps:
wherein C is l1 Represents when j 0 When < l, sequence number j 0 Previous j 0 -a set of left-hand motor imagery data with a corresponding SVM classification probability of more than 65% in 1 experimental data, C r1 Represents when j 0 When < l, sequence number j 0 Previous j 0 -a set of right-hand motor imagery data with a corresponding SVM classification probability of greater than 65% in the 1-time experimental data;representing the probability that the n-th experimental data is left-hand motor imagery through SVM classification results, and +.>Representing the probability that the N-th experimental data is classified by SVM as the right-hand motor imagery, N L And N R Respectively represent the set C l1 And C r1 Total experiment times corresponding to the data;
set C l1 The probability of classifying the experimental data into left-hand motor imagery data by the SVM is more than 65%; set C r1 The probability of classifying the experimental data into right-hand motor imagery data by the SVM is more than 65%;
C l2 represents when j 0 When not less than l, j in the group of test data 0 -l times to j th 0 -1 set of experimental data corresponding to left-hand motor imagery data with a SVM classification probability greater than 65%, C r2 Represents when j 0 When not less than l, j in the group of test data 0 -l times to j th 0 -1 set of experimental data corresponding to a right-hand motor imagery data with a SVM classification probability greater than 65%;
by means ofAnd->Computing the map->Calculating the sequence number j corresponding to the group of test data according to the mapping F 0 Test data of->Corresponding new test data->
And respectively carrying out the processing on each group of test data to be processed.
Other steps and parameters are the same as in one of the first to sixth embodiments.
Eighth embodiment: this embodiment is different from one of the first to seventh embodiments in that the specific process of the step nine is:
step nine, firstly, acquiring actual electroencephalogram signal data to be tested to construct a training set A, wherein the training set A contains acquired M 3 Secondary data, wherein the time length of each data acquisition is the same;
step nine, carrying out the processing of the step three and the step four on the data in the training set A to obtain a normalized feature matrix of the data in the training set A;
training the SVM function by utilizing the normalized feature matrix of the data in the training set A to obtain a trained model;
step nine, calculating a new threshold value threshold' by using the normalized feature matrix obtained in the step nine two; the method for calculating the new threshold value threshold' is the same as that in the step six;
step nine, acquiring actual electroencephalogram signal data of a tested once in real time;
after calculating the characteristic value of the secondary acquisition data, making a difference between the characteristic value of the secondary acquisition data and the average characteristic value of the data in the training set A, and if the 2-norm value of the difference result is smaller than or equal to a threshold value threshold', classifying the acquired secondary acquisition data by using the model trained in the step nine two to obtain a classification result;
otherwise, if the 2-norm value of the difference result is greater than the threshold value threshold', calculating mapping by adopting the method of the step seven, obtaining mapped new data corresponding to the acquired data according to the mapping, and classifying the mapped new data by utilizing the model trained in the step nine two to obtain a classification result;
step nine, acquiring actual electroencephalogram signal data to be tested in real time, and calculating average characteristic values of the data acquired at the current time and the data acquired in the step nine; the average characteristic value is differenced from the average characteristic value of the data in the training set A, and if the 2-norm value of the differenced result is smaller than or equal to a new threshold value threshold', the data acquired at the present time are classified by using the model trained in the step nine two to obtain a classification result; otherwise, the 2-norm value of the difference result is larger than a new threshold value threshold', mapping is calculated by adopting the method of the step seven, new mapped data corresponding to the current acquired data are obtained according to mapping, and the new mapped data are classified by utilizing the model trained in the step nine two to obtain a classification result;
step nine and six, repeatedly executing the process of step nine and five, and when the number of times of data collection exceeds the test data step length l corresponding to the tested data 0 Before (i.e. from step nine to four, the number of data acquisitions does not exceed the test data step l 0 ) All calculated are average characteristic values of data acquired from the ninth and fourth steps; in excess of the test data step length l 0 Thereafter, the most recently acquired l is calculated 0 Average eigenvalues of the secondary data;
until the motor imagery brain-computer interface is stopped.
The method of the present embodiment can be used for any test. When the mapping is calculated in step nine, whenStarting from the ninth and fourth steps, the number of times of data acquisition does not exceed the step length l of the test data 0 When mean covarianceIs the average covariance of all data acquired from step nine four, when the number of data acquired from step nine four exceeds the test data step l 0 Mean covariance +.>Is the most recently acquired l 0 Mean covariance of the secondary data.
The above examples of the present invention are only for describing the calculation model and calculation flow of the present invention in detail, and are not limiting of the embodiments of the present invention. Other variations and modifications of the above description will be apparent to those of ordinary skill in the art, and it is not intended to be exhaustive of all embodiments, all of which are within the scope of the invention.
Claims (4)
1. A domain adaptation method for solving a problem of feature migration in a motor imagery brain-computer interface, the method comprising the steps of:
taking a person using a brain-computer interface system as a tested, performing M motor imagery experiments on the tested person altogether, and collecting brain electrical signal data of the tested person in the whole process from the beginning of the motor imagery experiments to the end of the motor imagery experiments in each motor imagery experiment process;
dividing the acquired electroencephalogram data into a training set and a testing set, and numbering the data in the training set and the testing set according to the corresponding experimental sequence;
step two, intercepting electroencephalogram data of a motor imagery period in the training set and the testing set;
step three, band-pass filtering is carried out on the intercepted electroencephalogram signal data, and electroencephalogram signal data after band-pass filtering is obtained;
step four, processing the training set data and the test set data by adopting a CSP feature extraction algorithm to obtain a spatial filter;
the method comprises the steps of performing spatial filtering on electroencephalogram signal data after bandpass filtering processing corresponding to each experiment in a training set and a testing set by using a spatial filter, respectively obtaining spatial filtering matrixes corresponding to acquired data of each experiment in the training set and the testing set, respectively processing the spatial filtering matrixes, obtaining characteristic matrixes of the acquired electroencephalogram signal data of each experiment, and performing normalization processing on the characteristic matrixes;
the spatial filter is utilized to carry out spatial filtering on the electroencephalogram signal data after the bandpass filtering processing corresponding to each experiment in the training set and the testing set, so as to respectively obtain spatial filter matrixes corresponding to the acquired data of each experiment in the training set and the testing set, and the spatial filter matrixes are respectively processed so as to obtain characteristic matrixes of the acquired electroencephalogram signal data of the corresponding experiment; the specific process is as follows:
for electroencephalogram signal data after bandpass filtering processing corresponding to a certain experiment, multiplying a spatial filter with the data, and then solving variance of a multiplication result to obtain a spatial filter matrix;
extracting a first row, a second row and a rearmost two rows of the spatial filter matrix to serve as a characteristic matrix of electroencephalogram signal data acquired in the experiment;
similarly, feature matrixes of electroencephalogram signal data acquired in each experiment in a training set and a testing set are respectively obtained;
training the SVM function by utilizing a normalized feature matrix corresponding to the training set data to obtain a trained model;
step six, calculating a threshold value threshold by using a normalized feature matrix of the training set data, selecting a plurality of groups of test data from the test set data according to rules, and calculating an average feature value of each group of test data by using the normalized feature matrix corresponding to the test set data;
the method for calculating the threshold value threshold by using the normalized feature matrix of the training set data comprises the following steps:
selecting the kth to (k+l-1) -th experimental electroencephalogram data in the training set, and taking the selected data as a group of data, wherein the serial number l is 1-M 1 The values are sequentially taken, and the serial number k is 1 to M 1 The values between l+1 are sequentially taken, which are specifically as follows:
the sequence number l takes the value from 1, and when l=1, k takes the values of 1,2, … and M in sequence 1 -l+1, each time k takes a value, a corresponding set of data is obtained, i.e. when l=1, M is obtained altogether 1 -l+1 set of data, M 1 Representing the total experiment times corresponding to the training set data; similarly, sequentially taking l for 2-M 1 Obtaining each group of data;
for any group of data obtained, calculating the average eigenvalue of the group of data by using the normalized eigenvalue matrix corresponding to the group of data
Wherein C is lk For a data set imagining left-hand movement in the set of data, C rk For a data set of imagined right hand movement in the set of data,representing set C lk Total number of experiments corresponding to data in +.>Representing set C rk Total number of experiments corresponding to data in +.>Representing set C lk Characteristic values of normalized characteristic matrix of ith experimental data in (a),/th experimental data>Representing set C rk The characteristic value of the normalized characteristic matrix of the j-th experimental data;
after the data of each group are traversed, average characteristic values of the data of each group are obtained respectively;
calculating average characteristic value of all experimental data in training set
Wherein C is l Data set representing imagined left hand movement in training set, C r Data set representing imagined right hand movement in training set, L DL Representing set C l The total experiment times corresponding to the data in L DR Representing set C r The total number of experiments corresponding to the data in (c),representing set C l Normalized eigenvalue of eigenvalue matrix of ith experimental data in (i),. Sup.+ -. Of +>Representing set C r The eigenvalue of the normalized eigenvalue matrix of the j' th experimental data;
using the average eigenvalue sum of each set of dataCalculating a threshold value threshold:
wherein,representation->2-norm of>Representing the average eigenvalue of each group of data with +.>After the difference is made, respectively calculating 2-norms of difference making results corresponding to each group of data, and taking the calculated maximum 2-norms as a threshold value threshold;
selecting a plurality of groups of test data from the test set data according to rules, and calculating the average characteristic value of each group of test data by using a normalized characteristic matrix corresponding to the test set data; the specific process is as follows:
the sequence number l is 1 to M 1 Sequentially take values, serial number j 0 At 1 to M 2 Sequentially take values of M 2 Representing the total experiment times corresponding to the test set data;
the sequence number l takes a value from 1, and when l=1, the sequence number j 0 Sequentially take pass 1,2, …, M 2 When the serial number j 0 When the value of (a) is smaller than the value of the sequence number l, the 1 st to the j-th in the test set 0 -1 experimental data is selected as a set of data; when j is 0 When the test set is greater than or equal to l, j is the j in the test set 0 -l times to j th 0 -1 experimental data is selected as a set of data; up to sequence number j 0 Sequentially traversing 1,2, …, M 2 After that, M is obtained together 2 Group data, respectively calculating the average characteristic value of each group of data;
similarly, the sequence number l is sequentially taken from 1 to M 1 For each value of the sequence number l, M is correspondingly obtained 2 Group data;
respectively combining the average characteristic value of each group of test data with the average characteristic value of all data of the training setMaking a difference, and executing a step seven on the test data group with the 2-norm value of the difference result being larger than the threshold value threshold;
step seven, after the mapping F corresponding to each group of test data is calculated respectively, calculating new test data corresponding to the experiment by using the calculated mapping F;
for a certain value of the sequence number l, updating corresponding data in the test set by using new test data calculated under the value to obtain updated test set data under the value;
similarly, updated test set data under each value of the sequence number l is obtained respectively;
after the mapping F corresponding to each group of test data is calculated respectively, calculating new test data corresponding to the experiment by using the calculated mapping F; the specific process is as follows:
computing the ith in the training set 0 Covariance of the secondary experimental dataWherein (1)>Is the ith in training set 0 The secondary experimental data, tr (·) is the trace of the matrix, the mean covariance of the training set data +.>
Covariance of nth experimental data in a set of test dataWherein T is n For the nth experimental data in the set of test data, the mean covariance of the set of test data +.>The method comprises the following steps:
wherein C is l1 Represents when j 0 When < l, sequence number j 0 Previous j 0 -a set of left-hand motor imagery data with a corresponding SVM classification probability of more than 65% in 1 experimental data, C r1 Represents when j 0 When < l, sequence number j 0 Previous j 0 -a set of right-hand motor imagery data with a corresponding SVM classification probability of greater than 65% in the 1-time experimental data;representing the probability that the n-th experimental data is left-hand motor imagery through SVM classification results, and +.>Representing the probability that the N-th experimental data is classified by SVM as the right-hand motor imagery, N L And N R Respectively represent the set C l1 And C r1 Total experiment times corresponding to the data;
C l2 represents when j 0 When not less than l, j in the group of test data 0 -l times to j th 0 -1 set of experimental data corresponding to left-hand motor imagery data with a SVM classification probability greater than 65%, C r2 Represents when j 0 When not less than l, j in the group of test data 0 -l times to j th 0 -1 set of experimental data corresponding to a right-hand motor imagery data with a SVM classification probability greater than 65%;
by means ofAnd->Computing the map->Calculating the sequence number j corresponding to the group of test data according to the mapping F 0 Test data of->Corresponding new test data->
The test data of each group to be processed are respectively processed;
step eight, respectively processing the new test data of each experiment obtained in the step seven by using a CSP feature extraction algorithm, and respectively obtaining a feature matrix of the new test data; the spatial filter adopted in the processing is the same as the spatial filter obtained in the step four;
classifying feature matrixes of the test set data updated under each value of the sequence number l by using the model trained in the fifth step, and comparing the classification result with labels of the test set data to obtain classification accuracy of the test set data updated under each value of the sequence number l;
selecting a serial number l corresponding to the highest classification accuracy, and taking the selected l value as the tested test data step length;
step nine, classifying the acquired tested actual electroencephalogram signal data according to the step length of the test data obtained in the step eight;
the specific process of the step nine is as follows:
step nine, firstly, acquiring actual electroencephalogram signal data to be tested to construct a training set A, wherein the training set A contains acquired M 3 Secondary data, wherein the time length of each data acquisition is the same;
step nine, carrying out the processing of the step three and the step four on the data in the training set A to obtain a normalized feature matrix of the data in the training set A;
training the SVM function by utilizing the normalized feature matrix of the data in the training set A to obtain a trained model;
step nine, calculating a new threshold value threshold' by using the normalized feature matrix obtained in the step nine two;
step nine, acquiring actual electroencephalogram signal data of a tested once in real time;
after calculating the characteristic value of the secondary acquisition data, making a difference between the characteristic value of the secondary acquisition data and the average characteristic value of the data in the training set A, and if the 2-norm value of the difference result is smaller than or equal to a threshold value threshold', classifying the acquired secondary acquisition data by using the model trained in the step nine two to obtain a classification result;
otherwise, if the 2-norm value of the difference result is greater than the threshold value threshold', calculating mapping by adopting the method of the step seven, obtaining mapped new data corresponding to the acquired data according to the mapping, and classifying the mapped new data by utilizing the model trained in the step nine two to obtain a classification result;
step nine, acquiring actual electroencephalogram signal data to be tested in real time, and calculating average characteristic values of the data acquired at the current time and the data acquired in the step nine; the average characteristic value is differenced from the average characteristic value of the data in the training set A, if the 2-norm value of the difference result is smaller than or equal to a new threshold value threshold', the data acquired at the present time are classified by using the model trained in the step nine two, and a classification result is obtained; otherwise, the 2-norm value of the difference result is larger than a new threshold value threshold', the mapping is calculated by adopting the method of the step seven, new mapped data corresponding to the current acquired data are obtained according to the mapping, and the new mapped data are classified by utilizing the model trained in the step nine two, so that a classification result is obtained;
step nine and six, repeatedly executing the process of step nine and five, and when the number of times of data collection exceeds the test data step length l corresponding to the tested data 0 Previously, all calculated average characteristic values of data acquired from the step nine and four are obtained; in excess of the test data step length l 0 Thereafter, the most recently acquired l is calculated 0 Average eigenvalues of the secondary data;
until the motor imagery brain-computer interface is stopped.
2. The domain adaptation method for solving the problem of feature migration in a motor imagery brain-computer interface according to claim 1, wherein in the first step, the specific process of the motor imagery experiment is:
when the test is right against the computer screen, t=0 at the beginning of the test, a fixed cross symbol appears on the screen until t=t 1 An arrow prompt appears on the screen in seconds, the arrow prompt points to the left or points to the right, wherein the arrow prompt points to the left corresponding imagined left hand movement, the arrow prompt points to the right corresponding imagined right hand movement, and the arrow prompt lasts for t 0 Seconds until t=t 2 The cross symbol disappears in seconds, at t 1 ～t 2 During the period, the tested person always performs motor imagery;
at t=t 2 After seconds, enter a rest state for a rest time of t 0 After 'seconds', the current motor imagery experiment is completed; collecting the current motor imagery experiment from t=0 to t=t 2 +t 0 ' s electroencephalogram data;
the above motor imagery experiment process was repeated M times for the test co-ordinates.
3. A domain adaptation method for solving the problem of feature migration in a motor imagery brain-computer interface according to claim 2, wherein t is 1 Has a value of 2, t 0 Has a value of 1.25, t 2 Has a value of 6, t 0 ' has a value of 1.5.
4. A domain adaptation method for feature migration in a motor imagery brain-computer interface according to claim 3, wherein the specific process of step two is:
and intercepting the electroencephalogram data of the motor imagery period from the acquired electroencephalogram data by using a time window with the offset of 0.5 seconds and the window length of 3 seconds.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111003357.1A CN113705464B (en) | 2021-08-30 | 2021-08-30 | Domain adaptation method for solving characteristic migration problem in motor imagery brain-computer interface |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111003357.1A CN113705464B (en) | 2021-08-30 | 2021-08-30 | Domain adaptation method for solving characteristic migration problem in motor imagery brain-computer interface |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113705464A CN113705464A (en) | 2021-11-26 |
| CN113705464B true CN113705464B (en) | 2024-01-09 |
Family
ID=78656722
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111003357.1A Active CN113705464B (en) | 2021-08-30 | 2021-08-30 | Domain adaptation method for solving characteristic migration problem in motor imagery brain-computer interface |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113705464B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111931656A (en) * | 2020-08-11 | 2020-11-13 | 西安交通大学 | User independent motor imagery classification model training method based on transfer learning |
| CN112684891A (en) * | 2020-12-30 | 2021-04-20 | 杭州电子科技大学 | Electroencephalogram signal classification method based on multi-source manifold embedding migration |
| CN113010013A (en) * | 2021-03-11 | 2021-06-22 | 华南理工大学 | Wasserstein distance-based motor imagery electroencephalogram migration learning method |
-
2021
- 2021-08-30 CN CN202111003357.1A patent/CN113705464B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111931656A (en) * | 2020-08-11 | 2020-11-13 | 西安交通大学 | User independent motor imagery classification model training method based on transfer learning |
| CN112684891A (en) * | 2020-12-30 | 2021-04-20 | 杭州电子科技大学 | Electroencephalogram signal classification method based on multi-source manifold embedding migration |
| CN113010013A (en) * | 2021-03-11 | 2021-06-22 | 华南理工大学 | Wasserstein distance-based motor imagery electroencephalogram migration learning method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113705464A (en) | 2021-11-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110840402B (en) | Atrial fibrillation signal identification method and system based on machine learning | |
| CN110338786B (en) | Epileptic discharge identification and classification method, system, device and medium | |
| CN108363493B (en) | User characteristic model establishing method and system based on brain-computer interface and storage medium | |
| CN109657642A (en) | A kind of Mental imagery Method of EEG signals classification and system based on Riemann's distance | |
| CN110584660B (en) | Electrode selection method based on brain source imaging and correlation analysis | |
| CN110598676B (en) | Deep learning gesture electromyographic signal identification method based on confidence score model | |
| CN109770900B (en) | Method, system and device for issuing brain-computer interface instruction based on convolutional neural network | |
| CN113065526B (en) | Electroencephalogram signal classification method based on improved depth residual error grouping convolution network | |
| CN106108880B (en) | Automatic heart beat identification method and system | |
| CN112515685A (en) | Multi-channel electroencephalogram signal channel selection method based on time-frequency co-fusion | |
| CN112528834B (en) | Electroencephalogram signal cross-test classification method of subband target alignment co-space mode | |
| CN108154085B (en) | Method and device for identity recognition based on electrocardiogram data and electronic equipment | |
| Kang et al. | A method of denoising multi-channel EEG signals fast based on PCA and DEBSS algorithm | |
| CN113536882B (en) | Multi-class motor imagery electroencephalogram signal feature extraction and classification method | |
| CN110558975B (en) | Electrocardiosignal classification method and system | |
| CN111820876A (en) | Dynamic construction method of electroencephalogram spatial filter | |
| CN111310656A (en) | Single motor imagery electroencephalogram signal identification method based on multi-linear principal component analysis | |
| CN112465069A (en) | Electroencephalogram emotion classification method based on multi-scale convolution kernel CNN | |
| CN109770891B (en) | Electrocardiosignal preprocessing method and preprocessing device | |
| CN111832452A (en) | Feature optimization and recognition method of special gesture instruction based on electroencephalogram | |
| CN113197583A (en) | Electrocardiogram waveform segmentation method based on time-frequency analysis and recurrent neural network | |
| CN113705464B (en) | Domain adaptation method for solving characteristic migration problem in motor imagery brain-computer interface | |
| CN116807496B (en) | Method, device, equipment and medium for positioning epileptic interval brain wave abnormal signals | |
| CN111428601B (en) | P300 signal identification method, device and storage medium based on MS-CNN | |
| CN117520891A (en) | Motor imagery electroencephalogram signal classification method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |