CN112528834A

CN112528834A - Sub-band target alignment common space mode electroencephalogram signal cross-subject classification method

Info

Publication number: CN112528834A
Application number: CN202011422720.9A
Authority: CN
Inventors: 佘青山; 张献雄; 陈云; 梅从立
Original assignee: Hangzhou Dianzi University
Current assignee: Hangzhou Dianzi University
Priority date: 2020-12-08
Filing date: 2020-12-08
Publication date: 2021-03-19
Anticipated expiration: 2040-12-08
Also published as: CN112528834B

Abstract

The invention discloses a sub-band target alignment common space mode electroencephalogram signal cross-subject classification method. The electroencephalogram classification method comprises the steps of firstly carrying out band-pass filtering on an electroencephalogram signal to form a plurality of sub-frequency band signals (sub-band filtering), then extracting features by adopting a CSP algorithm, then selecting more representative features by a minimum redundancy maximum correlation method, and finally using a traditional linear discriminant analysis classifier for electroencephalogram classification. The invention combines sub-band filtering and target alignment, and can effectively improve the performance of cross-test classification.

Description

Sub-band target alignment common space mode electroencephalogram signal cross-subject classification method

Technical Field

The invention belongs to the field of pattern recognition, and relates to a feature extraction and cross-subject classification method for acquiring more frequency band information and providing a target alignment method to reduce domain distribution difference aiming at electroencephalogram signals.

Background

A brain-computer interface (BCI) is a human-computer interaction system based on brain signals, and has been widely applied to various fields such as exoskeleton rehabilitation robots, fatigue detection, smart home, entertainment games, and the like. In most BCI, electroencephalography (EEG) is the most commonly used input signal due to its advantages of being non-invasive and inexpensive, compared to signals such as Magnetoencephalography (MEG), functional magnetic resonance imaging (fMRI), and Positron Emission Tomography (PET). Motor Imagery (MI) is a popular example of an experiment. However, EEG is a very weak electrical signal acquired from the scalp, with poor spatial resolution, low signal-to-noise ratio, and significant differences between subjects and stages, which presents great difficulties to the analysis and identification of EEG signals. Therefore, the research of effective methods for extracting and identifying the electroencephalogram characteristics of different tasks becomes very significant.

In a brain-computer interface system based on electroencephalogram, decoding different electroencephalogram signals needs to be subjected to complex signal processing, feature extraction and classification. Common space mode (CSP) is a very popular and used feature extraction method. Its performance depends on the particular frequency band of the EEG, and classification using CSP features is less accurate when the EEG signal is filtered using an inappropriate frequency range. To address this problem, Novi and Guan propose subband co-spatial patterns (SBCSP) that use a set of filters to divide the EEG into multiple subbands, then input the extracted features into LDA to obtain scores that represent the classification capability of each subband, and finally fuse the scores to make a decision. Ang and Guan propose a filter bank common space mode (FBCSP) algorithm and design various feature selection methods to select more representative features. Both methods carry out subband filtering on original electroencephalogram data, new feature representation is obtained by fusion to obtain feature selection, and finally a better experimental result is obtained. Obviously, sub-band filtering becomes an effective electroencephalogram signal processing method.

However, methods such as SBCSP and FBCSP perform far more satisfactorily in cross-phase classification than in the cross-subject classification, because the individual differences between users are much greater than the differences between different phases of the same user. In recent years, more and more researchers introduce the idea of transfer learning into electroencephalogram classification to solve the problem of data distribution adaptation. Transfer learning utilizes data/information from one or more source domains to improve learning performance in target domains and has been successfully applied in BCI. Zanini et al propose a Riemann Alignment (RA) method to align EEG covariance matrices of different subjects, and the aligned covariance matrices as features can be directly classified using an MDRM classifier. Yair et al propose a method for domain adaptation using parallel transmission on the cone-streamlines of a symmetric positive definite matrix. He and Wu extend the RA method to the Euclidean Alignment (EA) method to align EEG tests from different subjects in euclidean space making them more consistent so that any euclidean space classifier can be used thereafter. Zhang and Wu propose a new method of popular embedded knowledge Migration (MEKT) that first aligns covariance matrices by RA to extract tangent space features, and then performs domain adaptation by minimizing joint probability distribution shifts between source and target domains while preserving their geometry.

Disclosure of Invention

Because the performance of the CSP depends on the working frequency band and the difficulty of cross-test classification is high, the invention provides a feature extraction method based on subband filtering and domain alignment, which is called a subband target alignment co-space mode (SBTACSP), and designs an electroencephalogram signal cross-test classification method based on the subband target alignment co-space mode.

The invention comprises the following steps:

step (1): and (4) preprocessing. Acquiring a multichannel motor imagery brain point signal of a subject; then, sub-band filtering is performed, and the electroencephalogram signal in a specific frequency range is filtered into a plurality of sub-band signals by a band-pass filter in a specified band-pass band.

Step (2): a new domain alignment method is provided, namely target alignment, source domain (test) samples are aligned to a target domain (training) space and applied to signals of each frequency band, only the source domain samples are aligned and changed, the target domain samples are not changed, and then the method is used for electroencephalogram signal feature extraction.

And (3): filtering the aligned electroencephalogram signals of each frequency band by using the CSPs respectively, and splicing the CSP characteristics of a plurality of frequency bands together to form a CSP characteristic vector with higher dimensionality.

And (4): and selecting p most representative features according to a minimum redundancy maximum correlation (mRMR) feature selection algorithm to form a final feature.

And (5): and classifying the features with the number p of the features after the feature selection by adopting an LDA classifier.

Compared with the FBCSP and EA-CSP-LDA methods, the method has the following advantages:

compared with a filter bank common space mode (FBCSP) method, the method has the advantages that the advantages of transfer learning in cross-tested classification are utilized, the inter-domain distribution difference is reduced, more representative features are obtained for classification, and the classification precision is improved. Compared with the Euclidean alignment method, the method adopts the sub-band filtering processing method to obtain more frequency band information, and improves the classification performance. Compared with the subband Euclidean alignment common space mode, the new domain alignment method provided by the invention, namely target alignment, is more suitable for being combined with subband filtering, and the obtained classification precision is more excellent.

Description of the drawings:

FIG. 1 is a flow chart of the method of the present invention;

FIG. 2(a) shows the result of 10-fold cross-validation of parameter p by BCI Competition IV Dataset IIa;

FIG. 2(b) shows the result of 10-fold cross-validation of parameter p by BCI Competition IV Dataset IIb;

FIG. 3(a) is a graph showing the visual effect of the characteristic t-SNE of CSP and SBTACSP of two subjects in the Dataset DataselIa in the multi-source domain classification;

FIG. 3(b) is a graph showing the visual effect of the characteristic t-SNE of CSP and SBTACSP of two subjects in the Dataset DataselIIb in the multi-source domain classification.

Detailed Description

The present invention is described in detail below with reference to the attached drawings.

In the field of brain-computer interfaces, a classical Common Spatial Pattern (CSP) method needs to effectively select a specific frequency band, and meanwhile, the high instability and individual variability of electroencephalogram signals increase the difficulty of cross-test classification. In order to solve these problems, the present invention proposes a Sub-Band Target Alignment common space mode (SBTACSP) method, and applies it to cross-test classification of electroencephalogram signals, and fig. 1 is a flowchart of the present invention.

As shown in fig. 1, the implementation of the method of the present invention mainly comprises 5 steps: (1) collecting multi-channel electroencephalogram signals and filtering sub-bands; (2) respectively carrying out target alignment on each sub-band; (3) performing common spatial mode feature extraction on each aligned sub-band, and splicing into a feature vector with a higher dimension; (4) selecting p most representative features through mRMR to form a final feature; (5) and inputting the extracted most representative features into a classifier for classification to obtain a result.

The respective steps are explained in detail one by one below.

Step (1): and (6) data acquisition. Selecting BCI Competition public data BCI Competition IV Dataset IIa: the data set consisted of electroencephalographic data from 9 healthy subjects. The data of each subject consists of electroencephalograms of four motor imagery of the left hand, the right hand, the feet and the tongue of one subject. Signals were recorded using 25 electrodes per experiment, with the electrode locations using the international 10/20 system. The 22 channel EEG signals and the 3 channel EOG signals were recorded and sampled at 250 Hz. This example selected only two types (left and right hand) of data for testing, 72 trials of each type. BCI Competition IV Dataset II b: the data set also consisted of electroencephalographic data for 9 healthy subjects. Its test only performed a left-right hand movement. Signals were recorded using 6 electrodes per experiment, using the international 10/20 system for electrode position. The 3 channel EEG signals and 3 channel EOG signals were recorded and sampled at 250 Hz.

And (3) subband filtering: the same subband filtering process is performed for all subjects in the data set. Each subject selects EEG motor imagery data of the subject to be extracted in a time period of 0.5-3.5s after visual cue, and then filtering is carried out on the data in a frequency range of 8-32Hz by using a 50-order FIR filter with 4Hz as a bandwidth to obtain 6 sub-bands, namely 8-12Hz, 12-16Hz, … and 28-32 Hz.

Step (2): and respectively carrying out target alignment on each sub-band. The method comprises the following steps: giving a N-channel space-time EEG signal matrix

Where T represents the number of samples per channel. First, a reference matrix for target alignment is calculated:

wherein n is_tIndicates the total number of trials of the target domain t,

for the ith test signal of the kth sub-band of the target domain,

the euclidean mean of all experiments for the kth subband of the target domain.

Then, the same calculation method as the formula (1) is used for obtaining the Euclidean mean value of all experiments of the kth subband of the source domain

Finally, the n of the kth sub-band of the source domain is processed by the formula (2)_sThe secondary trial performed target alignment.

Wherein the content of the first and second substances,

for the ith test signal of the kth sub-band in the source domain,

and (4) aligning signals of the ith test target for the kth subband of the source domain.

And (3): and performing CSP feature extraction on each aligned sub-band, and splicing into a high-dimensional feature vector. The method comprises the following steps: firstly, performing linear transformation on the aligned electroencephalogram signals by utilizing a CSP algorithm to realize spatial filtering:

wherein the content of the first and second substances,

is the CSP projection matrix for the k-th subband. Spatial filtering signal Z in formula (3)_k,iUse of

Thereby maximizing the variance difference of the two classes of band-pass filtered EEG. These two categories may include left and right handed motor imagery data, with the m pairs of CSP features for the kth subband, trial i, defined as:

wherein

Representing front and rear m columns

Diag (-) represents diagonal elements of the matrix, and tr (-) represents the sum of diagonal elements of the matrix. Thus, the SBTACSP characteristics of the ith trial can be expressed as:

v_i＝[v_1,i,v_2,i,...,v_6,i] (5)

wherein

Thus, the training data from the source domain may be characterized as

And (4): and selecting p characteristics with the most representativeness by mRMR to form a final characteristic. According to step (3)) Feature extraction, which can obtain the features of source domain (training) data

And a genuine label

Since the features of one trial are spliced from 6 subband features, this makes the feature dimension too large. In order to reduce the feature dimension, the difficulty of a learning task is reduced, and the efficiency of the model is improved. Using the mRMR feature selection algorithm, from each f_iThe most representative CSP features are selected out of the features. This can be achieved by maximizing the following expression:

wherein n is_f6 x 2m, I (-) is mutual information, r_iRepresenting a value that measures the ith characteristic mRMR. To r_iSorting is carried out, the features corresponding to p maximum r values are screened out, and finally a source domain (training) feature matrix is obtained

And (5): the obtained source domain feature matrix

And inputting the training model in the LDA classifier, testing the target domain to obtain a test result, and verifying the classification performance of the method in electroencephalogram signal feature extraction and cross-test classification.

To validate the effectiveness of the method of the present invention, evaluations will be made in two cross-tested scenarios, including single source to Single Target (STS) migration and multi-source to single target (MTS) migration. Meanwhile, the invention compares the performances of the BCI composition IV Dataset IIa and IIb with the MDRM, RA-MDRM, CSP-LDA, EA-CPS-LDA, FBCSP, SBEACSP and other methods, and the feasibility of the invention is measured by testing the classification accuracy.

According to the CSP method, increasing m does not significantly improve the classification accuracy. For dataset iia, the parameter m for CSP feature extraction was set to 3, so the number of preliminary features extracted per trial was 36(6 × 6). For data set IIb, only 3 EEG channels are used for this data set, and in equation (3)

The maximum choice of m is limited, so the parameter m is set to 1 and the number of preliminary features extracted per trial is 12(2 × 6).

Fig. 2(a) and 2(b) show the experimental results of 10-fold cross-validation of two datasets after SBTACSP selects different feature numbers p. As can be seen from fig. 2(a), when the number of features is 18, the cross-validation accuracy is the highest, and therefore, the p value is 18 the best for the data set iia. As can be seen from fig. 2(b), when the number of features is 12, the cross-validation result is optimal, and therefore, the p value for data set iib is 12 optimal.

Table 1 shows the optimal p-values obtained from fig. 2(a) and 2(b), the accuracy results of the 10-fold cross-validation of the proposed method with 3 methods on two data sets. As can be seen from Table 1, the average results of the SBTACSP method are superior to RA-MDRM, EA-CSP-LDA and FBCSP in both data sets IIa and IIb, and are also optimal in more than half of the subjects, with the results of

subjects

3, 5, 6, 7, 8 and 9 being more optimal in data set IIa. In data set IIb,

subjects

3, 4, 5, 6 and 8 performed better. SBTACSP is superior to FBCSP because the data are aligned before CSP extracts features, SBTACSP is superior to EA-CAP-LDA because sub-band processing is performed on the frequency band of 8-32Hz, and the latter three methods are superior to RA-MDRM because real labels are used in the test. These also indicate that the SBTACSP method is feasible and that the choice of parameter p is reasonable and efficient.

TABLE 1 results of the accuracy of 10-fold cross-validation of RA-MDRM, EA-CSP-LDA, FBCSP and SBTACSP on BCI compatibility IV data sets IIa and IIb

Table 2 and Table 3 show the classification correctness of BCI composition IV data sets IIa and IIb on STS and MTS migration, respectively. In general, SBTACSP is superior to the other five methods in classification accuracy. Specifically, for data set 2a, the present invention achieved the best average accuracy on subjects a01 and a06, best results on a03 for sbcsp, best results on a02 and a07 for FBCSP, and best results on a04, a05, a08, and a09 for RA-MDRM in STS migration; in MTS migration, the present invention achieved the best accuracy on subjects A01, A04, A06 and A07, whereas FBCSP performed the best on A02 and A09 and EA-CSP-LDA performed the best on A03 and A08. For data set 2B, the present invention performed optimally on subjects B01, B03, B04, B05, B08, and B09 during STS migration, with sbcsp achieving the best average accuracy outside of B02, B06, and B07. In MTS migration, SBTACSP gave the best accuracy on subjects B02, B04, B06 and B09, SBEACSP performed the best on B05, and FBCSP gave the best results on B03, B08. Taking MTS migration as an example, the classification accuracy of the invention in the subject A01 is 88.19%, which is better than EA-CSP-LDA (86.81%) and RA-MDRM (73.61%), and the method has the highest average accuracy (75%) in all nine tests, which is 7.29% higher than FBCSP, 2.55% higher than RA-MDRM, 1.39% higher than SBEACSP, and 1.23% higher than EA-CSP-LDA. Compared with STS migration and MTS migration, the accuracy of MTS migration is much higher than that of STS migration, which shows that the accuracy of multi-source domain migration can be improved properly.

In the cross-tested scene, the FBCSP does not achieve a good effect, and the effect of the FBCSP in the cross-stage classification is far better than that of the displayed cross-tested classification because the data of the training set and the test set of the cross-stage classification belong to the same person and are only acquired in different time, and the difference degree between the training data and the test data is far less than that of the cross-tested classification. Except for the STS migration condition of the data set 2b, the average classification accuracy of FBCSP is higher than CSP-LDA, which shows that the sub-band filtering achieves positive effect. And simultaneously comparing EA-CSP-LDA and SBTACSP, the classification accuracy of SBTACSP is at least 1.2% higher than that of EA-CSP-LDA, the maximum difference can reach 5.22%, and the subband filtering is necessary before alignment. Comparing FBCSP and SBTACSP, SBTACSP performs better, which indicates that TA can effectively resolve the problem of data differences between subjects. Meanwhile, the EA-CSP-LDA classification effect is superior to that of the FBCSP. These illustrate the great advantage of transfer learning across the subject classes, which has great prospects for development. Combining the above three points shows that the combination of subband filtering and target alignment is feasible. Finally, SBTACSP is preferred over sbeacp, which illustrates that the proposed variant of the invention is more suitable for combining with subband filtering.

TABLE 2 Classification accuracy (%)% of BCI Competition IV Datasetli A in STS and MTS migration

TABLE 3 Classification accuracy (%) of BCI Competition IV Dataset IIb in STS and MTS migration

Fig. 3(a) and (b) visualize the classification results using a non-linear dimensionality reduction technique t-SNE to compare the differences between features extracted using CSP and SBTACSP. FIG. 3(a) shows the t-SNE visualization of subjects A01 and A03 in dataset IIa during MTS migration, each row corresponding to a different test object, Xs1 and Xs2 representing two classes of features of the source domain, respectively, and Xt1 and Xt2 representing two classes of features of the target domain, respectively. The left panel shows features of direct CSP extraction of EEG data, and the right panel shows features of SBTACSP. The visualization of subjects B04 and B09 in data set IIb is shown in FIG. 3 (B). By contrast, the source domain (training) and target domain (testing) features of SBTACSP overlap, i.e., reduce the differences between them, while the distributions between the same classes in different domains overlap as much as possible. It can thus be seen that the visualization effect is consistent with the classification result. This also demonstrates the feasibility of the invention.

In summary, the invention provides a sub-band target alignment common space mode motor imagery electroencephalogram signal cross-subject classification method, more frequency band information is obtained through a sub-band filtering signal processing method, and meanwhile, the distribution difference of a source domain and a target domain is reduced by using an unsupervised target alignment method. The invention combines the advantages of sub-band filtering and domain alignment, and improves the accuracy of characteristics and cross-test classification.

Claims

1. The sub-band target alignment common space mode electroencephalogram signal cross-subject classification method is characterized by comprising the following steps: the method comprises the following main steps:

step (1): data acquisition, wherein BCI Competition open data is selected and comprises two data sets of BCI Competition IV Dataset IIa and BCI Competition IV Dataset IIb;

and (3) subband filtering: performing the same subband filtering process on subjects of all data sets; selecting EEG motor imagery data of each subject from 0.5-3.5s time period after visual cue, filtering the data with a 50-order FIR filter in a frequency range of 8-32Hz and with 4Hz as bandwidth to obtain 6 sub-bands, namely 8-12Hz, 12-16Hz, … and 28-32 Hz;

step (2): aligning a source domain sample into a target domain space, namely aligning a test sample into a training space by adopting a domain alignment method, namely target alignment;

the method comprises the following steps: giving a N-channel space-time EEG signal matrix

Where T represents the number of samples per channel, a target-aligned reference matrix is calculated:

wherein n is_tIndicates the total number of trials of the target domain t,

for the ith test signal of the kth sub-band of the target domain,

the Euclidean mean value of all tests of the kth subband of the target domain;

then, the same calculation mode as the formula (1) is used for obtaining the Euclidean mean value of all experiments of the kth subband of the source domain

Finally, the n of the kth sub-band of the source domain is processed by the formula (2)_sPerforming target alignment in the secondary test;

wherein the content of the first and second substances,

for the ith test signal of the kth sub-band in the source domain,

aligning signals of an ith test target of a kth sub-band of a source domain;

and (3): filtering the aligned electroencephalogram signals of each frequency band by using a CSP algorithm, and splicing CSP characteristics of a plurality of frequency bands together to form a characteristic vector with a higher dimension;

and (4): selecting p most representative features according to a minimum redundancy maximum correlation feature selection algorithm to form a final feature;

and (5): the obtained source domain feature matrix

Inputting the training model in the LDA classifier, and testing the target domain to obtain a test result.

2. The sub-band target-aligned co-spatial-mode electroencephalogram signal cross-subject classification method according to claim 1, characterized in that: in the step (1), BCI Competition IV Dataset IIa consists of electroencephalogram data of 9 healthy subjects; the data of each subject consists of electroencephalograms of four motor imagery of the left hand, the right hand, the foot and the tongue of one subject; each experimental signal is recorded by using 25 electrodes, and the position of each electrode adopts the international 10/20 system; 22 channels of EEG signals and 3 channels of EOG signals were recorded and sampled at 250Hz, while only the left and right hand data were selected for testing, 72 trials each.

3. The sub-band target-aligned co-spatial-mode electroencephalogram signal cross-subject classification method according to claim 1, characterized in that: BCI Competition IV Dataset IIb consists of electroencephalogram data of 9 healthy subjects; the test only executes left-right hand movement; recording signals of each experiment by using 6 electrodes, wherein the position of each electrode adopts an international 10/20 system; the 3 channel EEG signals and 3 channel EOG signals were recorded and sampled at 250 Hz.

4. The sub-band target-aligned co-spatial-mode electroencephalogram signal cross-subject classification method according to claim 1, characterized in that: the step (3) is specifically as follows: and (3) performing linear transformation on the aligned electroencephalogram signals by utilizing a CSP algorithm to realize spatial filtering:

wherein the content of the first and second substances,

CSP projection matrix of kth sub-band;

the m pairs of CSP features for the kth subband, trial i, are defined as:

wherein

Representing front and rear m columns

diag (-) represents the diagonal elements of the matrix, tr (-) represents the sum of the diagonal elements of the matrix;

the SBTACSP characteristics of the i-th trial are expressed as:

v_i＝[v_1,i,v_2,i,...,v_6,i] (5)

wherein

The training data from the source domain is characterized as

5. The sub-band target-aligned co-spatial-mode electroencephalogram signal cross-subject classification method according to claim 4, characterized in that: the step (4) is specifically as follows: obtaining the characteristics of the source domain data according to the characteristic extraction in the step (3)

And a genuine label

Using the mRMR feature selection algorithm, from each f_iThe most representative CSP features are selected from the features by maximizingThe expression is realized:

wherein n is_f6 x 2m, I (-) is mutual information, r_iA value representing the measurement of the ith feature mRMR;

to r_iSorting is carried out, features corresponding to p maximum r values are screened out, and finally a source domain feature matrix is obtained