CN117473406A - Electroencephalogram domain self-adaption method of multi-source flow shape measurement characteristics - Google Patents

Abstract

The invention relates to an electroencephalogram domain self-adaption method of multi-source flow shape measurement characteristics, which comprises the following steps: extracting brain electricity differential entropy characteristics according to frequency bands; step two: performing similarity measurement on the plurality of source domain data, and selecting a high-quality source domain; step three: the electroencephalogram signal characteristics are transformed into manifold space, manifold characteristics are further extracted, and good geometric characteristics are maintained; step four: learning a Markov metric, minimizing class inner distance, maximizing class distance in manifold feature space, and restricting source domain and target domain to similar distribution under the Markov metric to finally obtain a feature matrix so as to train a classifier and apply the classifier to classification; step five: and carrying out weighted fusion on the multi-identification result of the target domain according to the classifier result to obtain a final classification result. Compared with the traditional method, the method can learn more accurate measurement matrix between brain electrical data fields, improves the quality of the feature mapping matrix, and improves the generalization performance and accuracy of the classifier model.

Description

Electroencephalogram domain self-adaption method of multi-source flow shape measurement characteristics

Technical Field

The invention relates to an electroencephalogram domain self-adaption method for multi-source flow shape measurement characteristics of an electroencephalogram, which relates to preprocessing of the electroencephalogram, manifold characteristic extraction, manifold characteristic measurement learning and multi-source domain migration frame design, and belongs to the technical field of nervous system motion control.

Background

In recent years, emotion recognition has been studied in the fields of emotion computing (affective computing, AC) and emotion brain-computer interface (apci). The good emotion recognition method can bring convenience to human-computer interaction and has wide application prospect in the fields of medicine, education, traffic, military and the like. Emotion recognition methods are generally classified into subjective recognition methods based on facial expressions or intonation and objective recognition methods based on physiological signals. Wherein an electroencephalogram (EEG) signal can directly record electrophysiological activity of the central nervous system under external stimulus and cannot be expressed by camouflage. The most widely used input signal in the aBCI is due to the advantages of EEG signals. A typical aBCI paradigm operates as follows: firstly, presenting emotion stimulus inducing specific emotion to a user, and recording an electroencephalogram signal according to expected emotion; EEG data features are then extracted from the recorded signals and a classifier is trained using the selected features and emotion tags; next, real-time emotion classification based on the electroencephalogram signal is performed using a classifier that has been trained. Many researchers have reported satisfactory classification performance achieved under this paradigm.

However, the use of aBCI is still limited by several factors. In particular, the brain electrical signals have great non-stationarity and individual variability, and the brain electrical data distribution of different subjects has great differences. Even in the same subject, the distribution of the electroencephalogram data at different periods often differs. Secondly, the decoding of EEG signals is critical to achieving accurate emotion recognition, and emotion feature extraction is a technical challenge therein. Most studies are based on the assumption that the data in the training set and the test set are independently co-distributed. Obviously, the EEG data distribution is different between different subjects. In order to improve the adaptability and accuracy of the tested person and shorten the training time of the tested person, an electroencephalogram signal analysis model with strong self-adaptation capability and high emotion recognition accuracy is designed and realized, a plurality of research teams are actively researching the migration learning theory and method, trying to find a general algorithm model suitable for all the tested person and solving the key scientific problems commonly existing in the brain-computer interface system in practical application.

Transfer learning (transfer learning, TL) is a machine learning technique that aims to extract common knowledge from one or more source tasks and apply that knowledge to the relevant target tasks. Specifically, the transfer learning in emotion recognition uses a source domain (electroencephalogram data from other users) to assist a target domain (electroencephalogram data from new users) in learning. One of its main tasks is to reduce the data distribution difference between the source domain and the target domain by mapping.

The use of domain adaptation techniques in transfer learning to address differences in EEG data distribution between different subjects is feasible and has become a new direction of research for researchers in recent years. Zheng Weilong et al use differential entropy features based on domain adaptation techniques to find stable EEG patterns between different subjects. Their work was also the first time to apply the early-emerging migration component analysis (transfer components analysis, TCA) method to EEG emotion recognition, where TCA is used to project the data of the source and target domains into a new subspace to reduce the distribution variance. Experimental results demonstrate that TL is superior to support vector machines (support vector machine, SVM) in the task of emotion recognition across subjects. A learner developed a style conversion mapping (style transfer mapping, STM) framework for emotion recognition across subjects. It maps the target domain distribution to the source domain by learning some mapping parameters. On the reference SEED dataset, classification accuracy was improved by 12.72% compared to the non-migration method.

Previous studies have effectively applied TL technology to the field of apci and achieved good results in recognition of emotion across subjects. However, these strategies still have some specific limitations. First, the mobility of TL is largely affected by the similarity between different domains. Previous TL models typically integrate all subjects into one domain, ignoring those that are poorly related to the target domain, resulting in negative migration. Because in practical applications not all source data is suitable for knowledge migration. Second, previous TL methods typically use a fixed distance to measure the distribution distance between two domains, which is very coarse, does not achieve an accurate similarity measure, and results in the feature mapping matrix often being affected. Metric Learning (ML) can provide a solution to this problem. Metric learning treats distance as a learnable target. Each dataset has specific problems in terms of classification and clustering, and distance metrics without good learning ability cannot build a good data classification model. Therefore, measurement learning and migration learning can be combined, and in multi-source migration learning, a selection method of a source domain is introduced, so that the brain electricity emotion recognition effect crossing the tested brain electricity emotion is improved.

Disclosure of Invention

In order to solve the problems of inaccurate measurement of the characteristics of negative migration and extracted brain electricity in the existing brain electricity migration learning method, the invention provides a multi-source flow shape measurement characteristic domain self-adaptive method facing brain electricity signals, which can learn accurate measurement matrixes among brain electricity data domains, improve the quality of characteristic mapping matrixes, reduce the sample distribution difference of a source domain and a target domain and perform multi-source brain electricity measurement migration learning.

An electroencephalogram domain adaptive method of multi-source flow shape metric features, comprising the steps of:

step one: acquiring and preprocessing an electroencephalogram signal;

step two: performing similarity measurement on the plurality of source domain data, and selecting a high-quality source domain;

step three: the electroencephalogram signal characteristics are transformed into manifold space, manifold characteristics are extracted, and good geometric characteristics are maintained;

step four: learning a Markov metric in a manifold feature space, minimizing class inner distance, maximizing class distance, and constraining a source domain and a target domain to be similarly distributed under the Markov metric to finally obtain a feature matrix so as to train a classifier and apply the classifier to classification;

step five: and carrying out weighted fusion on the multi-identification result of the target domain according to the classifier result to obtain a final classification result.

The pretreatment method in the first step comprises the following steps: and extracting the electroencephalogram differential entropy characteristics according to the frequency bands.

The core of the second step is to find acceptable similarity between two domains, if there is no similarity or the similarity is not significant between the two domains, the source domain is removed, and the selecting step in the second step is as follows:

step 2.1: for each target domain, calculating the similarity between the target domain and the source domain by using a formula (1);

wherein the sample pair x _i ，x _j From the target domain and the source domain, respectively, d _M Is the similarity metric distance, C is the covariance matrix;

step 2.2: and obtaining similarity sorting according to the similarity measurement result of each source domain and each target domain, wherein the similarity measurement value is smaller for more similar domains, and selecting different proper number of source domains according to the similarity sorting for different data sets.

The fourth step specifically comprises:

step 4.1: in metric learning, the distance metric is no longer limited to the inverse covariance matrix of the type in equation (1), but needs to be obtained through the process of metric learning, which is defined as:

wherein m=a ^T A, a semi-positive definite matrix;

step 4.2: according to formula (2), constraint of conditional probability distribution dist of source domain and target domain respectively _M (Q _S (Y _S |X _S ),Q _T (Y _T |X _T ) And edge probability distribution dist) _M (P(X _S ),P(X _T ) Defined as:

wherein, is F norm, x ^(c) Representing samples belonging to class c in the domain; l (L) ₀ And L _c The measurement adaptive matrix is respectively a conditional probability distribution and an edge probability distribution;

wherein:

step 4.3: according to equations (3) and (4), the distribution constraints of the source domain and the target domain under metric learning can be written as:

dist _M (D _s ,D _t )＝(1-λ)·dist _M (P(X _S ),P(X _T ))+λ·dist _M (Q _S (Y _S |X _S ),Q _T (Y _T |X _T )) (7)

wherein λ is a dynamic factor, measuring the importance of both distributions;

wherein d ₀ Is the edge under the metric matrix MDistance d of edge distribution _c Is the conditional distribution distance under the metric matrix M;

equation (7) can be rewritten as:

dist _M (D _s ,D _t )＝tr(X((1-λ)L ₀ +λL _c )X ^T M) (9)

step 4.4: by introducing a Laplace regularization term, the objective function can be written as:

where γ is the hyper-parameter, ρ is the Laplacian regularization parameter, and G is the manifold feature transformation matrix.

Compared with the prior art, the invention has the beneficial effects that:

the mobility of the mobility learning is greatly affected by the similarity between different domains. Previous migration learning models typically integrate all subjects into one domain, ignoring those subjects that have poor similarity to the target domain, resulting in negative migration. Because in practical applications not all source data is suitable for knowledge migration.

Previous migration learning methods typically use a fixed distance to measure the distribution distance between two domains, which is very coarse, does not achieve an accurate similarity measure, and results in feature mapping matrices being often impacted, resulting in reduced classifier performance.

In order to solve the problems, the invention provides a novel electroencephalogram domain self-adaption method based on multi-source flow shape measurement characteristics. Compared with the common non-supervision selection method, the method is simple and visual, is more rapid and effective, can learn the reliable measurement among the features, improves the quality of the source domain, and simultaneously further improves the quality of the electroencephalogram feature mapping, and is an effective multi-source measurement transfer learning framework.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a method flow diagram of an electroencephalogram domain adaptation method of the multi-source flow metric feature of the present invention;

FIG. 2 is a diagram of an algorithm framework of the present invention;

FIG. 3 is a graph showing the effect (1) of the number of source domains on the emotion recognition accuracy curve of a subject;

FIG. 4 is a graph showing the effect (2) of the number of source domains on the emotion recognition accuracy curve of a subject according to the present invention;

FIG. 5 is a graph comparing recognition accuracy of the method of the present invention with various field adaptation methods on a SEED dataset;

FIG. 6 shows the recognition accuracy of the method of the present invention on a DEAP dataset compared to various field-adaptive methods.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Example 1

The data sets used herein are SEED data sets and DEAP data sets, respectively. The specific description is as follows:

(1) SEED is an open source emotion EEG dataset published by the university of Shanghai transportation. In particular, the SEED dataset recorded EEG signals generated by 15 subjects when watching 15 movie fragments. EEG signals were collected three times for each subject, one week apart. Each session for each subject contained 3394 samples, with emotional tags being classified as positive, negative, and neutral. EEG signals were placed and recorded by a 62-channel electrode cap in accordance with the International 10-20 System. The original EEG signal is recorded at a sampling rate of 1000Hz, then downsampled to 200Hz, and filtered with a bandpass filter of 0-75 Hz.

(2) DEAP is another public data set collected and published by Kolestra et al in the field of artificial emotion computing. It uses a 40-channel electrode cap to record multi-modal physiological signals of 32 subjects while watching selected mood-arousal videos, with 32 channels being EEG signals and 8 channels being peripheral physiological signals. After viewing the video, each subject was asked to perform a self-assessment (SAM) to quantify mood, including pleasure, arousal, dominance and preference, with a continuous scale of 1 to 9. The raw EEG signal was recorded at a sampling rate of 512Hz, downsampled to 128Hz, filtered with a 4-45 Hz bandpass filter, and then split into 60 seconds of test data and 3 seconds of baseline data.

As shown in fig. 1 and 2, the implementation steps of the embodiment of the present invention are as follows:

step one: acquiring and preprocessing an electroencephalogram signal, and extracting electroencephalogram differential entropy characteristics according to frequency bands;

for a sequence of EEG signals x over a period of time, its differential entropy characteristics are defined as:

h(x)＝-∫f(x)log[f(x)]dx (11)

where f (x) is a probability density function of the EEG signal, and after bandpass filtering, the time series of EEG signals obey a Gaussian distribution N (μ, σ) ² ) Thus, equation (11) can be written as:

according to formula (12), EEG differential entropy characteristics of each sample can be calculated.

Step two: according to the EEG differential entropy characteristics obtained in the first step, selecting a source domain for each target domain, and selecting a source domain with higher similarity, wherein the method comprises the following specific steps:

the core of the second step is to find acceptable similarity between two domains, if there is no similarity or the similarity is not significant between the two domains, the source domain is removed, and the selecting step in the second step is as follows:

step 2.1: for each target domain, the similarity between the target domain and the source domain is calculated by using the formula (1), and for more similar domains, the value of the similarity measure is smaller

Wherein the sample pair x _i ，x _j From the target domain and the source domain, respectively, d _M Is the similarity metric distance, C is the covariance matrix;

step 2.2: and obtaining similarity sorting according to the similarity measurement result of each source domain and each target domain, wherein the similarity measurement value is smaller for more similar domains, and selecting different proper number of source domains according to the similarity sorting for different data sets.

Step three: the electroencephalogram signal characteristics are transformed into manifold space, manifold characteristics are extracted, and good geometric characteristics are maintained;

step 3.1: a geodesic flow kernel method (Geodesic Flow Kernel, GFK) is introduced to transform the original EEG differential entropy features into a Grassmann manifold G, completing the manifold feature transformation. The definition of the geodesic flow core GFK is as follows:

wherein x is _i And x _j For the d-dimensional feature vector,<z _i ,z _i >representing transformed feature z _i And z _j Inner product of the two, semi-positive definite matrix G E R ^d×d The singular value decomposition may be used for the calculation.

Step 3.2: according to the solved semi-positive definite matrix G, the transformed characteristic z can be calculated by the formulaAnd (5) calculating to obtain EEG manifold characteristics.

Step four: learning a Markov metric in a manifold feature space, minimizing class inner distance, maximizing class distance, and constraining a source domain and a target domain to be similarly distributed under the Markov metric to finally obtain a feature matrix so as to train a classifier and apply the classifier to classification;

the method specifically comprises the following steps:

step 4.1: in metric learning, the distance metric is no longer limited to the inverse covariance matrix of the type in equation (1), but needs to be obtained through the process of metric learning, which is defined as:

wherein m=a ^T A, a semi-positive definite matrix;

step 4.2: according to formula (2), constraint of conditional probability distribution dist of source domain and target domain respectively _M (Q _S (Y _S |X _S ),Q _T (Y _T |X _T ) And edge probability distribution dist) _M (P(X _S ),P(X _T ) Defined as:

wherein, is F norm, x ^(c) Representing samples belonging to class c in the domain; l (L) ₀ And L _c Metrics of conditional probability distribution and edge probability distribution, respectivelyAn adaptive matrix;

wherein:

step 4.3: according to equations (3) and (4), the distribution constraints of the source domain and the target domain under metric learning can be written as:

dist _M (D _s ,D _t )＝(1-λ)·dist _M (P(X _S ),P(X _T ))+λ·dist _M (Q _S (Y _S |X _S ),Q _T (Y _T |X _T )) (7)

wherein λ is a dynamic factor, measuring the importance of both distributions;

wherein d ₀ Is the edge distribution distance under the metric matrix M, d _c Is the conditional distribution distance under the metric matrix M;

equation (7) can be rewritten as:

dist _M (D _s ,D _t )＝tr(X((1-λ)L ₀ +λL _c )X ^T M) (9)

step 4.4: by introducing a Laplace regularization term, the objective function can be written as:

where γ is the hyper-parameter, ρ is the Laplacian regularization parameter, and G is the manifold feature transformation matrix.

Step five: according to the previous step, obtaining a feature matrix, sending the feature matrix into a classifier for training and classifying, obtaining the prediction label and the accuracy of each group of source domains to the target domains, and carrying out weighted fusion according to the recognition results of a plurality of groups of classifiers to obtain the final classification label and the recognition accuracy, wherein the method comprises the following specific steps:

after obtaining the recognition accuracy of multiple groups of target domain data for multiple groups of source domain data, taking the recognition accuracy of each classifier as a weight, and carrying out weighted fusion on a final prediction label, wherein the weighted fusion calculation mode of each sample is as shown in a formula (14):

where n is the sequence number of the source domain, w is the weight,the pre-result of the sample representing n source domains is the sum of weights of class c, and the class label corresponding to the largest result of the sum of weights is the final identification label result.

According to the invention, 15 subjects are taken as experimental objects on the SEED data set and the DEAP data set, 1 subject is selected as a target domain, the rest 14 subjects are taken as source domains, and the experiment is carried out, so that the whole process is repeated 15 times, and each subject is ensured to be taken as the target domain. The experimental results are shown in fig. 3, 4, 5 and 6. Fig. 3 shows a change curve of recognition accuracy of each subject on the SEED data set along with the increase of the number of source domains, and can see that when the number of source domains increases from 1 to 9, the accuracy starts to increase, and when the number of source domains increases from 9 to 14, the accuracy curve starts to stabilize, even a slide down occurs, which indicates that some source domains with poor quality exist, and negative migration is caused, so that the number of source domains is 9, and bad source domains can be effectively removed. Similarly, fig. 4 shows a plot of recognition accuracy per subject for a DEAP dataset as a function of the increase in the number of source domains, with a final number of 8 suitable source domains selected. Fig. 5 and fig. 6 show comparison of the recognition accuracy of the method of the present invention and the adaptive methods of various fields for each subject on SEED and DEAP, and it can be seen that the method obtains the best recognition accuracy on most subjects, which illustrates the effectiveness of the adaptive methods of multi-source manifold brain electrical characteristics field proposed by the present invention.

The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. It will be apparent to those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, and yet fall within the scope of the invention.

Claims (5)

1. An electroencephalogram domain self-adaption method of multi-source flow shape measurement features is characterized in that: the method comprises the following steps:

step one: acquiring and preprocessing an electroencephalogram signal;

step two: performing similarity measurement on the plurality of source domain data, and selecting a high-quality source domain;

step three: the electroencephalogram signal characteristics are transformed into manifold space, manifold characteristics are extracted, and good geometric characteristics are maintained;

step four: learning a Markov metric in a manifold feature space, minimizing class inner distance, maximizing class distance, and constraining a source domain and a target domain to be similarly distributed under the Markov metric to finally obtain a feature matrix so as to train a classifier and apply the classifier to classification;

step five: and carrying out weighted fusion on the multi-identification result of the target domain according to the classifier result to obtain a final classification result.

2. The electroencephalogram domain adaptive method of multi-source flow metric features of claim 1, wherein: the pretreatment method in the first step comprises the following steps: and extracting the electroencephalogram differential entropy characteristics according to the frequency bands.

3. The electroencephalogram domain adaptive method of multi-source flow metric features of claim 1, wherein: the selection step in the second step is as follows:

step 2.1: for each target domain, calculating the similarity between the target domain and the source domain by using a formula (1);

wherein the sample pair x _i ，x _j From the target domain and the source domain, respectively, d _M Is the similarity metric distance, C is the covariance matrix;

step 2.2: and obtaining similarity sorting according to the similarity measurement result of each source domain and each target domain, wherein the similarity measurement value is smaller for more similar domains, and selecting different proper number of source domains according to the similarity sorting for different data sets.

4. The electroencephalogram domain adaptive method of multi-source flow metric features of claim 1, wherein: the fourth step specifically comprises:

step 4-1: in metric learning, a similarity metric needs to be obtained through the process of metric learning, which is defined as:

wherein m=a ^T A, a semi-positive definite matrix;

step 4-2: according to formula (2), constraint of conditional probability distribution dist of source domain and target domain respectively _M (Q _S (Y _S |X _S ),Q _T (Y _T |X _T ) And edge probability distribution dist) _M (P(X _S ),P(X _T ) Defined as:

wherein, is F norm, x ^(c) Representing samples belonging to class c in the domain; l (L) ₀ And L _c The measurement adaptive matrix is respectively a conditional probability distribution and an edge probability distribution;

wherein:

step 4.3: according to equations (3) and (4), the distribution constraints of the source domain and the target domain under metric learning can be written as:

dist _M (D _s ,D _t )＝(1-λ)·dist _M (P(X _S ),P(X _T ))+λ·dist _M (Q _S (Y _S |X _S ),Q _T (Y _T |X _T )) (7)

wherein λ is a dynamic factor, measuring the importance of both distributions;

wherein d ₀ Is the edge distribution distance under the metric matrix M, d _c Is the conditional distribution distance under the metric matrix M;

equation (7) can be rewritten as:

dist _M (D _s ,D _t )＝tr(X((1-λ)L ₀ +λL _c )X ^T M) (9)

step 4.4: by introducing a Laplace regularization term, the objective function can be written as:

where γ is the hyper-parameter, ρ is the Laplacian regularization parameter, and G is the manifold feature transformation matrix.

5. The electroencephalogram domain adaptation method of a multisource shape metric feature of claim 2, wherein: the first step specifically comprises the following steps: acquiring and preprocessing an electroencephalogram signal, and extracting electroencephalogram differential entropy characteristics according to frequency bands;

for a sequence of EEG signals x, its differential entropy characteristics are defined as:

h(x)＝-∫f(x)log[f(x)]dx (11)

where f (x) is a probability density function of the EEG signal, and after bandpass filtering, the time series of EEG signals obey a Gaussian distribution N (μ, σ) ² ) Thus, equation (11) can be written as:

according to formula (12), EEG differential entropy characteristics of each sample can be calculated.