CN113392733A - Multi-source domain self-adaptive cross-tested EEG cognitive state evaluation method based on label alignment - Google Patents

Multi-source domain self-adaptive cross-tested EEG cognitive state evaluation method based on label alignment Download PDF

Info

Publication number
CN113392733A
CN113392733A CN202110601409.9A CN202110601409A CN113392733A CN 113392733 A CN113392733 A CN 113392733A CN 202110601409 A CN202110601409 A CN 202110601409A CN 113392733 A CN113392733 A CN 113392733A
Authority
CN
China
Prior art keywords
domain
source domain
eeg
label
cognitive state
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110601409.9A
Other languages
Chinese (zh)
Other versions
CN113392733B (en
Inventor
方欣
戴国骏
赵月
李秀峰
张振炎
吴政轩
吴靖
曾虹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Dianzi University
Original Assignee
Hangzhou Dianzi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Dianzi University filed Critical Hangzhou Dianzi University
Priority to CN202110601409.9A priority Critical patent/CN113392733B/en
Publication of CN113392733A publication Critical patent/CN113392733A/en
Application granted granted Critical
Publication of CN113392733B publication Critical patent/CN113392733B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/12—Classification; Matching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/08—Feature extraction

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Signal Processing (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)

Abstract

The invention discloses a label alignment-based multi-source domain self-adaptive cross-tested EEG cognitive state evaluation method. The invention comprises the following steps: 1: acquiring data; 2: preprocessing data; 3: a cross-tested EEG cognitive state assessment method based on an LA-MSDA model. The method adopts a shared common feature extractor and an unshared sub-feature extractor which are used in stages to further learn the tested invariant features and specific features of the source domain sample and the target domain sample; secondly, considering the relation and similarity among the cross-testees, a method for aligning the inter-domain distribution of local and global representations is provided to evaluate the cross-testee cognitive state, and the problems that the class condition information with fine granularity is difficult to learn and the decision boundary samples which are suitable for the cross-testees are solved. Finally, the method effectively avoids the problem of individual difference of electroencephalogram signals in the field of brain cognitive computation, is suitable for cognitive state recognition based on EEG under any task, has strong generalization capability, and can be well suitable for clinical diagnosis and practical application.

Description

Multi-source domain self-adaptive cross-tested EEG cognitive state evaluation method based on label alignment
Technical Field
The invention relates to a neuroelectrophysiological signal analysis technology in the field of brain cognitive computation and a multi-source domain adaptation model construction method in the field of unsupervised learning, in particular to a method for analyzing electroencephalogram (EEG) to evaluate cognitive states based on label alignment, and can effectively solve the problems of remarkable difference and low signal-to-noise ratio of different tested individuals.
Background
EEG-based cognitive state analysis methods have received increasing attention in recent years due to the characteristics of non-invasiveness, portability and low cost, as well as the advantages of machine learning or deep learning in extracting and classifying features from large amounts of data. Existing EEG-based analysis typically performs classification tasks combining appropriate feature extraction with classifiers, where: common methods for feature extraction include Common Space Pattern (CSP), Discrete Wavelet Transform (DWT), etc., and common classifiers include SVM, Bayes, some deep learning models, etc.
Although the signal analysis of these methods has a high discriminant performance, there are some drawbacks in the EEG-based cross-domain predictive analysis: in the real context of EEG analysis, there are significant differences in EEG between subjects, mainly due to physical (e.g., environmental and skin electrode impedance) and biological (e.g., differences in gender, age, and brain activity patterns) factors, and furthermore, EEG changes over time despite the same subject. Therefore, a general cross-subject electroencephalogram analysis model needs to be constructed.
At present, the rich results based on deep learning benefit from the supervised learning of large amounts of labeled data. However, for unsupervised learning, the main obstacle in designing a general network model is to extend the model trained from known label data to a new label-free domain. Aiming at the target task lacking the tag data, the key breakthrough point for solving the problem is to fully utilize the characteristic information of the source domain tagged data. However, the domain shift problem of the trained model when applied to a new domain often causes the performance of the model to be greatly reduced.
An Unsupervised Domain Adaptation (UDA) algorithm in migration learning is an algorithm for solving the distribution bias of a source domain and a target domain, and has been proved to be capable of effectively reducing the distribution gap between each domain by learning domain-invariant features. However, because of the high nonlinearity and significant individual difference of brain electricity, it is difficult to extract the same or similar features between different subjects, so the existing UDA method has the following two limitations: (1) the problem of feature confusion near decision boundaries cannot be completely solved, the target function is difficult to achieve the optimum, and the target function may fall into a local optimum state; (2) feature-based alignment is difficult to achieve to extract feature-based domain-invariant features. Feature-based alignment is a technology for minimizing feature distribution differences, can effectively optimize the domain offset problem, and can effectively reduce the problem of individual differences by mapping the relation between the tested objects to a space conforming to a certain structure from the perspective of EEG signal characteristics and aligning the space.
In addition, relevant documents show that, different from the single-source domain adaptation problem, the multi-source domain transfer learning simultaneously transfers knowledge of a plurality of source domains to the target domain to assist the learning of the target domain, the data are different from the target domain and different from each other, and the method can effectively solve the problem of individual difference of electroencephalogram signals.
In conclusion, the invention takes the cross-test, domain adaptation, multi-source domain and feature-based alignment as key starting points to construct a label-alignment-based multi-source domain adaptive cross-test model, so as to be more efficiently applied in the field of neuro-physiological signal analysis.
Disclosure of Invention
The invention provides a label alignment-based multi-source domain self-adaptive model, fully learns the structural characteristics of a label sample aiming at the conditions of high nonlinearity and obvious individual difference existing in EEG data, adopts a local label alignment strategy, extracts the label-based domain invariant characteristics, and aligns the characteristic distribution of a target domain and the multi-source domain to realize the effective transfer of the inter-domain characteristics. The proposed algorithm has mainly the following two aspects: on one hand, the alignment strategy based on the local labels is adopted, because the labels contained in different tested EEG data are the same in type when the data are collected, the domain invariant features of the corresponding labels are extracted through local label alignment, and the influence of the EEG individual difference on the model performance can be effectively avoided. On the other hand, an improved global optimization UDA method is provided, weight constraints are set according to probability distribution results of all classifiers, a global objective function optimization strategy is adopted, the problem of fuzzy decision boundary of the classifiers is solved, and the generalization capability of a model in cross-tested EEG analysis is improved. The algorithm can effectively avoid the influence of high nonlinearity of the EEG and obvious individual difference characteristics, can achieve better performance in the cognitive state detection based on the EEG, and has wide application scenes in practice.
In summary, the cognitive state identification based on the EEG is performed by the invention, and the cognitive state to be tested is evaluated by taking nonlinearity and individual difference as key starting points, and the core technology of the invention is mainly to construct a label alignment-based multi-source domain adaptive cross-tested model (LA-MSDA). According to the method, each training individual is used as a single domain to form a plurality of source domains, a new cross-tested individual is used as a target domain, cross-tested relevance and similarity are comprehensively considered, a local alignment and global optimization strategy is introduced to evaluate a cross-tested cognitive state, the domain invariant features based on the labels can be effectively extracted, and unobvious features near a decision boundary are correctly predicted. The method fully considers the characteristic space distribution structure of the label-free data, can greatly improve the model training efficiency, has higher universality, has wide application prospect in the actual brain-computer interaction (BCI), and provides technical support for clinical application.
Existing techniques for EEG-based cognitive state assessment focus mainly on aligning global feature distributions to minimize the difference between each subject, or combining all active classifiers to make a final decision, it is still difficult to extract domain-invariant features, and it is difficult to solve the ambiguity problem near the classifier decision boundary. In order to overcome the defects of the prior method, the invention adopts the following technical scheme:
the cognitive state evaluation method based on the EEG comprehensively considers the relevance and the similarity of the cross-testees, classifies the cognitive states of the testees by carrying out feature analysis on the EEG, realizes the distinction of different states under various experimental tasks, such as the cognitive states (addiction and normal control group) of a network game addiction patient, the cognitive states (waking and fatigue) of a driver driving task, the cognitive states (pleasant mood and passive mood) of the experimental testees under different mood stimulation tasks and the like, and can process different EEG data sets.
The invention is based on fatigue driving electroencephalogram data set, and is concretely realized as follows:
step 1: data acquisition
The fatigue driving electroencephalogram data set adopted by the invention is 15 electroencephalogram EEG data of healthy tested subjects with better driving experience, and each tested subject fills in an NASA-TLX questionnaire after the test so as to provide subjective workload perception. According to the NASA-TLX questionnaire, the present invention selected two mental states, TAV3 and DROWS, as analyses.
Step 2: data pre-processing
In order to further filter noise and remove artifacts, the EEG signal processing method aims at original EEG data, firstly, a band-pass filter (1-30Hz) is used for eliminating signals such as high-frequency noise, power frequency interference and the like except for spontaneous EEG signals, then, an Independent Component Analysis (ICA) method is used for processing, and finally, EEG signal features are extracted by using Power Spectral Density (PSD) so as to provide stable signal features for subsequent model construction.
And step 3: cross-tested EEG cognitive state evaluation method based on LA-MSDA model
Inputting: firstly, multi-source domain sample data with cognitive state label
Figure BDA0003093122610000041
Where N is the total number of source domains (i.e., the total number of subjects), N is the nth source domain,
Figure BDA0003093122610000042
a sample representing the nth source domain,
Figure BDA0003093122610000043
representing the true label corresponding to the nth source domain sample.
Second, target domain sample data without cognitive state label
Figure BDA0003093122610000044
Wherein
Figure BDA0003093122610000045
The ith sample, | U, representing the target domainTAnd | is the total number of target domain samples.
And the maximum iteration number T.
3-1. extracting potential common domain invariant features from the source domain and the target domain by a common EEGNet-based feature extractor C-EEGNet f (-) and mapping the extracted common domain invariant features to a common feature space.
3-2, passing through N subnets S-CNNs F not sharing weightn(ii) mapping each pair of source and target domains to a particular feature space, extracting particular features within the source and target domains, respectively
Figure BDA0003093122610000046
And
Figure BDA0003093122610000047
3-3. for each S-CNNs, a specific classifier G is trainednAnd adding a classification penalty to each classifier that passes through the domain from multiple sourcesLearn the ideal values of the weights and biases and try to find a way to minimize the loss. The losses are as follows:
Figure BDA0003093122610000048
wherein J is a cross entropy loss function, | XsnL is the total number of samples of the nth source domain,
Figure BDA0003093122610000049
is the true label of the ith sample of the nth source domain.
3-4, adding weight constraint for each sample through a local category maximum mean difference method (LLMMD), and adjusting the distribution of local subcategories on the assumption that the active domain XSAnd a target domain XTUsing xiRepresenting a source domain XSOf (a), n samples in total, xjRepresenting a target domain XTThe j-th sample in (b), m samples in total, the MMD is defined as follows:
Figure BDA0003093122610000051
where sup denotes the supremum of the supremum,
Figure BDA0003093122610000052
representing a feature mapping function that maps a domain-specific feature distribution to a Regenerated Kernel Hilbert Space (RKHS), H being the regenerated kernel Hilbert space, p and q representing the domain XSAnd XTDistribution of (c), k (x)i,xj) Is a gaussian kernel function.
3-5. align the specific feature distributions of each domain. Suppose alphacFor the probability that a sample belongs to the class label c, LLMMD is expressed as:
Figure BDA0003093122610000053
wherein C represents the total of class labelsThe number of the first and second groups is,
Figure BDA0003093122610000054
the ith sample representing the nth source domain
Figure BDA0003093122610000055
A local class label weight assigned to class label c,
Figure BDA0003093122610000056
i-th sample representing a target domain
Figure BDA0003093122610000057
A local class label weight assigned to the class label c.
3-6. align the predicted distributions of the target samples output by each classifier, using the representative outputs from the different classifiers to calculate the loss of variance as follows:
Figure BDA0003093122610000058
and 3-7, setting similarity weighting weight constraint according to the prediction probability distribution result of each classifier, wherein the smaller the difference between the two classifiers is, the higher the weight is. Weight of mth classifier
Figure BDA0003093122610000059
Is defined as:
Figure BDA0003093122610000061
wherein
Figure BDA0003093122610000062
Representing the difference loss weight between the nth and mth classifiers.
All classifiers are integrated based on weight constraints, and the resulting global penalty is as follows:
Figure BDA0003093122610000063
3-8. calculating the total objective optimization function, defined as follows:
Ltotal＝Lc+βLlocal+γLglobal#(7)
where β and γ are the corresponding weights.
3-9, repeating the steps 3-1 to 3-8 until the iteration is performed for T times.
And (3) outputting: the target domain samples correspond to class labels of the integrated results of all classifiers.
The main contributions of the invention are:
firstly, a method of using a shared common feature extractor and an unshared sub-feature extractor in stages is adopted to further learn the tested invariant features and specific features of the source domain samples and the target domain samples, and the defect that the traditional method only uses the domain invariant features is optimized. Secondly, considering the relation and similarity among the cross-testees, a method for aligning the inter-domain distribution of local and global representations is provided to evaluate the cross-testee cognitive state, and the problems that the class condition information with fine granularity is difficult to learn and the decision boundary samples which are suitable for the cross-testees are solved. Finally, the method effectively avoids the problem of individual difference of electroencephalogram signals in the field of brain cognitive computation, is suitable for cognitive state recognition based on EEG under any task, has strong generalization capability, and can be well suitable for clinical diagnosis and practical application.
Drawings
FIG. 1 is a diagram of a model architecture of the present invention.
Detailed Description
The invention is further illustrated with reference to the following figures and examples.
As shown in fig. 1, the method is a structural diagram of a multi-source domain adaptive cross-tested EEG cognitive state assessment method based on label alignment, and mainly includes the following steps:
step 1: data acquisition
The fatigue driving electroencephalogram data set adopted by the invention is 15 electroencephalogram EEG data of healthy tested subjects with better driving experience, and each tested subject fills in an NASA-TLX questionnaire after the test so as to provide subjective workload perception. According to the NASA-TLX questionnaire, the present invention selected two mental states, TAV3 and DROWS, as analyses.
Step 2: data pre-processing
Taking fatigue driving EEG data as an example, the raw EEG data processing steps are as follows:
2-1, artifact removal: performing artifact removal operation on the acquired original EEG data, firstly performing 1-30Hz band-pass filtering processing, and simultaneously removing power frequency interference and direct current components in the signals; and then removing artifacts in the signal by ICA independent component analysis.
2-2, PSD feature extraction: extracting PSD (phase-sensitive Detector) characteristics of the EEG with the artifacts removed, performing data segmentation on the EEG of each tested individual through a sliding window of 0.5s to obtain 1400 samples, wherein the number of EEG data acquisition channels is 61, extracting a frequency band 4-30Hz (wherein theta is 4-7Hz, alpha is 8-13Hz, and beta is 14-30Hz) related to fatigue driving, splicing the extracted frequency band and the channel dimensions to obtain 61-27 dimensional characteristic vectors, and each tested individual corresponds to 1400-1647 dimensional sample data.
And step 3: cross-tested EEG cognitive state evaluation method based on LA-MSDA model
Inputting: firstly, a multi-source domain sample data set U with a cognitive state labelsThe method specifically comprises the following steps:
Figure BDA0003093122610000071
where N is the total number of source domains (i.e., the total number of subjects), N is the nth source domain,
Figure BDA0003093122610000072
a sample representing the nth source domain,
Figure BDA0003093122610000073
the true label corresponding to the nth source domain sample is represented, more specifically, as follows:
Figure BDA0003093122610000074
wherein
Figure BDA0003093122610000075
The ith sample representing the nth source domain,
Figure BDA0003093122610000076
the real label, | X, corresponding to the ith sample of the nth source domain is representedsnL is the total number of samples of the nth source domain;
② target domain sample data U without cognition state labelTThe method specifically comprises the following steps:
Figure BDA0003093122610000081
wherein
Figure BDA0003093122610000082
The ith sample, | U, representing the target domainTL is the total number of target domain samples;
and the maximum iteration number T.
3-1. extracting potential common domain invariant features from the source domain and the target domain by a common EEGNet-based feature extractor C-EEGNet f (-) and mapping the extracted common domain invariant features to a common feature space. Ith sample in nth source domain
Figure BDA0003093122610000083
The obtained public domain invariant features are
Figure BDA0003093122610000084
Ith sample in target domain
Figure BDA0003093122610000085
The obtained public domain invariant features are
Figure BDA0003093122610000086
3-2, considering the local distribution information of each pair of related subclasses between the source domain and the target domain, further passing through N subnets S-CNNs F not sharing weightn(. each pair of source and target domains is mapped to a particular feature space that enables extraction of particular features within the domain.
Ith sample in nth source domain
Figure BDA0003093122610000087
The obtained domain has the specific characteristics of
Figure BDA0003093122610000088
It is shown simplified as
Figure BDA0003093122610000089
Ith sample in target domain
Figure BDA00030931226100000810
The obtained domain has the specific characteristics of
Figure BDA00030931226100000811
It is shown simplified as
Figure BDA00030931226100000812
The current source domain is represented as:
Figure BDA00030931226100000813
wherein the content of the first and second substances,
Figure BDA00030931226100000814
the sample data representing the current nth source domain is specifically as follows:
Figure BDA00030931226100000815
the current target domain is represented as:
Figure BDA00030931226100000816
wherein the content of the first and second substances,
Figure BDA00030931226100000817
and the sample data of the current nth source domain corresponding to the target domain is represented.
3-3. for each S-CNNs, a specific classifier G is trainednAnd adds a classification penalty to each classifier that learns the ideal values of weights and biases through the labeled samples from the multi-source domain and attempts to find a way to minimize the penalty. Supervised loss L for multi-source domainscComprises the following steps:
Figure BDA00030931226100000818
where J is the cross entropy loss function.
3-4, adjusting the distribution of local subcategories in the source domain and the target domain: a local category maximum mean difference method (LLMMD) is proposed based on the Maximum Mean Difference (MMD), and a local category weight constraint is added to each sample so as to effectively realize the comparison of local categories and realize the alignment of label-based fine-grained feature distribution in a source domain and a target domain.
Suppose an active domain XSAnd a target domain XTThen its MMD can be expressed as follows:
Figure BDA0003093122610000091
where sup denotes the supremum of the supremum,
Figure BDA0003093122610000092
representing a feature mapping function that maps a domain-specific feature distribution to a Regenerated Kernel Hilbert Space (RKHS), H being a regenerated kernel Hilbert spaceM, p and q represent the domain XSAnd XTDistribution of (2). Further expanding equation (8) yields:
Figure BDA0003093122610000093
wherein xiRepresenting a source domain XSOf (a), n samples in total, xjRepresenting a target domain XTFor the jth sample in (e), there are m samples, as follows:
Figure BDA0003093122610000094
will DH(XS,XT) Equation (9) further expands to:
Figure BDA0003093122610000095
wherein k (x)i,xj) For Gaussian kernel functions, an infinite dimensional space can be mapped, each kernel function k corresponding to one RKHS, k (x)i,xj) As defined below:
Figure BDA0003093122610000096
where σ is the gaussian filter width.
3-5, by aligning the specific feature distribution of each domain, the domain offset and target domain label-free problems are effectively solved. Suppose alphacA multi-source domain set alpha based on the prior distribution of class labels is the probability that a sample belongs to the class label ccsIs defined as:
Figure BDA0003093122610000101
wherein alpha iscnA class label prior probability representing the nth source domain,yi cnthe probability that the sample true label representing the nth source domain belongs to the class c class label,
Figure BDA0003093122610000102
the ith sample representing the nth source domain
Figure BDA0003093122610000103
A local class label weight assigned to the class label c.
For samples of the target domain, the nth classifier G is used since the samples of the target domain are unlabelednTo indicate that it belongs to the class label c, and to sample the target domain
Figure BDA0003093122610000104
Representation of belonging to class label c as
Figure BDA0003093122610000105
Its weight
Figure BDA0003093122610000106
Is defined as:
Figure BDA0003093122610000107
wherein the content of the first and second substances,
Figure BDA0003093122610000108
representing a sample
Figure BDA0003093122610000109
The local class label weight assigned to the class label c by the g-th classifier,
Figure BDA00030931226100001010
the probability that the ith sample pseudo label of the g-th target domain belongs to the class c class label is represented.
Denote LLMMD as loss Llocal：
Figure BDA00030931226100001011
Where C represents the total number of category labels.
And 3-6, further considering global distribution difference, aligning the predicted distribution of the target sample output by each classifier, and making a correct decision by the sample near the decision boundary in an alignment mode, so as to improve the generalization capability of the model in the cross-tested electroencephalogram analysis. The loss of variance is calculated using the representation outputs from the different classifiers, as follows:
Figure BDA0003093122610000111
where N is the total number of classifiers and N and m represent the indices of 2 different classifiers.
And 3-7, considering the similarity between the testees, setting a similarity weighting weight constraint according to the prediction probability distribution result of each classifier, wherein the smaller the difference between the two classifiers is, the higher the weight is, so as to realize global optimization. Weight of mth classifier
Figure BDA0003093122610000112
Is defined as:
Figure BDA0003093122610000113
wherein
Figure BDA0003093122610000114
Representing the difference loss weight between the nth and mth classifiers.
All classifiers are integrated based on weight constraints, and the resulting global penalty is as follows:
Figure BDA0003093122610000115
3-8. calculating the total objective optimization function, defined as follows:
Liotal＝Lc+βLlocal+γLglobal#(19)
where β and γ are the hyper-parameters of the model.
3-9, repeating the steps 3-1 to 3-8 until the iteration is performed for T times.
And (3) outputting: the target domain samples correspond to class labels of all classifier integration results.
The method is suitable for any cognitive state recognition based on EEG, solves the problem of individual difference of the EEG to a certain extent, and has the advantages of small time complexity, high calculation efficiency, strong generalization capability and the like.

Claims (8)

1. A multi-source domain self-adaptive cross-tested EEG cognitive state assessment method based on label alignment is characterized by comprising the following steps:
step 1: acquiring data;
the adopted fatigue driving electroencephalogram data set is 15 electroencephalogram EEG data of healthy tested subjects with better driving experience, and each tested subject fills in a NASA-TLX questionnaire after the test so as to provide subjective workload perception; selecting two mental states, TAV3 and DROWS, as analysis according to NASA-TLX questionnaire;
step 2: preprocessing data;
firstly, eliminating signals such as high-frequency noise, power frequency interference and the like except for self-generated electroencephalogram signals by using a band-pass filter, then processing by using an independent component analysis method, and finally extracting EEG signal characteristics by using power spectral density;
and step 3: a cross-tested EEG cognitive state evaluation method based on an LA-MSDA model;
inputting: firstly, multi-source domain sample data with cognitive state label
Figure FDA0003093122600000011
Wherein N is the total number of source domains, N is the nth source domain,
Figure FDA0003093122600000012
a sample representing the nth source domain,
Figure FDA0003093122600000013
representing a real label corresponding to the nth source domain sample;
second, target domain sample data without cognitive state label
Figure FDA0003093122600000014
Wherein
Figure FDA0003093122600000015
The ith sample, | U, representing the target domainTL is the total number of target domain samples;
the maximum iteration number T;
3-1, extracting potential public domain invariant features from the source domain and the target domain through a public EEGNet-based feature extractor C-EEGNetf (-) and mapping the extracted public domain invariant features to a public feature space;
3-2. passing through N subnets S-CNNsF not sharing weightn(ii) mapping each pair of source and target domains to a particular feature space, extracting particular features within the source and target domains, respectively
Figure FDA0003093122600000016
And
Figure FDA0003093122600000017
3-3. for each S-CNNs, a specific classifier G is trainednAnd adding a classification penalty to each classifier that learns the ideal values of weights and biases through the labeled samples from the multi-source domain and attempts to find a way to minimize the penalty;
3-4, adding weight constraint for each sample through a local category maximum mean difference method (LLMMD), and adjusting the distribution of local subcategories;
3-5, aligning the specific feature distribution of each domain;
3-6, aligning the predicted distribution of the target sample output by each classifier, and calculating the difference loss by using the representation output from different classifiers;
3-7, setting similarity weighting weight constraint according to the prediction probability distribution result of each classifier, wherein the smaller the difference between the two classifiers is, the higher the weight is;
3-8, calculating a total objective optimization function;
3-9, repeating the step 3-1 to the step 3-8 until the iteration is carried out for T times;
and (3) outputting: the target domain samples correspond to class labels of the integrated results of all classifiers.
2. The label alignment-based multi-source domain adaptive cross-tested EEG cognitive state assessment method according to claim 1, characterized in that step 3-2 is implemented as follows:
ith sample in nth source domain
Figure FDA0003093122600000021
The obtained domain has the specific characteristics of
Figure FDA0003093122600000022
It is shown simplified as
Figure FDA0003093122600000023
Ith sample in target domain
Figure FDA0003093122600000024
The obtained domain has the specific characteristics of
Figure FDA0003093122600000025
It is shown simplified as
Figure FDA0003093122600000026
The current source domain is represented as:
Figure FDA0003093122600000027
wherein the content of the first and second substances,
Figure FDA0003093122600000028
the sample data representing the current nth source domain is specifically as follows:
Figure FDA0003093122600000029
the current target domain is represented as:
Figure FDA00030931226000000210
wherein the content of the first and second substances,
Figure FDA00030931226000000211
and the sample data of the current nth source domain corresponding to the target domain is represented.
3. The label alignment-based multi-source domain adaptive cross-tested EEG cognitive state assessment method according to claim 2, characterized in that the loss calculation of step 3-3 is as follows:
Figure FDA00030931226000000212
wherein J is a cross entropy loss function, | XsnL is the total number of samples of the nth source domain,
Figure FDA00030931226000000213
is the true label of the ith sample of the nth source domain.
4. The label alignment-based multi-source domain adaptive cross-tested EEG cognitive state assessment method according to claim 3, characterized in that steps 3-4 are embodied as follows:
suppose an active domain XSAnd a target domain XTUsing xiRepresenting a source domain XSOf (a), n samples in total, xjRepresenting a target domain XTThe j-th sample in (b), m samples in total, the MMD is defined as follows:
Figure FDA0003093122600000031
where sup denotes the supremum of the supremum,
Figure FDA0003093122600000032
representing a feature mapping function that maps a domain-specific feature distribution to a Regenerated Kernel Hilbert Space (RKHS), H being the regenerated kernel Hilbert space, p and q representing the domain XSAnd XTDistribution of (c), k (x)i,xj) Is a gaussian kernel function.
5. The label alignment-based multi-source domain adaptive cross-tested EEG cognitive state assessment method according to claim 3, characterized in that steps 3-5 are embodied as follows:
aligning the specific feature distribution of each domain; suppose alphacFor the probability that a sample belongs to the class label c, LLMMD is expressed as:
Figure FDA0003093122600000033
where C represents the total number of category labels,
Figure FDA0003093122600000034
the ith sample representing the nth source domain
Figure FDA0003093122600000035
A local class label weight assigned to class label c,
Figure FDA0003093122600000036
i-th sample representing a target domain
Figure FDA0003093122600000041
A local class label weight assigned to the class label c.
6. The label alignment-based multi-source domain adaptive cross-tested EEG cognitive state assessment method according to claim 5, characterized in that steps 3-6 are embodied as follows:
the predicted distributions of the target samples for each classifier output are aligned and the loss of variance is calculated using the representation outputs from the different classifiers as follows:
Figure FDA0003093122600000042
。
7. the label alignment-based multi-source domain adaptive cross-tested EEG cognitive state assessment method according to claim 6, characterized in that steps 3-7 are embodied as follows:
setting similarity weighted weight constraint according to the prediction probability distribution result of each classifier, wherein the smaller the difference between the two classifiers is, the higher the weight is; weight of mth classifier
Figure FDA0003093122600000043
Is defined as:
Figure FDA0003093122600000044
wherein
Figure FDA0003093122600000045
Representing the difference loss weight between the nth and mth classifiers;
all classifiers are integrated based on weight constraints, and the resulting global penalty is as follows:
Figure FDA0003093122600000046
。
8. the label alignment-based multi-source domain adaptive cross-tested EEG cognitive state assessment method according to claim 7, characterized in that the overall objective optimization function of steps 3-8 is defined as follows:
Ltotal＝Lc+βLlocal+γLglobal#(7)
where β and γ are the corresponding weights.
CN202110601409.9A 2021-05-31 2021-05-31 Multi-source domain self-adaptive cross-tested EEG cognitive state evaluation method based on label alignment Active CN113392733B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110601409.9A CN113392733B (en) 2021-05-31 2021-05-31 Multi-source domain self-adaptive cross-tested EEG cognitive state evaluation method based on label alignment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110601409.9A CN113392733B (en) 2021-05-31 2021-05-31 Multi-source domain self-adaptive cross-tested EEG cognitive state evaluation method based on label alignment

Publications (2)

Publication Number Publication Date
CN113392733A true CN113392733A (en) 2021-09-14
CN113392733B CN113392733B (en) 2022-06-21

Family

ID=77619597

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110601409.9A Active CN113392733B (en) 2021-05-31 2021-05-31 Multi-source domain self-adaptive cross-tested EEG cognitive state evaluation method based on label alignment

Country Status (1)

Country Link
CN (1) CN113392733B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113807299A (en) * 2021-09-26 2021-12-17 河南工业大学 Sleep stage staging method and system based on parallel frequency domain electroencephalogram signals
CN113842151A (en) * 2021-09-30 2021-12-28 杭州电子科技大学 Cross-tested EEG (electroencephalogram) cognitive state detection method based on efficient multi-source capsule network
CN114065852A (en) * 2021-11-11 2022-02-18 合肥工业大学 Multi-source combined self-adaption and cohesion feature extraction method based on dynamic weight

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120101401A1 (en) * 2009-04-07 2012-04-26 National University Of Ireland Method for the real-time identification of seizures in an electroencephalogram (eeg) signal
US20190142291A1 (en) * 2015-03-23 2019-05-16 Temple University-Of The Commonwealth System Of Higher Education System and Method for Automatic Interpretation of EEG Signals Using a Deep Learning Statistical Model
CN111723661A (en) * 2020-05-18 2020-09-29 华南理工大学 Brain-computer interface transfer learning method based on manifold embedding distribution alignment
CN111728609A (en) * 2020-08-26 2020-10-02 腾讯科技（深圳）有限公司 Electroencephalogram signal classification method, classification model training method, device and medium
CN112274162A (en) * 2020-09-18 2021-01-29 杭州电子科技大学 Cross-tested EEG fatigue state classification method based on generation of anti-domain self-adaption
CN112274154A (en) * 2020-09-18 2021-01-29 杭州电子科技大学 Cross-subject fatigue driving classification method based on electroencephalogram sample weight adjustment
CN112488081A (en) * 2020-12-23 2021-03-12 杭州电子科技大学 Electroencephalogram mental state detection method based on DDADSM (distributed denial of service) cross-test transfer learning
CN112580518A (en) * 2020-12-22 2021-03-30 杭州电子科技大学 Cross-tested EEG cognitive state identification method based on prototype clustering domain adaptive algorithm
CN112641450A (en) * 2020-12-28 2021-04-13 中国人民解放军战略支援部队信息工程大学 Time-varying brain network reconstruction method for dynamic video target detection
CN112684891A (en) * 2020-12-30 2021-04-20 杭州电子科技大学 Electroencephalogram signal classification method based on multi-source manifold embedding migration

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120101401A1 (en) * 2009-04-07 2012-04-26 National University Of Ireland Method for the real-time identification of seizures in an electroencephalogram (eeg) signal
US20190142291A1 (en) * 2015-03-23 2019-05-16 Temple University-Of The Commonwealth System Of Higher Education System and Method for Automatic Interpretation of EEG Signals Using a Deep Learning Statistical Model
CN111723661A (en) * 2020-05-18 2020-09-29 华南理工大学 Brain-computer interface transfer learning method based on manifold embedding distribution alignment
CN111728609A (en) * 2020-08-26 2020-10-02 腾讯科技（深圳）有限公司 Electroencephalogram signal classification method, classification model training method, device and medium
CN112274162A (en) * 2020-09-18 2021-01-29 杭州电子科技大学 Cross-tested EEG fatigue state classification method based on generation of anti-domain self-adaption
CN112274154A (en) * 2020-09-18 2021-01-29 杭州电子科技大学 Cross-subject fatigue driving classification method based on electroencephalogram sample weight adjustment
CN112580518A (en) * 2020-12-22 2021-03-30 杭州电子科技大学 Cross-tested EEG cognitive state identification method based on prototype clustering domain adaptive algorithm
CN112488081A (en) * 2020-12-23 2021-03-12 杭州电子科技大学 Electroencephalogram mental state detection method based on DDADSM (distributed denial of service) cross-test transfer learning
CN112641450A (en) * 2020-12-28 2021-04-13 中国人民解放军战略支援部队信息工程大学 Time-varying brain network reconstruction method for dynamic video target detection
CN112684891A (en) * 2020-12-30 2021-04-20 杭州电子科技大学 Electroencephalogram signal classification method based on multi-source manifold embedding migration

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
HONG ZENG, CHEN YANG, GUOJUN DAI, FEIWEI QIN, JIANHAI ZHANG & WA: "EEG classification of driver mental states by deep learning", 《COGNITIVE NEURODYNAMICS》 *
XINCHAI ET AL.: "Unsupervised domain adaptation techniques based on auto-encoder for non-stationary EEG-based emotion recognition", 《COMPUTERS IN BIOLOGY AND MEDICINE》 *
许旋: "基于运动想象的小样本分类算法研究", 《中国优秀硕士学位论文全文数据库-医药卫生科技辑》 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113807299A (en) * 2021-09-26 2021-12-17 河南工业大学 Sleep stage staging method and system based on parallel frequency domain electroencephalogram signals
CN113807299B (en) * 2021-09-26 2023-08-08 河南工业大学 Sleep stage staging method and system based on parallel frequency domain electroencephalogram signals
CN113842151A (en) * 2021-09-30 2021-12-28 杭州电子科技大学 Cross-tested EEG (electroencephalogram) cognitive state detection method based on efficient multi-source capsule network
CN113842151B (en) * 2021-09-30 2024-01-05 杭州电子科技大学 Cross-test EEG cognitive state detection method based on efficient multi-source capsule network
CN114065852A (en) * 2021-11-11 2022-02-18 合肥工业大学 Multi-source combined self-adaption and cohesion feature extraction method based on dynamic weight
CN114065852B (en) * 2021-11-11 2024-04-16 合肥工业大学 Multisource joint self-adaption and cohesive feature extraction method based on dynamic weight

Also Published As

Publication number Publication date
CN113392733B (en) 2022-06-21

Similar Documents

Publication Publication Date Title
CN113392733B (en) Multi-source domain self-adaptive cross-tested EEG cognitive state evaluation method based on label alignment
Dissanayake et al. Deep learning for patient-independent epileptic seizure prediction using scalp EEG signals
Kumar et al. Classification of seizure and seizure-free EEG signals using local binary patterns
CN112244873A (en) Electroencephalogram time-space feature learning and emotion classification method based on hybrid neural network
CN112580518B (en) Cross-test EEG cognitive state recognition method based on prototype clustering domain adaptation algorithm
CN114533086B (en) Motor imagery brain electrolysis code method based on airspace characteristic time-frequency transformation
CN114176607B (en) Electroencephalogram signal classification method based on vision transducer
Zeng et al. GRP-DNet: A gray recurrence plot-based densely connected convolutional network for classification of epileptiform EEG
Zhang et al. Human identification driven by deep CNN and transfer learning based on multiview feature representations of ECG
Al-Saegh et al. CutCat: An augmentation method for EEG classification
CN112749635A (en) Cross-tested EEG cognitive state identification method based on prototype clustering domain adaptive algorithm
Khalighi et al. Adaptive automatic sleep stage classification under covariate shift
Wen et al. A deep learning-based classification method for different frequency EEG data
Lian et al. The improved ELM algorithms optimized by bionic WOA for EEG classification of brain computer interface
Nie et al. Recsleepnet: An automatic sleep staging model based on feature reconstruction
CN116211320A (en) Pattern recognition method of motor imagery brain-computer interface based on ensemble learning
Jiang et al. Analytical comparison of two emotion classification models based on convolutional neural networks
Alsuwaiket Feature Extraction of EEG Signals for Seizure Detection Using Machine Learning Algorthims
Li et al. MVF-sleepnet: Multi-view fusion network for sleep stage classification
Chang et al. Dssnet: A deep sequential sleep network for self-supervised representation learning based on single-channel eeg
Zan et al. Local Pattern Transformation-Based convolutional neural network for sleep stage scoring
CN116821764A (en) Knowledge distillation-based multi-source domain adaptive EEG emotion state classification method
CN113842151B (en) Cross-test EEG cognitive state detection method based on efficient multi-source capsule network
Boernama et al. Multiclass classification of brain-computer interface motor imagery system: a systematic literature review
CN113177482A (en) Cross-individual electroencephalogram signal classification method based on minimum category confusion

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant