CN113392733B - Multi-source domain self-adaptive cross-tested EEG cognitive state evaluation method based on label alignment - Google Patents

Multi-source domain self-adaptive cross-tested EEG cognitive state evaluation method based on label alignment Download PDF

Info

Publication number
CN113392733B
CN113392733B CN202110601409.9A CN202110601409A CN113392733B CN 113392733 B CN113392733 B CN 113392733B CN 202110601409 A CN202110601409 A CN 202110601409A CN 113392733 B CN113392733 B CN 113392733B
Authority
CN
China
Prior art keywords
domain
label
source domain
sample
representing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110601409.9A
Other languages
Chinese (zh)
Other versions
CN113392733A (en
Inventor
方欣
戴国骏
赵月
李秀峰
张振炎
吴政轩
吴靖
曾虹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Dianzi University
Original Assignee
Hangzhou Dianzi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Dianzi University filed Critical Hangzhou Dianzi University
Priority to CN202110601409.9A priority Critical patent/CN113392733B/en
Publication of CN113392733A publication Critical patent/CN113392733A/en
Application granted granted Critical
Publication of CN113392733B publication Critical patent/CN113392733B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/12—Classification; Matching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/08—Feature extraction

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Signal Processing (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)

Abstract

The invention discloses a label alignment-based multi-source domain self-adaptive cross-tested EEG cognitive state evaluation method. The invention comprises the following steps: 1: acquiring data; 2: preprocessing data; 3: a cross-tested EEG cognitive state assessment method based on an LA-MSDA model. The method adopts a shared common feature extractor and an unshared sub-feature extractor which are used in stages to further learn the tested invariant features and specific features of the source domain sample and the target domain sample; secondly, considering the relation and similarity among the cross-testees, a method for aligning the inter-domain distribution of local and global representations is provided to evaluate the cross-testee cognitive state, and the problems that the class condition information with fine granularity is difficult to learn and the decision boundary samples which are suitable for the cross-testees are solved. Finally, the method effectively avoids the problem of individual difference of electroencephalogram signals in the field of brain cognitive computation, is suitable for cognitive state recognition based on EEG under any task, has strong generalization capability, and can be well suitable for clinical diagnosis and practical application.

Description

Multi-source domain self-adaptive cross-tested EEG cognitive state evaluation method based on label alignment
Technical Field
The invention relates to a neuroelectrophysiological signal analysis technology in the field of brain cognitive computation and a multi-source domain adaptation model construction method in the field of unsupervised learning, in particular to a method for analyzing electroencephalogram (EEG) to evaluate cognitive states based on label alignment, and can effectively solve the problems of remarkable difference and low signal-to-noise ratio of different tested individuals.
Background
EEG-based cognitive state analysis methods have received increasing attention in recent years due to the characteristics of non-invasiveness, portability and low cost, as well as the advantages of machine learning or deep learning in extracting and classifying features from large amounts of data. Existing EEG-based analysis typically performs classification tasks combining appropriate feature extraction with classifiers, where: common methods for feature extraction include Common Space Pattern (CSP), Discrete Wavelet Transform (DWT), etc., and common classifiers include SVM, Bayes, some deep learning models, etc.
Although the signal analysis of these methods has a high discriminant performance, there are some drawbacks in the EEG-based cross-domain predictive analysis: in the real context of EEG analysis, there are significant differences in EEG between subjects, mainly due to physical (e.g., environmental and skin electrode impedance) and biological (e.g., differences in gender, age, and brain activity patterns) factors, and furthermore, EEG changes over time despite the same subject. Therefore, a general cross-subject electroencephalogram analysis model needs to be constructed.
At present, the rich results based on deep learning benefit from the supervised learning of large amounts of labeled data. However, for unsupervised learning, the main obstacle in designing a general network model is to extend the model trained from known label data to a new label-free domain. Aiming at the target task lacking the tag data, the key breakthrough point for solving the problem is to fully utilize the characteristic information of the tagged data in the source domain. However, the domain shift problem of the trained model when applied to a new domain often causes the performance of the model to be greatly reduced.
An Unsupervised Domain Adaptation (UDA) algorithm in migration learning is an algorithm for solving the distribution bias of a source domain and a target domain, and has been proved to be capable of effectively reducing the distribution gap between each domain by learning domain-invariant features. However, because of the high nonlinearity and significant individual difference of brain electricity, it is difficult to extract the same or similar features between different subjects, so the existing UDA method has the following two limitations: (1) the problem of feature confusion near decision boundaries cannot be completely solved, the target function is difficult to achieve the optimum, and the target function may fall into a local optimum state; (2) feature-based alignment is difficult to achieve to extract feature-based domain invariant features. Feature-based alignment is a technology for minimizing feature distribution differences, can effectively optimize the domain offset problem, and can effectively reduce the problem of individual differences by mapping the relation between the tested objects to a space conforming to a certain structure from the perspective of EEG signal characteristics and aligning the space.
In addition, relevant documents show that, different from the single-source domain adaptation problem, the multi-source domain transfer learning simultaneously transfers knowledge of a plurality of source domains to the target domain to assist the learning of the target domain, the data are different from the target domain and different from each other, and the method can effectively solve the problem of individual difference of electroencephalogram signals.
In conclusion, the invention takes the cross-test, domain adaptation, multi-source domain and feature-based alignment as key starting points to construct a label-alignment-based multi-source domain adaptive cross-test model, so as to be more efficiently applied in the field of neuro-physiological signal analysis.
Disclosure of Invention
The invention provides a label alignment-based multi-source domain self-adaptive model, fully learns the structural characteristics of a label sample aiming at the conditions of high nonlinearity and obvious individual difference existing in EEG data, adopts a local label alignment strategy, extracts the label-based domain invariant characteristics, and aligns the characteristic distribution of a target domain and the multi-source domain to realize the effective transfer of the inter-domain characteristics. The proposed algorithm has mainly the following two aspects: on one hand, the alignment strategy based on the local labels is adopted, because the labels contained in different tested EEG data are the same in type when the data are collected, the domain invariant features of the corresponding labels are extracted through local label alignment, and the influence of the EEG individual difference on the model performance can be effectively avoided. On the other hand, an improved global optimization UDA method is provided, weight constraints are set according to probability distribution results of all classifiers, a global objective function optimization strategy is adopted, the problem of fuzzy decision boundary of the classifiers is solved, and the generalization capability of a model in cross-tested EEG analysis is improved. The algorithm can effectively avoid the influence of high nonlinearity of the EEG and obvious individual difference characteristics, can achieve better performance in the cognitive state detection based on the EEG, and has wide application scenes in practice.
In summary, the cognitive state identification based on the EEG is performed by the invention, and the cognitive state to be tested is evaluated by taking nonlinearity and individual difference as key starting points, and the core technology of the invention is mainly to construct a label alignment-based multi-source domain adaptive cross-tested model (LA-MSDA). According to the method, each training individual is used as a single domain to form a plurality of source domains, a new cross-tested individual is used as a target domain, cross-tested relevance and similarity are comprehensively considered, a local alignment and global optimization strategy is introduced to evaluate a cross-tested cognitive state, the domain invariant features based on the labels can be effectively extracted, and unobvious features near a decision boundary are correctly predicted. The method fully considers the characteristic space distribution structure of the label-free data, can greatly improve the model training efficiency, has higher universality, has wide application prospect in the actual brain-computer interaction (BCI), and provides technical support for clinical application.
Existing techniques for EEG-based cognitive state assessment focus mainly on aligning global feature distributions to minimize the difference between each subject, or combining all active classifiers to make a final decision, it is still difficult to extract domain-invariant features, and it is difficult to solve the ambiguity problem near the classifier decision boundary. In order to overcome the defects of the prior method, the invention adopts the following technical scheme:
the cognitive state evaluation method based on the EEG comprehensively considers the relevance and the similarity of the cross-testees, classifies the cognitive states of the testees by carrying out feature analysis on the EEG, realizes the distinction of different states under various experimental tasks, such as the cognitive states (addiction and normal control group) of a network game addiction patient, the cognitive states (waking and fatigue) of a driver driving task, the cognitive states (pleasant mood and passive mood) of the experimental testees under different mood stimulation tasks and the like, and can process different EEG data sets.
The invention is based on the fatigue driving electroencephalogram data set, and is concretely realized as follows:
step 1: data acquisition
The fatigue driving electroencephalogram data set adopted by the invention is 15 electroencephalogram EEG data of healthy tested subjects with better driving experience, and each tested subject fills in an NASA-TLX questionnaire after the test so as to provide subjective workload perception. According to the NASA-TLX questionnaire, the present invention selected two mental states, TAV3 and DROWS, as analyses.
Step 2: data pre-processing
In order to further filter noise and remove artifacts, the EEG signal processing method aims at original EEG data, firstly, a band-pass filter (1-30Hz) is used for eliminating signals such as high-frequency noise, power frequency interference and the like except for spontaneous EEG signals, then, an Independent Component Analysis (ICA) method is used for processing, and finally, EEG signal features are extracted by using Power Spectral Density (PSD) so as to provide stable signal features for subsequent model construction.
And step 3: cross-test EEG cognitive state evaluation method based on LA-MSDA model
Inputting: firstly, multi-source domain sample data with cognitive state label
Figure GDA0003614608840000041
Where N is the total number of source domains (i.e., the total number of subjects), N is the nth source domain,
Figure GDA0003614608840000042
to representThe sample of the n-th source domain,
Figure GDA0003614608840000043
representing the corresponding real label of the nth source domain sample.
Second, target domain sample data without cognitive state label
Figure GDA0003614608840000044
Wherein
Figure GDA0003614608840000045
The ith sample, | U, representing the target domainTAnd | is the total number of target domain samples.
And the maximum iteration number T.
3-1. extracting potential common domain invariant features from the source domain and the target domain by a common EEGNet-based feature extractor C-EEGNet f (-) and mapping the extracted common domain invariant features to a common feature space.
3-2, passing through N subnets S-CNNs F not sharing weightn(ii) mapping each pair of source and target domains to a particular feature space, extracting particular features within the source and target domains, respectively
Figure GDA0003614608840000046
And
Figure GDA0003614608840000047
3-3. for each S-CNNs, a specific classifier G is trainednAnd adds a classification penalty to each classifier that learns the ideal values of weights and biases through the labeled samples from the multi-source domain and attempts to find a way to minimize the penalty. The losses are as follows:
Figure GDA0003614608840000048
wherein J is a cross entropy loss function, | XsnL is the total number of samples of the nth source domain,
Figure GDA0003614608840000049
is the true label of the ith sample of the nth source domain.
3-4, adding weight constraint for each sample through a local category maximum mean difference method (LLMMD), and adjusting the distribution of local subcategories on the assumption that the active domain XSAnd a target domain XTUsing xiRepresenting a source field XSOf (a), n samples in total, xjRepresenting a target domain XTThe j-th sample in (b), m samples in total, the MMD is defined as follows:
Figure GDA0003614608840000051
where sup denotes the supremum of the supremum,
Figure GDA0003614608840000052
representing a feature mapping function that maps a domain-specific feature distribution to a Regenerated Kernel Hilbert Space (RKHS), H being the regenerated kernel Hilbert space, p and q representing the domain XSAnd XTDistribution of (c), k (x)i,xj) Is a gaussian kernel function.
3-5. align the specific feature distributions of each domain. Suppose alphacFor the probability that a sample belongs to the class label c, LLMMD is expressed as:
Figure GDA0003614608840000053
where C represents the total number of category labels,
Figure GDA0003614608840000054
the ith sample representing the nth source domain
Figure GDA0003614608840000055
A local class label weight assigned to class label c,
Figure GDA0003614608840000056
i-th sample representing a target domain
Figure GDA0003614608840000057
A local class label weight assigned to the class label c.
3-6. align the predicted distributions of the target samples output by each classifier, using the representative outputs from the different classifiers to calculate the loss of variance as follows:
Figure GDA0003614608840000058
and 3-7, setting similarity weighting weight constraint according to the prediction probability distribution result of each classifier, wherein the smaller the difference between the two classifiers is, the higher the weight is. Weight of mth classifier
Figure GDA0003614608840000059
Is defined as follows:
Figure GDA0003614608840000061
wherein
Figure GDA0003614608840000062
Representing the difference loss weight between the nth and mth classifiers.
All classifiers are integrated based on weight constraints, and the resulting global penalty is as follows:
Figure DEST_PATH_FDA0003614608830000054
3-8. calculating the total objective optimization function, defined as follows:
Ltotal＝Lc+βLlocal+γLglobal
where β and γ are the corresponding weights.
3-9, repeating the steps 3-1 to 3-8 until the iteration is performed for T times.
And (3) outputting: the target domain samples correspond to class labels of the integrated results of all classifiers.
The main contributions of the invention are:
firstly, a method of using a shared common feature extractor and an unshared sub-feature extractor in stages is adopted to further learn the tested invariant features and specific features of the source domain samples and the target domain samples, and the defect that the traditional method only uses the domain invariant features is optimized. Secondly, considering the relation and similarity among the cross-testees, a method for aligning the inter-domain distribution of local and global representations is provided to evaluate the cross-testee cognitive state, and the problems that the class condition information with fine granularity is difficult to learn and the decision boundary samples which are suitable for the cross-testees are solved. Finally, the method effectively avoids the problem of individual difference of electroencephalogram signals in the field of brain cognitive computation, is suitable for cognitive state recognition based on EEG under any task, has strong generalization capability, and can be well suitable for clinical diagnosis and practical application.
Drawings
FIG. 1 is a diagram of a model architecture of the present invention.
Detailed Description
The invention is further illustrated with reference to the following figures and examples.
As shown in fig. 1, the method is a structural diagram of a multi-source domain adaptive cross-tested EEG cognitive state assessment method based on label alignment, and mainly includes the following steps:
step 1: data acquisition
The fatigue driving electroencephalogram data set adopted by the invention is 15 electroencephalogram EEG data of healthy tested subjects with better driving experience, and each tested subject fills in an NASA-TLX questionnaire after the test so as to provide subjective workload perception. According to the NASA-TLX questionnaire, the present invention selected two mental states, TAV3 and DROWS, as analyses.
Step 2: data pre-processing
Taking fatigue driving EEG data as an example, the raw EEG data processing steps are as follows:
2-1, artifact removal: performing artifact removal operation on the acquired original EEG data, firstly performing 1-30Hz band-pass filtering processing, and simultaneously removing power frequency interference and direct current components in the signals; and then removing artifacts in the signal by ICA independent component analysis.
2-2, PSD feature extraction: extracting PSD (phase-sensitive Detector) characteristics of the EEG with the artifacts removed, performing data segmentation on the EEG of each tested individual through a sliding window of 0.5s to obtain 1400 samples, wherein the number of EEG data acquisition channels is 61, extracting a frequency band 4-30Hz (wherein theta is 4-7Hz, alpha is 8-13Hz, and beta is 14-30Hz) related to fatigue driving, splicing the extracted frequency band and the channel dimensions to obtain 61-27 dimensional characteristic vectors, and each tested individual corresponds to 1400-1647 dimensional sample data.
And step 3: cross-tested EEG cognitive state evaluation method based on LA-MSDA model
Inputting: firstly, a multi-source domain sample data set U with a cognitive state labelsThe method specifically comprises the following steps:
Figure GDA0003614608840000071
where N is the total number of source domains (i.e., the total number of subjects), N is the nth source domain,
Figure GDA0003614608840000072
a sample representing the nth source domain,
Figure GDA0003614608840000073
the true label corresponding to the nth source domain sample is represented, more specifically, as follows:
Figure GDA0003614608840000074
wherein
Figure GDA0003614608840000075
The ith sample representing the nth source domain,
Figure GDA0003614608840000076
the real label, | X, corresponding to the ith sample of the nth source domain is representedsnL is the total number of samples of the nth source domain;
② target domain sample data U without cognition state labelTThe method specifically comprises the following steps:
Figure GDA0003614608840000081
wherein
Figure GDA0003614608840000082
The ith sample, | U, representing the target domainTL is the total number of target domain samples;
and the maximum iteration number T.
3-1. extracting potential common domain invariant features from the source domain and the target domain by a common EEGNet-based feature extractor C-EEGNet f (-) and mapping the extracted common domain invariant features to a common feature space. Ith sample in nth source domain
Figure GDA0003614608840000083
The obtained public domain invariant features are
Figure GDA0003614608840000084
Ith sample in target domain
Figure GDA0003614608840000085
The obtained public domain invariant features are
Figure GDA0003614608840000086
3-2, considering the local distribution information of each pair of related subclasses between the source domain and the target domain, further passing through N subnets S-CNNs F not sharing weightn(. each pair of source and target domains is mapped to a particular feature space that enables extraction of particular features within the domain.
Ith sample in nth source domain
Figure GDA0003614608840000087
The obtained domain has the specific characteristics of
Figure GDA0003614608840000088
It is simplified to be represented as
Figure GDA0003614608840000089
Ith sample in target domain
Figure GDA00036146088400000810
The obtained domain has the specific characteristics of
Figure GDA00036146088400000811
It is shown simplified as
Figure GDA00036146088400000812
The current source domain is represented as:
Figure GDA00036146088400000813
wherein the content of the first and second substances,
Figure GDA00036146088400000814
the sample data representing the current nth source domain is specifically as follows:
Figure GDA00036146088400000815
the current target domain is represented as:
Figure GDA00036146088400000816
wherein the content of the first and second substances,
Figure GDA00036146088400000817
and the sample data of the current nth source domain corresponding to the target domain is represented.
3-3. for each S-CNNs, a specific classifier G is trainednAnd adds a classification penalty to each classifier that learns the ideal values of weights and biases through the labeled samples from the multi-source domain and attempts to find a way to minimize the penalty. Supervised loss L for multi-source domainscComprises the following steps:
Figure GDA00036146088400000818
where J is the cross entropy loss function.
3-4, adjusting the distribution of local subcategories in the source domain and the target domain: a local category maximum mean difference method (LLMMD) is proposed based on the Maximum Mean Difference (MMD), and a local category weight constraint is added to each sample so as to effectively realize the comparison of local categories and realize the alignment of label-based fine-grained feature distribution in a source domain and a target domain.
Suppose an active domain XSAnd target domain XTThen its MMD can be expressed as follows:
Figure GDA0003614608840000091
where sup denotes the supremum of the supremum,
Figure GDA0003614608840000092
representing a feature mapping function that maps a domain-specific feature distribution to a Regenerated Kernel Hilbert Space (RKHS), H being the regenerated kernel Hilbert space, p and q representing the domain XSAnd XTDistribution of (2). Further expanding equation (8) yields:
Figure GDA0003614608840000093
wherein xiRepresenting a source domain XSThe ith sample, for a total of n samples,xjrepresenting a target domain XTFor the jth sample in (e), there are m samples, as follows:
Figure GDA0003614608840000094
will DH(XS,XT) Equation (9) further expands to:
Figure GDA0003614608840000095
wherein k (x)i,xj) For Gaussian kernel functions, an infinite dimensional space can be mapped, each kernel function k corresponding to one RKHS, k (x)i,xj) As defined below:
Figure GDA0003614608840000096
where σ is the gaussian filter width.
3-5, by aligning the specific feature distribution of each domain, the domain offset and target domain label-free problems are effectively solved. Suppose αcA multi-source domain set alpha based on the prior distribution of class labels is the probability that a sample belongs to the class label ccsIs defined as:
Figure GDA0003614608840000101
wherein alpha iscnClass label prior probability, y, representing the nth source domaini cnThe probability that the sample true label representing the nth source domain belongs to the class c class label,
Figure GDA0003614608840000102
the ith sample representing the nth source domain
Figure GDA0003614608840000103
Is distributed toLocal class label weight for class label c.
For samples of the target domain, the nth classifier G is used since the samples of the target domain are unlabelednTo indicate that it belongs to the class label c, and to sample the target domain
Figure GDA0003614608840000104
Representation of belonging to class label c as
Figure GDA0003614608840000105
Its weight
Figure GDA0003614608840000106
Is defined as:
Figure GDA0003614608840000107
wherein the content of the first and second substances,
Figure GDA0003614608840000108
representing a sample
Figure GDA0003614608840000109
The local class label weight assigned to the class label c by the g-th classifier,
Figure GDA00036146088400001010
the probability that the ith sample pseudo label of the g-th target domain belongs to the class c class label is represented.
Denote LLMMD as loss Llocal：
Figure GDA00036146088400001011
Where C represents the total number of category labels.
And 3-6, further considering global distribution difference, aligning the predicted distribution of the target sample output by each classifier, and making a correct decision by the sample near the decision boundary in an alignment mode, so as to improve the generalization capability of the model in the cross-tested electroencephalogram analysis. The difference loss is calculated using the representation outputs from the different classifiers, as follows:
Figure GDA0003614608840000111
where N is the total number of classifiers and N and m represent the indices of 2 different classifiers.
And 3-7, considering the similarity between the testees, setting a similarity weighting weight constraint according to the prediction probability distribution result of each classifier, wherein the smaller the difference between the two classifiers is, the higher the weight is, so as to realize global optimization. Weight of mth classifier
Figure GDA0003614608840000112
Is defined as:
Figure GDA0003614608840000113
wherein
Figure GDA0003614608840000114
Representing the difference loss weight between the nth and mth classifiers.
All classifiers are integrated based on weight constraints, and the resulting global penalty is as follows:
Figure GDA0003614608840000115
3-8. calculating the total objective optimization function, defined as follows:
Ltotal＝Lc+βLlocal+γLglobal (19)
where β and γ are the hyper-parameters of the model.
3-9, repeating the steps 3-1 to 3-8 until the iteration is performed for T times.
And (3) outputting: the target domain samples correspond to class labels of the integrated results of all classifiers.
The method is suitable for any cognitive state recognition based on EEG, solves the problem of individual difference of the EEG to a certain extent, and has the advantages of small time complexity, high calculation efficiency, strong generalization capability and the like.

Claims (8)

1. A multi-source domain self-adaptive cross-tested EEG cognitive state assessment method based on label alignment is characterized by comprising the following steps:
step 1: acquiring data;
the adopted fatigue driving electroencephalogram data set is 15 electroencephalogram EEG data of healthy tested subjects with better driving experience, and each tested subject fills in a NASA-TLX questionnaire after the test so as to provide subjective workload perception; selecting two mental states, TAV3 and DROWS, as analysis according to NASA-TLX questionnaire;
step 2: preprocessing data;
firstly, eliminating signals such as high-frequency noise, power frequency interference and the like except for self-generated electroencephalogram signals by using a band-pass filter, then processing by using an independent component analysis method, and finally extracting EEG signal characteristics by using power spectral density;
and step 3: a cross-tested EEG cognitive state evaluation method based on an LA-MSDA model;
inputting: firstly, multi-source domain sample data with cognitive state label
Figure FDA0003614608830000011
Wherein N is the total number of source domains, N is the nth source domain,
Figure FDA0003614608830000012
a sample representing the nth source domain,
Figure FDA0003614608830000013
representing a real label corresponding to the nth source domain sample;
second, target domain sample data without cognitive state label
Figure FDA0003614608830000014
Wherein
Figure FDA0003614608830000015
The ith sample, | U, representing the target domainTL is the total number of target domain samples;
the maximum iteration number T;
3-1, extracting potential public domain invariant features from the source domain and the target domain through a public EEGNet-based feature extractor C-EEGNet f (-) and mapping the extracted public domain invariant features to a public feature space;
3-2, passing through N subnets S-CNNs F not sharing weightn(ii) mapping each pair of source and target domains to a particular feature space, extracting particular features within the source and target domains, respectively
Figure FDA0003614608830000016
And
Figure FDA0003614608830000017
3-3. for each S-CNNs, a specific classifier G is trainednAnd adding a classification penalty to each classifier that learns the ideal values of weights and biases through the labeled samples from the multi-source domain and attempts to find a way to minimize the penalty;
3-4, adding weight constraint for each sample through a local category maximum mean difference method (LLMMD), and adjusting the distribution of local subcategories;
3-5, aligning the specific feature distribution of each domain;
3-6, aligning the predicted distribution of the target sample output by each classifier, and calculating the difference loss by using the representation output from different classifiers;
3-7, setting similarity weighting weight constraint according to the prediction probability distribution result of each classifier, wherein the smaller the difference between the two classifiers is, the higher the weight is;
3-8, calculating a total objective optimization function;
3-9, repeating the step 3-1 to the step 3-8 until the iteration is carried out for T times;
and (3) outputting: the target domain samples correspond to class labels of the integrated results of all classifiers.
2. The label alignment-based multi-source domain adaptive cross-tested EEG cognitive state assessment method according to claim 1, characterized in that step 3-2 is implemented as follows:
ith sample in nth source domain
Figure FDA0003614608830000021
The obtained domain has the specific characteristics of
Figure FDA0003614608830000022
It is shown simplified as
Figure FDA0003614608830000023
Ith sample in target domain
Figure FDA0003614608830000024
The obtained domain has the specific characteristics of
Figure FDA0003614608830000025
It is shown simplified as
Figure FDA0003614608830000026
The current source domain is represented as:
Figure FDA0003614608830000027
wherein the content of the first and second substances,
Figure FDA0003614608830000028
the sample data representing the current nth source domain is specifically as follows:
Figure FDA0003614608830000029
wherein, | XsnL is the total number of samples of the nth source domain; the current target domain is represented as:
Figure FDA00036146088300000210
wherein, the first and the second end of the pipe are connected with each other,
Figure FDA00036146088300000211
and the sample data of the current nth source domain corresponding to the target domain is represented.
3. The label alignment-based multi-source domain adaptive cross-tested EEG cognitive state assessment method according to claim 2, characterized in that the loss calculation of step 3-3 is as follows:
Figure FDA00036146088300000212
wherein J is a cross entropy loss function, | XsnL is the total number of samples of the nth source domain,
Figure FDA00036146088300000213
is the true label of the ith sample of the nth source domain.
4. The label alignment-based multi-source domain adaptive cross-tested EEG cognitive state assessment method according to claim 3, characterized in that steps 3-4 are embodied as follows:
suppose an active domain XSAnd a target domain XTUsing xiRepresenting a source domain XSOf the ith sample, n samples, xjRepresenting a target domain XTThe j-th sample in (b), m samples in total, the MMD is defined as follows:
Figure FDA0003614608830000031
where sup denotes the supremum of the supremum,
Figure FDA0003614608830000032
representing a feature mapping function that maps a domain-specific feature distribution to a Regenerated Kernel Hilbert Space (RKHS), H being the regenerated kernel Hilbert space, p and q representing the domain XSAnd XTDistribution of (E), E*Indicating a desire; k (x)i,xj) Is a gaussian kernel function.
5. The label alignment-based multi-source domain adaptive cross-tested EEG cognitive state assessment method according to claim 3, characterized in that steps 3-5 are embodied as follows:
by aligning the specific feature distribution of each domain, the domain offset and the target domain label-free problem are effectively solved; suppose alphacA multi-source domain set alpha based on the prior distribution of class labels is the probability that a sample belongs to the class label ccsIs defined as:
Figure FDA0003614608830000033
wherein alpha iscnClass label prior probability, y, representing the nth source domaini cnThe probability that the sample true label representing the nth source domain belongs to the class c class label,
Figure FDA0003614608830000034
the ith sample representing the nth source domain
Figure FDA0003614608830000035
A local class label weight assigned to class label c;
samples for the target domain, becauseSamples of the target domain are unlabeled using an nth classifier GnTo indicate that it belongs to the class label c, and to sample the target domain
Figure FDA0003614608830000041
Representation of belonging to class label c as
Figure FDA0003614608830000042
Its weight
Figure FDA0003614608830000043
Is defined as follows:
Figure FDA0003614608830000044
wherein the content of the first and second substances,
Figure FDA0003614608830000045
representing a sample
Figure FDA0003614608830000046
The local class label weight assigned to the class label c by the g-th classifier,
Figure FDA0003614608830000047
representing the probability that the ith sample pseudo label of the g target domain belongs to the class c class label;
denote LLMMD as loss Llocal：
Figure FDA0003614608830000048
Where C represents the total number of category labels,
Figure FDA0003614608830000049
the ith sample representing the nth source domain
Figure FDA00036146088300000410
A local class label weight assigned to the class label c,
Figure FDA00036146088300000411
i-th sample representing a target domain
Figure FDA00036146088300000412
A local class label weight assigned to the class label c.
6. The label alignment-based multi-source domain adaptive cross-tested EEG cognitive state assessment method according to claim 5, characterized in that steps 3-6 are embodied as follows:
the predicted distributions of the target samples for each classifier output are aligned and the loss of variance is calculated using the representation outputs from the different classifiers as follows:
Figure FDA00036146088300000413
wherein G isn、GmRespectively representing the n-th classifier and the m-th classifier.
7. The label alignment-based multi-source domain adaptive cross-tested EEG cognitive state assessment method according to claim 6, characterized in that steps 3-7 are embodied as follows:
setting similarity weighted weight constraint according to the prediction probability distribution result of each classifier, wherein the smaller the difference between the two classifiers is, the higher the weight is; weight of mth classifier
Figure FDA0003614608830000051
Is defined as:
Figure FDA0003614608830000052
wherein
Figure FDA0003614608830000053
Representing the difference loss weight between the nth and mth classifiers;
integrating all classifiers based on weight constraints, the resulting global penalty is as follows:
Figure FDA0003614608830000054
8. the label alignment-based multi-source domain adaptive cross-tested EEG cognitive state assessment method according to claim 7, characterized in that the overall objective optimization function of steps 3-8 is defined as follows:
Ltotal＝Lc+βLlocal+γLglobal
where β and γ are the corresponding weights.
CN202110601409.9A 2021-05-31 2021-05-31 Multi-source domain self-adaptive cross-tested EEG cognitive state evaluation method based on label alignment Active CN113392733B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110601409.9A CN113392733B (en) 2021-05-31 2021-05-31 Multi-source domain self-adaptive cross-tested EEG cognitive state evaluation method based on label alignment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110601409.9A CN113392733B (en) 2021-05-31 2021-05-31 Multi-source domain self-adaptive cross-tested EEG cognitive state evaluation method based on label alignment

Publications (2)

Publication Number Publication Date
CN113392733A CN113392733A (en) 2021-09-14
CN113392733B true CN113392733B (en) 2022-06-21

Family

ID=77619597

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110601409.9A Active CN113392733B (en) 2021-05-31 2021-05-31 Multi-source domain self-adaptive cross-tested EEG cognitive state evaluation method based on label alignment

Country Status (1)

Country Link
CN (1) CN113392733B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113807299B (en) * 2021-09-26 2023-08-08 河南工业大学 Sleep stage staging method and system based on parallel frequency domain electroencephalogram signals
CN113842151B (en) * 2021-09-30 2024-01-05 杭州电子科技大学 Cross-test EEG cognitive state detection method based on efficient multi-source capsule network
CN114065852B (en) * 2021-11-11 2024-04-16 合肥工业大学 Multisource joint self-adaption and cohesive feature extraction method based on dynamic weight

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111723661A (en) * 2020-05-18 2020-09-29 华南理工大学 Brain-computer interface transfer learning method based on manifold embedding distribution alignment
CN111728609A (en) * 2020-08-26 2020-10-02 腾讯科技（深圳）有限公司 Electroencephalogram signal classification method, classification model training method, device and medium
CN112274162A (en) * 2020-09-18 2021-01-29 杭州电子科技大学 Cross-tested EEG fatigue state classification method based on generation of anti-domain self-adaption
CN112274154A (en) * 2020-09-18 2021-01-29 杭州电子科技大学 Cross-subject fatigue driving classification method based on electroencephalogram sample weight adjustment
CN112488081A (en) * 2020-12-23 2021-03-12 杭州电子科技大学 Electroencephalogram mental state detection method based on DDADSM (distributed denial of service) cross-test transfer learning
CN112580518A (en) * 2020-12-22 2021-03-30 杭州电子科技大学 Cross-tested EEG cognitive state identification method based on prototype clustering domain adaptive algorithm
CN112641450A (en) * 2020-12-28 2021-04-13 中国人民解放军战略支援部队信息工程大学 Time-varying brain network reconstruction method for dynamic video target detection
CN112684891A (en) * 2020-12-30 2021-04-20 杭州电子科技大学 Electroencephalogram signal classification method based on multi-source manifold embedding migration

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB0906029D0 (en) * 2009-04-07 2009-05-20 Nat Univ Ireland Cork A method of analysing an electroencephalogram (EEG) signal
US20190142291A1 (en) * 2015-03-23 2019-05-16 Temple University-Of The Commonwealth System Of Higher Education System and Method for Automatic Interpretation of EEG Signals Using a Deep Learning Statistical Model

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111723661A (en) * 2020-05-18 2020-09-29 华南理工大学 Brain-computer interface transfer learning method based on manifold embedding distribution alignment
CN111728609A (en) * 2020-08-26 2020-10-02 腾讯科技（深圳）有限公司 Electroencephalogram signal classification method, classification model training method, device and medium
CN112274162A (en) * 2020-09-18 2021-01-29 杭州电子科技大学 Cross-tested EEG fatigue state classification method based on generation of anti-domain self-adaption
CN112274154A (en) * 2020-09-18 2021-01-29 杭州电子科技大学 Cross-subject fatigue driving classification method based on electroencephalogram sample weight adjustment
CN112580518A (en) * 2020-12-22 2021-03-30 杭州电子科技大学 Cross-tested EEG cognitive state identification method based on prototype clustering domain adaptive algorithm
CN112488081A (en) * 2020-12-23 2021-03-12 杭州电子科技大学 Electroencephalogram mental state detection method based on DDADSM (distributed denial of service) cross-test transfer learning
CN112641450A (en) * 2020-12-28 2021-04-13 中国人民解放军战略支援部队信息工程大学 Time-varying brain network reconstruction method for dynamic video target detection
CN112684891A (en) * 2020-12-30 2021-04-20 杭州电子科技大学 Electroencephalogram signal classification method based on multi-source manifold embedding migration

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
EEG classification of driver mental states by deep learning;Hong Zeng, Chen Yang, Guojun Dai, Feiwei Qin, Jianhai Zhang & Wa;《Cognitive Neurodynamics》;20180718;全文 *
Unsupervised domain adaptation techniques based on auto-encoder for non-stationary EEG-based emotion recognition;XinChai et al.;《Computers in Biology and Medicine》;20161201;全文 *
基于运动想象的小样本分类算法研究;许旋;《中国优秀硕士学位论文全文数据库-医药卫生科技辑》;20210215;全文 *

Also Published As

Publication number Publication date
CN113392733A (en) 2021-09-14

Similar Documents

Publication Publication Date Title
CN113392733B (en) Multi-source domain self-adaptive cross-tested EEG cognitive state evaluation method based on label alignment
Dissanayake et al. Deep learning for patient-independent epileptic seizure prediction using scalp EEG signals
CN112244873A (en) Electroencephalogram time-space feature learning and emotion classification method based on hybrid neural network
CN112580518B (en) Cross-test EEG cognitive state recognition method based on prototype clustering domain adaptation algorithm
CN110353673B (en) Electroencephalogram channel selection method based on standard mutual information
CN114533086B (en) Motor imagery brain electrolysis code method based on airspace characteristic time-frequency transformation
CN114176607B (en) Electroencephalogram signal classification method based on vision transducer
Zeng et al. GRP-DNet: A gray recurrence plot-based densely connected convolutional network for classification of epileptiform EEG
Al-Saegh et al. CutCat: An augmentation method for EEG classification
CN112749635A (en) Cross-tested EEG cognitive state identification method based on prototype clustering domain adaptive algorithm
Khalighi et al. Adaptive automatic sleep stage classification under covariate shift
Wen et al. A deep learning-based classification method for different frequency EEG data
Lian et al. The improved ELM algorithms optimized by bionic WOA for EEG classification of brain computer interface
Nie et al. Recsleepnet: An automatic sleep staging model based on feature reconstruction
Jiang et al. Analytical comparison of two emotion classification models based on convolutional neural networks
Alsuwaiket Feature Extraction of EEG Signals for Seizure Detection Using Machine Learning Algorthims
Li et al. MVF-sleepnet: Multi-view fusion network for sleep stage classification
Chang et al. Dssnet: A deep sequential sleep network for self-supervised representation learning based on single-channel eeg
Zan et al. Local Pattern Transformation-Based convolutional neural network for sleep stage scoring
CN116821764A (en) Knowledge distillation-based multi-source domain adaptive EEG emotion state classification method
CN113842151B (en) Cross-test EEG cognitive state detection method based on efficient multi-source capsule network
Rani et al. Effective Epileptic Seizure Detection Using Enhanced Salp Swarm Algorithm-based Long Short-Term Memory Network
Boernama et al. Multiclass classification of brain-computer interface motor imagery system: a systematic literature review
CN113177482A (en) Cross-individual electroencephalogram signal classification method based on minimum category confusion
CN116616800B (en) Scalp electroencephalogram high-frequency oscillation signal identification method and device based on meta-shift learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant