CN114886383A - Electroencephalogram signal emotional feature classification method based on transfer learning - Google Patents
Electroencephalogram signal emotional feature classification method based on transfer learning Download PDFInfo
- Publication number
- CN114886383A CN114886383A CN202210526697.0A CN202210526697A CN114886383A CN 114886383 A CN114886383 A CN 114886383A CN 202210526697 A CN202210526697 A CN 202210526697A CN 114886383 A CN114886383 A CN 114886383A
- Authority
- CN
- China
- Prior art keywords
- loss
- classification
- electroencephalogram signal
- electroencephalogram
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 68
- 238000013526 transfer learning Methods 0.000 title claims abstract description 35
- 230000002996 emotional effect Effects 0.000 title claims abstract description 34
- 230000008451 emotion Effects 0.000 claims abstract description 72
- 238000013145 classification model Methods 0.000 claims abstract description 40
- 238000012549 training Methods 0.000 claims description 73
- 239000011159 matrix material Substances 0.000 claims description 51
- 238000005259 measurement Methods 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 13
- 230000003044 adaptive effect Effects 0.000 claims description 10
- 238000000605 extraction Methods 0.000 claims description 10
- 230000007704 transition Effects 0.000 claims description 10
- 230000006978 adaptation Effects 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 7
- 230000008909 emotion recognition Effects 0.000 claims description 6
- 238000012546 transfer Methods 0.000 claims description 6
- 238000009827 uniform distribution Methods 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 4
- 230000003042 antagnostic effect Effects 0.000 claims description 2
- 238000009826 distribution Methods 0.000 abstract description 11
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 230000000694 effects Effects 0.000 abstract description 2
- 230000003993 interaction Effects 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 9
- 238000004590 computer program Methods 0.000 description 7
- 239000005557 antagonist Substances 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000013508 migration Methods 0.000 description 4
- 230000005012 migration Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 230000036651 mood Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000007935 neutral effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/08—Feature extraction
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/369—Electroencephalography [EEG]
- A61B5/372—Analysis of electroencephalograms
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/12—Classification; Matching
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Pathology (AREA)
- Surgery (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Veterinary Medicine (AREA)
- Public Health (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Animal Behavior & Ethology (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Psychology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Signal Processing (AREA)
- Physiology (AREA)
- Fuzzy Systems (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computing Systems (AREA)
- Child & Adolescent Psychology (AREA)
- Developmental Disabilities (AREA)
- Educational Technology (AREA)
- Hospice & Palliative Care (AREA)
- Social Psychology (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention belongs to the field of medical artificial intelligence and emotional brain-computer interaction, and particularly relates to a method, a system and equipment for classifying electroencephalogram emotional characteristics based on transfer learning, aiming at solving the problem that the accuracy of classification in cross-individual electroencephalogram emotional classification caused by inconsistent distribution of electroencephalogram data of different individuals cannot be overcome by the existing electroencephalogram emotional characteristic classification method. The method comprises the following steps: acquiring emotion electroencephalogram signal data to be classified as input data; extracting differential entropy characteristics of input data, and inputting the differential entropy characteristics into a multilayer perceptron of an electroencephalogram signal emotion classification model to obtain depth characteristics; and obtaining a classification result corresponding to the input data through a classifier of the electroencephalogram signal emotion classification model based on the depth characteristics. The method not only solves the problem of inconsistent distribution of electroencephalogram data of different individuals, but also has good classification effect on cross-individual electroencephalogram emotion classification, and simultaneously ensures necessary generalization and robustness.
Description
Technical Field
The invention belongs to the field of medical artificial intelligence and emotional brain-computer interaction, and particularly relates to a method, a system and equipment for classifying electroencephalogram emotional characteristics based on transfer learning.
Background
Among the emotion recognition methods, Electroencephalogram (EEG) has significant advantages in reliability and accuracy. However, individual differences in EEG limit the generalization ability of mood classifiers between subjects. Furthermore, due to the non-stationarity of EEG, the subject's signal may change over time, which is a significant challenge for emotion recognition work across time. As can be seen from the foregoing, unlike other mainstream classification problems, EEG data distribution is different for different persons in an individual-independent problem based on EEG data, resulting in poor results of the conventional method. An important premise of conventional machine learning is that existing data and unknown data obey the same laws, statistically, obey the same distributions. How to draw close the distribution of EEG data of different people is the key for solving the problem, and therefore a transfer learning method is introduced to finish the classification of the emotion electroencephalogram signals.
Based on the method, global-local confrontation and joint domain adaptation are combined for the first time to realize the electroencephalogram emotional feature classification method based on transfer learning.
Disclosure of Invention
In order to solve the above problems in the prior art, that is, to solve the problem that the existing electroencephalogram emotional characteristic classification method based on electroencephalogram cannot overcome the problem that the classification accuracy in cross-individual electroencephalogram emotional classification is poor due to inconsistent distribution of electroencephalogram data of different individuals, the invention provides a method for classifying electroencephalogram emotional characteristics based on transfer learning in a first aspect, and the method comprises the following steps:
s100, obtaining emotion electroencephalogram signal data to be classified as input data;
s200, extracting differential entropy characteristics of the input data, and inputting the differential entropy characteristics into a multilayer perceptron of an electroencephalogram signal emotion classification model to obtain depth characteristics corresponding to the input data;
s300, obtaining a classification result corresponding to the input data through a classifier of an electroencephalogram signal emotion classification model based on the depth features;
the electroencephalogram signal emotion classification model comprises a multilayer perceptron and a classifier.
In some preferred embodiments, the electroencephalogram signal emotion classification model is trained by the following method:
a100, acquiring a first data set and a second data set; the first data set is a source domain training data set and comprises training samples and real classification labels corresponding to the training samples; the second data set is a target domain training data set; the training sample is emotion electroencephalogram signal data;
a200, respectively extracting the differential entropy characteristics D of each training sample of the first training set s And the differential entropy characteristics D of each training sample in the second training set t After extraction, inputting the differential entropy characteristics into a multilayer perceptron of the electroencephalogram signal emotion classification model to obtain the depth characteristics f of each training sample of the first training set s Depth feature f of each training sample in the second training set t ；
A300, mixing f s 、f t Inputting a pre-constructed global countermeasure module and calculating global countermeasure loss; will f is s 、f t Inputting the data into softmax of the classifier, then inputting a pre-constructed local area countermeasure module, and calculating local area countermeasure loss;
a400, combining a preset balance coefficient, carrying out balance processing on the global area countermeasure loss and the local area countermeasure loss to obtain global-local area countermeasure loss;
a500, mapping Ds and Dt to a low-dimensional space through a full connection layer, which is marked as A i And B j (ii) a Calculation of A i 、B j The similarity between the first matrix and the second matrix is calculated, and the probability of transferring from any element of the first matrix to any element of the second matrix is calculated according to the similarity
Probability of transfer from any element of the second matrix to any element of the first matrix
Then obtaining the probability of transferring from any element of the first matrix to any element of the second matrix and then transferring to any element of the first matrix
(ii) a Based on
Calculating the measurement loss with high similarity of the same type label data and the measurement loss with high similarity between a source domain and a target domain;
a represents a feature matrix of a source domain training sample mapped to a low-dimensional space, and the feature matrix is used as a first matrix, and B represents a feature matrix of a target domain training sample mapped to the low-dimensional space, and the feature matrix is used as a second matrix;
a600, carrying out weighted summation on global-local domain confrontation loss, measurement loss with large similarity of tag data of the same category and measurement loss with large similarity between a source domain and a target domain to obtain adaptive loss of a united domain;
a700, inputting the depth features into the classifier to obtain a classification result, and taking the classification result as a prediction result; calculating classification loss through a cross entropy loss function based on the prediction result and the real classification label; summing the classification loss and the joint domain adaptive loss to obtain a total loss, and further updating network parameters of an electroencephalogram signal emotion classification model;
and A800, circulating the steps A100-A700 until a trained electroencephalogram signal emotion classification model is obtained.
In some preferred embodiments, the global countermeasure module and the local countermeasure module are each a plurality of classifiers constructed by fully connected layers.
In some preferred embodiments, the global-local area countermeasure loss is calculated by:
L doamin ＝(1-ω)L g +ωL l
wherein L is doamin Representing global-local area countermeasure loss, L g Denotes global countermeasure loss, L l Representing the sum of local antagonistic losses, n s 、n t Respectively representing the number of training samples, x, of the source domain training data set and the target domain training data set i Representing training samples, d i Denotes a domain tag, G, corresponding to each domain d Denotes gradient inversion, G f Representing feature extractors, i.e. multi-layer perceptrons, L d Represents the cross entropy loss function, ω represents the equilibrium coefficient, i represents the subscript,
denotes f s Or f t The probability obtained after the softmax processing, C represents a set of class labels, C represents a specific certain class label,
the gradient inversion feature of the local region discriminator corresponding to the c category is shown,
local region discriminator for expressing c category correspondence based on gradient inversionRotary character and label
The resulting cross entropy loss.
In some preferred embodiments, the preset balance coefficient is:
wherein the content of the first and second substances,
representing local area fight loss.
In some preferred embodiments, the joint domain adaptive loss is calculated by:
L trans ＝L doamin +β 1 L walker +β 2 L visit
L walker ＝H(T，P aba )
L visit ＝(V，P visit )
wherein L is trans Denotes joint domain adaptation loss, L walker Represents the measurement loss with large similarity of the label data of the same category, L visit Represents the loss of measure, beta, with large similarity between the source domain and the target domain 1 、β 2 Representing a preset regularization coefficient, H represents a cross-entropy loss function, M ij Is represented by A i 、B j Similarity between them, class () denotes a class, T denotes P aba Labels of transition probabilities of elements of the same type in accordance with uniform distribution, y representing P ab The transition probabilities conform to the uniformly distributed labels.
In some preferred embodiments, the total loss is calculated by:
L＝L trans +L y
L y ＝H(y，A)
wherein L represents the total loss, L y Representing the classification loss, y representing the real classification label, and A representing the feature matrix after the source domain training sample is mapped to the low-dimensional space.
The second aspect of the invention provides a system for classifying electroencephalogram emotional features based on transfer learning, which comprises: the emotion recognition system comprises a data acquisition module, a feature extraction module and an emotion classification module;
the data acquisition module is configured to acquire emotion electroencephalogram signal data to be classified as input data;
the feature extraction module is configured to extract differential entropy features of the input data, and input the differential entropy features into a multilayer perceptron of an electroencephalogram signal emotion classification model to obtain depth features corresponding to the input data;
the emotion classification module is configured to obtain a classification result corresponding to the input data through a classifier of an electroencephalogram signal emotion classification model based on the depth features;
the electroencephalogram signal emotion classification model comprises a multilayer perceptron and a classifier.
In a third aspect of the present invention, an electronic device is provided, including: at least one processor; and a memory communicatively coupled to at least one of the processors; the memory stores instructions which can be executed by the processor, and the instructions are used for being executed by the processor to realize the electroencephalogram emotional feature classification method based on the transfer learning.
In a fourth aspect of the present invention, a computer-readable storage medium is provided, where computer instructions are stored in the computer-readable storage medium for being executed by the computer to implement the above-mentioned electroencephalogram signal emotion feature classification method based on transfer learning.
The invention has the beneficial effects that:
the method not only solves the problem of inconsistent distribution of electroencephalogram data of different individuals, but also has good classification effect on cross-individual electroencephalogram emotion classification, and can ensure necessary generalization and robustness.
1) The method has simple network structure and small required weight quantity, and does not need to construct private feature extractors for different individuals. The method is used for performing experiments on cross-time electroencephalogram emotion classification tasks, the accuracy can reach 89.4%, and the result proves that the method has good classification performance on various emotion recognition tasks.
2) According to the method, global differences and local differences of different individual electroencephalogram characteristics are considered at the electroencephalogram signal characteristic level, the domain confrontation method is fully utilized, meanwhile, the domain confrontation method is combined with the combined domain adaptation method and balanced to form the transfer learning module, the electroencephalogram signal characteristics of different people tend to be uniformly distributed, the emotion type characteristics are distinguished from each other, the problem that the electroencephalogram signal characteristics of different people are in different distributions can be effectively solved, and the final result has good generalization performance.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings.
FIG. 1 is a schematic flow chart of a classification method for electroencephalogram emotional features based on transfer learning according to an embodiment of the present invention;
FIG. 2 is a schematic frame diagram of a classification system for electroencephalogram emotional features based on transfer learning according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a training process of an electroencephalogram signal emotion classification model according to an embodiment of the present invention;
FIG. 4 is a detailed schematic diagram of a training process of an electroencephalogram signal emotion classification model according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a computer system suitable for implementing an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
The electroencephalogram emotional feature classification method based on transfer learning disclosed by the invention comprises the following steps of:
s100, obtaining emotion electroencephalogram signal data to be classified as input data;
s200, extracting differential entropy characteristics of the input data, and inputting the differential entropy characteristics into a multilayer perceptron of an electroencephalogram signal emotion classification model to obtain depth characteristics corresponding to the input data;
s300, obtaining a classification result corresponding to the input data through a classifier of an electroencephalogram signal emotion classification model based on the depth features;
the electroencephalogram signal emotion classification model comprises a multilayer perceptron and a classifier.
In order to more clearly explain the electroencephalogram emotional feature classification system based on transfer learning, the following describes in detail the steps of an embodiment of the method in accordance with the present invention with reference to the accompanying drawings.
In the following embodiments, the detailed description is first performed on the training process of the electroencephalogram signal emotion classification model, and then the detailed description is performed on the process of obtaining the classification result of emotion electroencephalogram signal data by the electroencephalogram signal emotion feature classification method based on transfer learning.
1. Training process of electroencephalogram signal emotion classification model
The method is used for extracting Differential Entropy (DE) characteristics aiming at emotion electroencephalogram signal data, inputting the characteristics into a multilayer perceptron and outputting depth characteristics. And (3) performing transfer learning by using the depth features, wherein the depth features are mainly input into a global-local domain confrontation module and a joint domain adaptation module to obtain transfer learning loss. And inputting the depth features into a classifier module to calculate cross entropy to obtain classification loss. The resulting migration loss summed with the classification loss is used for error back propagation as shown in fig. 3. The method comprises the following specific steps:
a100, acquiring a first data set and a second data set; the first data set is a source domain training data set and comprises training samples and real classification labels corresponding to the training samples; the second data set is a target domain training data set; the training sample is emotion electroencephalogram signal data;
a200, respectively extracting the differential entropy characteristics D of each training sample of the first training set s And a second training setDifferential entropy feature D of each training sample t After extraction, inputting the differential entropy characteristics into a multilayer perceptron of the electroencephalogram signal emotion classification model to obtain the depth characteristics f of each training sample of the first training set s Depth feature f of each training sample in the second training set t ；
In this embodiment, there are two data sets, one data set comprising ns training samples from the source domain and their corresponding true class labels y (e.g., positive, negative), referred to as the source domain training data set (i.e., training set), and one data set comprising n t A training sample (i.e., validation set) from the target domain, wherein the training sample represents x i Each domain corresponding to a domain label representing d i (belonging to either the source domain or the target domain, as in FIG. 4
All represent domain tags, c is used to distinguish categories).
Differential entropy features (DE features) D of training samples of a source domain training data set are extracted respectively s And the differential entropy characteristic D of each training sample in the target domain training data set t After extraction, inputting the differential entropy characteristics into a multilayer perceptron of the electroencephalogram signal emotion classification model to obtain the depth characteristics f of each training sample of the first training set s Depth feature f of each training sample in the second training set t As shown in fig. 4.
A300, mixing f s 、f t Inputting a pre-constructed global countermeasure module and calculating global countermeasure loss; will f is s 、f t Inputting the data into softmax of the classifier, then inputting a pre-constructed local area countermeasure module, and calculating local area countermeasure loss;
in the embodiment, fine-grained confrontation of depth characteristics of an electroencephalogram signal source domain and a target domain is realized, electroencephalogram characteristics of different domains tend to be uniformly distributed, and the obtained confrontation loss participates in calculation of migration loss. The method specifically comprises the following steps:
first, the depth features are input into a global countermeasure module to obtain global countermeasure loss, where the global countermeasure module is a discriminator (or simply, discriminator, and therefore the global countermeasure module is also called a global discriminator module, as shown in fig. 4) constructed by multiple global connection layers.
Global area countermeasure loss L g The calculation method comprises the following steps:
wherein G is f Representing feature extractors, i.e. multi-layer perceptrons, L d (G d (G f (x i ))，d i ) Representing the use of gradient inversion G for features d Method of computing domain confrontation cross entropy loss, L d Represents the cross entropy loss, G d Indicating a gradient inversion.
Then, the depth features are simultaneously input into softmax of the classifier, and then input into local domain antagonists (i.e. local domain antagonist modules, discriminators constructed for a plurality of fully-connected layers, local domain antagonist modules are also called local domain discriminator modules, as shown in fig. 4) with the same number of categories, and losses of the local domain antagonists are summed to obtain local domain antagonist loss
And sum of local area countermeasure loss L l The following formula shows:
wherein the content of the first and second substances,
denotes f s Or f t The probability obtained after softmax processing is 0, 1]，
Multiplication, embodying scaling, i.e. in fig. 4
The global area challenge loss equation does not have this term because the global area arbiter does not experience softmax or the probability is constant 1,
the gradient inversion characteristic of the local domain discriminator corresponding to the c category is shown,
local domain discriminators representing class c correspondences based on gradient inversion features and labels
The resulting cross entropy loss, C represents the set of class labels, and C represents a particular class label.
A400, combining a preset balance coefficient, carrying out balance processing on the global area countermeasure loss and the local area countermeasure loss to obtain global-local area countermeasure loss;
in this embodiment, the global-local area countermeasure loss L is obtained by balancing the global countermeasure loss and the local area countermeasure loss according to the formula (4) doamin (i.e., loss in FIG. 4) d )：
L doamin ＝(1-ω)L g +ωL l (4)
Where ω is an equilibrium coefficient calculated by equation (5):
a500, mapping Ds and Dt to a low-dimensional space through a full connection layer, which is marked as A i And B j (ii) a Calculation of A i 、B j The similarity between the first matrix and the second matrix is calculated, and the probability of transferring from any element of the first matrix to any element of the second matrix is calculated according to the similarity
Probability of transfer from any element of the second matrix to any element of the first matrix
Then, the probability of transferring any element of the first matrix to any element of the second matrix and then transferring any element of the first matrix is obtained
Based on
Calculating the measurement loss with high similarity of the same type label data and the measurement loss with high similarity between a source domain and a target domain;
although the domains in which the electroencephalogram signals are located are different, the inherent properties (emotion classes) of different individuals are the same, and it is a reasonable assumption that the characteristics of different individuals are similar. According to the method, data from a source domain and a target domain can be mapped to a low-dimensional space to extract the similarity of different domain samples.
In the embodiment, a joint domain adaptation method is adopted to measure the recent expectation of the distribution of the data of the source domain and the data of the target domain, and the obtained loss participates in the calculation of the migration loss. The method comprises the following specific steps:
for the combined domain adaptation module aiming at the electroencephalogram emotional characteristics of the electroencephalogram signal source domain and the target domain, D is converted into a full connection layer s And D t Mapping to a low dimensional space, denoted A i And B j The corresponding characteristic sample is recorded as
And
by inner product M ij To identify the source domain A i And a target domain B j To a similar degree. Then sample
And
the probability of a transition from any element of the first matrix to any element of the second matrix can be expressed by:
wherein A represents a characteristic matrix of the source domain training sample after being mapped to the low-dimensional space, the characteristic matrix is taken as a first matrix, and the dimensionality is n s D, B represents a feature matrix after the target domain training sample is mapped to the low-dimensional space and serves as a second matrix, and the dimension is n t D, d represents the dimension of the feature of the current training sample;
the similarity between the element i in A and the element j in B is shown, and the similarity itself does not have the direction, but the similarity is understood as i->Probability of transition of j.
The probability that a target domain sample is associated to a sample of the source domain, i.e. the probability of a transition from any element of the second matrix to any element of the first matrix, is similarly defined:
can be understood as j->i probability of transition (actually covariance matrix)
Transpose of (c).
The invention contemplates that the label is not changed during the transfer process, for example: positive, neutral, negative mood. The target domain has no category label at this time, but the source domain data has, so that the measurement of the similarity between the source domain and the target domain is realized, and the category label of the source domain is utilized. The result of the desired transfer is a uniform distribution, i.e. each element in the source domain A passes through A->B->The probability after A transfer is the same, T represents P aba The transition probability of the elements of the same type is consistent with the label under the condition of uniform distribution, namely the elements of the same type are similar as much as possible, and the elements of different types are dissimilar, so that the data distribution in the same type is drawn. I.e. this cycle P of joint domain adaptation to the desired association metric aba Should be a uniform distribution T, it can thus be calculated by the cross entropy H, as shown in equation (8):
L walker ＝H(T，P aba ) (8)
wherein L is walker And the loss of the same category of label data with high similarity is represented.
Furthermore, to discover more similarities of the source domain to the target domain, L is introduced visit To make the two domains perform the association measurement, the target domain is covered as much as possible.
L visit ＝(V，P visit ) (10)
Wherein L is visit Represents the loss of the measure with large similarity between the source domain and the target domain, and V represents P ab The transition probabilities conform to the uniformly distributed labels.
A600, carrying out weighted summation on global-local domain confrontation loss, measurement loss with high similarity of tag data of the same type and measurement loss with high similarity between a source domain and a target domain to obtain joint domain adaptive loss;
in the present embodiment, L is determined by walker 、L visit The two combinations are balanced to achieve the goal of zooming in the source domain and the target domain and the similar samples keep higher similarity, namely the loss of the adaptive part of the joint domain (the adaptive loss L of the joint domain) trans Also known as migration loss, as shown in fig. 3), can be represented by equation (11), where β is a fixed regularization term.
L tranns ＝L doamin +β 1 L walker +β 2 L visit (12)
Wherein, beta 1 、β 2 Representing a preset regularization coefficient, L trans Loss in FIG. 4 a 。
A700, inputting the depth features into the classifier to obtain a classification result, and taking the classification result as a prediction result; calculating classification loss through a cross entropy loss function based on the prediction result and the real classification label; summing the classification loss and the joint domain adaptive loss to obtain a total loss, and further updating network parameters of the electroencephalogram signal emotion classification model;
in this embodiment, the classifier is a full-connection layer classifier, which includes a full-connection layer (i.e. f in fig. 4), a classifier (e.g. softmax).
Total loss L in the transfer learning part trans After the calculation is finished, combining the class label y of the source domain, calculating the classification loss L aiming at the classifier module of the depth feature y (i.e., loss in FIG. 4) y ) And further, obtaining the total loss (namely total loss) L of the whole network:
L y ＝H(y，A) (13)
L＝L trans +L y (14)
and A800, circulating the steps A100-A700 until a trained electroencephalogram signal emotion classification model is obtained.
In this embodiment, the total loss L is used to perform back propagation optimization on the network, and finally, a trained electroencephalogram signal emotion classification model is iterated.
2. Electroencephalogram signal emotional feature classification method based on transfer learning
S100, obtaining emotion electroencephalogram signal data to be classified as input data;
in this embodiment, the emotion electroencephalogram signal data to be classified is acquired first.
S200, extracting differential entropy characteristics of the input data, and inputting the differential entropy characteristics into a multilayer perceptron of an electroencephalogram signal emotion classification model to obtain depth characteristics corresponding to the input data;
and S300, obtaining a classification result corresponding to the input data through a classifier of an electroencephalogram signal emotion classification model based on the depth features.
In the embodiment, the differential entropy characteristics of the emotion electroencephalogram signal data are extracted, and the classification result corresponding to the emotion electroencephalogram signal data is obtained through a multilayer perceptron and a classifier of the electroencephalogram signal emotion classification model.
The invention combines global-local confrontation and joint domain adaptation for the first time to realize a method for classifying electroencephalogram emotional characteristics based on transfer learning. A deep neural network is used to extract emotional features of the EEG data. To take into account inter-individual variability, a countermeasure approach is proposed to reduce the distance of EEG data fields between subjects, to obtain better results, while taking into account global and local countermeasures between fields, and adjusting the ratio of the two by a balancing factor to approximately adjust the joint distribution. Further addressing the non-stationarity of EEG, the challenge is that the subject's signal will vary over time.
A classification system for electroencephalogram emotional features based on transfer learning according to a second embodiment of the present invention, as shown in fig. 2, includes: the emotion recognition system comprises a data acquisition module 100, a feature extraction module 200 and an emotion classification module 300;
the data acquisition module 100 is configured to acquire emotion electroencephalogram signal data to be classified as input data;
the feature extraction module 200 is configured to extract differential entropy features of the input data, and input the differential entropy features into a multi-layer perceptron of an electroencephalogram signal emotion classification model to obtain depth features corresponding to the input data;
the emotion classification module 300 is configured to obtain a classification result corresponding to the input data through a classifier of an electroencephalogram signal emotion classification model based on the depth features;
the electroencephalogram signal emotion classification model comprises a multilayer perceptron and a classifier.
It should be noted that, the electroencephalogram emotional feature classification system based on transfer learning provided in the foregoing embodiment is only illustrated by the division of the above functional modules, and in practical applications, the functions may be allocated to different functional modules according to needs, that is, the modules or steps in the embodiment of the present invention are further decomposed or combined, for example, the modules in the foregoing embodiment may be combined into one module, or may be further split into multiple sub-modules, so as to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not to be construed as unduly limiting the present invention.
An electronic apparatus according to a third embodiment of the present invention includes: at least one processor; and a memory communicatively coupled to at least one of the processors; the memory stores instructions which can be executed by the processor, and the instructions are used for being executed by the processor to realize the electroencephalogram emotional feature classification method based on the transfer learning.
A computer-readable storage medium of a fourth embodiment of the present invention stores computer instructions for being executed by the computer to implement the above-mentioned electroencephalogram signal emotion feature classification method based on transfer learning.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes and related descriptions of the storage device and the processing device described above may refer to the corresponding processes in the foregoing method examples, and are not described herein again.
Referring now to FIG. 5, there is illustrated a block diagram of a computer system suitable for use as a server in implementing embodiments of the method, system, and apparatus of the present application. The server shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 5, the computer system includes a Central Processing Unit (CPU) 501 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM503, various programs and data necessary for system operation are also stored. The CPU501, ROM 502, and RAM503 are connected to each other via a bus 504. An Input/Output (I/O) interface 505 is also connected to bus 504.
The following components are connected to the I/O interface 505: an input portion 506 including a keyboard, a mouse, and the like; an output section 507 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage portion 508 including a hard disk and the like; and a communication section 509 including a Network interface card such as a LAN (Local Area Network) card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The driver 510 is also connected to the I/O interface 505 as necessary. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as necessary, so that a computer program read out therefrom is mounted on the storage section 508 as necessary.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 509, and/or installed from the removable medium 511. More specific examples of a computer-readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), a compact disc read-only memory (CD-ROM), Optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing or implying a particular order or sequence.
The terms "comprises," "comprising," or any other similar term are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
Claims (10)
1. A classification method aiming at electroencephalogram emotional features based on transfer learning is characterized by comprising the following steps:
s100, obtaining emotion electroencephalogram signal data to be classified as input data;
s200, extracting differential entropy characteristics of the input data, and inputting the differential entropy characteristics into a multilayer perceptron of an electroencephalogram signal emotion classification model to obtain depth characteristics corresponding to the input data;
s300, obtaining a classification result corresponding to the input data through a classifier of an electroencephalogram signal emotion classification model based on the depth features;
the electroencephalogram signal emotion classification model comprises a multilayer perceptron and a classifier.
2. The electroencephalogram signal emotion feature classification method based on transfer learning as claimed in claim 1, wherein the electroencephalogram signal emotion classification model is trained by the following method:
a100, acquiring a first data set and a second data set; the first data set is a source domain training data set and comprises training samples and real classification labels corresponding to the training samples; the second data set is a target domain training data set; the training sample is emotion electroencephalogram signal data;
a200, respectively extracting the differential entropy characteristics D of each training sample of the first training set s And the differential entropy characteristics D of each training sample in the second training set t After extraction, inputting the differential entropy characteristics into a multilayer perceptron of the electroencephalogram signal emotion classification model to obtain the depth characteristics f of each training sample of the first training set s Depth feature f of each training sample in the second training set t ；
A300, mixing f s 、f t Inputting a pre-constructed global countermeasure module and calculating global countermeasure loss; will f is s 、f t Inputting the softmax of the classifier, inputting a pre-constructed local area countermeasure module, and calculating local area countermeasure loss;
a400, combining a preset balance coefficient, carrying out balance processing on the global area countermeasure loss and the local area countermeasure loss to obtain global-local area countermeasure loss;
a500, connecting D with D through a full connection layer s 、D t Mapping to a lower dimensional space, denoted A i And B j (ii) a Calculation of A i 、B j The similarity between the first matrix and the second matrix is calculated, and the probability of transferring from any element of the first matrix to any element of the second matrix is calculated according to the similarity
Probability of transfer from any element of the second matrix to any element of the first matrix
Then obtaining the probability of transferring from any element of the first matrix to any element of the second matrix and then transferring to any element of the first matrix
Based on
Calculating the measurement loss, the source domain and the target domain with large similarity of the label data of the same categoryThe measurement loss with large inter-similarity;
a represents a feature matrix of a source domain training sample mapped to a low-dimensional space, and the feature matrix is used as a first matrix, and B represents a feature matrix of a target domain training sample mapped to the low-dimensional space, and the feature matrix is used as a second matrix;
a600, carrying out weighted summation on global-local domain confrontation loss, measurement loss with high similarity of tag data of the same type and measurement loss with high similarity between a source domain and a target domain to obtain joint domain adaptive loss;
a700, inputting the depth features into the classifier to obtain a classification result, and taking the classification result as a prediction result; calculating classification loss through a cross entropy loss function based on the prediction result and the real classification label; summing the classification loss and the joint domain adaptive loss to obtain a total loss, and further updating network parameters of the electroencephalogram signal emotion classification model;
and A800, circulating the steps A100-A700 until a trained electroencephalogram signal emotion classification model is obtained.
3. The electroencephalogram emotional feature classification method based on the transfer learning as claimed in claim 2, wherein the global countermeasure module and the local countermeasure module are classifiers constructed by a plurality of full connection layers.
4. The electroencephalogram emotional feature classification method based on the transfer learning as claimed in claim 2, wherein the global-local area confrontation loss is calculated by:
L doamin ＝(1-ω)L g +ωL l
wherein L is doamin Representing global-local area countermeasure loss, L g Denotes global countermeasure loss, L l Representing the sum of local antagonistic losses, n s 、n t Respectively representing the number of training samples, x, of the source domain training data set and the target domain training data set i Representing training samples, d i Denotes a domain tag, G, corresponding to each domain d Denotes gradient inversion, G f Representing feature extractors, i.e. multi-layer perceptrons, L d Represents the cross entropy loss function, ω represents the equilibrium coefficient, i represents the subscript,
denotes f s Or f t The probability obtained after the softmax processing is that C represents a set of class labels, C represents a specific class label,
the gradient inversion feature of the local region discriminator corresponding to the c category is shown,
local region discriminators corresponding to the class c representation based on gradient inversion features and labels
The resulting cross entropy loss.
5. The electroencephalogram signal emotional feature classification method based on the transfer learning as claimed in claim 4, wherein the preset balance coefficients are:
wherein, the first and the second end of the pipe are connected with each other,
representing local area fight loss.
6. The electroencephalogram signal emotional feature classification method based on the transfer learning as claimed in claim 4, wherein the joint domain adaptive loss is calculated by:
L trans ＝L doamin +β 1 L walker +β 2 L visit
L walker ＝H(T，P aba )
L visti ＝(V，P visit )
wherein L is trans Denotes joint domain adaptation loss, L walker Represents the measurement loss with large similarity of the label data of the same category, L visit Representing source and destinationLoss of measure, beta, with large inter-domain similarity 1 、β 2 Representing a preset regularization coefficient, H represents a cross-entropy loss function, M ij Is represented by A i 、B j Similarity between them, class () denotes a class, T denotes P aba The transition probability of the elements of the same type is consistent with the label under the condition of uniform distribution, V represents P ab The transition probabilities conform to the uniformly distributed labels.
7. The electroencephalogram signal emotional feature classification method based on the transfer learning as claimed in claim 6, wherein the total loss is calculated by:
L＝L trans +L y
L y ＝H(y，A)
wherein L represents the total loss, L y Representing the classification loss, y representing the real classification label, and A representing the feature matrix of the source domain training sample after being mapped to the low-dimensional space.
8. A classification system aiming at electroencephalogram emotional features based on transfer learning is characterized by comprising: the emotion recognition system comprises a data acquisition module, a feature extraction module and an emotion classification module;
the data acquisition module is configured to acquire emotion electroencephalogram signal data to be classified as input data;
the feature extraction module is configured to extract differential entropy features of the input data, and input the differential entropy features into a multilayer perceptron of an electroencephalogram signal emotion classification model to obtain depth features corresponding to the input data;
the emotion classification module is configured to obtain a classification result corresponding to the input data through a classifier of an electroencephalogram signal emotion classification model based on the depth features;
the electroencephalogram signal emotion classification model comprises a multilayer perceptron and a classifier.
9. An electronic device, comprising:
at least one processor; and a memory communicatively coupled to at least one of the processors;
wherein the memory stores instructions executable by the processor for implementing the method for classifying electroencephalogram signal emotional features based on transfer learning of any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions for being executed by the computer to implement the method for classifying electroencephalogram signal emotional features based on transfer learning according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210526697.0A CN114886383A (en) | 2022-05-16 | 2022-05-16 | Electroencephalogram signal emotional feature classification method based on transfer learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210526697.0A CN114886383A (en) | 2022-05-16 | 2022-05-16 | Electroencephalogram signal emotional feature classification method based on transfer learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114886383A true CN114886383A (en) | 2022-08-12 |
Family
ID=82723482
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210526697.0A Pending CN114886383A (en) | 2022-05-16 | 2022-05-16 | Electroencephalogram signal emotional feature classification method based on transfer learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114886383A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117708682A (en) * | 2024-02-06 | 2024-03-15 | 吉林大学 | Intelligent brain wave acquisition and analysis system and method |
-
2022
- 2022-05-16 CN CN202210526697.0A patent/CN114886383A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117708682A (en) * | 2024-02-06 | 2024-03-15 | 吉林大学 | Intelligent brain wave acquisition and analysis system and method |
| CN117708682B (en) * | 2024-02-06 | 2024-04-19 | 吉林大学 | Intelligent brain wave acquisition and analysis system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Klibisz et al. | Fast, simple calcium imaging segmentation with fully convolutional networks | |
| CN111738172B (en) | Cross-domain target re-identification method based on feature counterstudy and self-similarity clustering | |
| CN113705772A (en) | Model training method, device and equipment and readable storage medium | |
| Ma et al. | A deep-learning based citation count prediction model with paper metadata semantic features | |
| CN113539408B (en) | Medical report generation method, training device and training equipment of model | |
| Meng et al. | Few-shot image classification algorithm based on attention mechanism and weight fusion | |
| CN110110724A (en) | The text authentication code recognition methods of function drive capsule neural network is squeezed based on exponential type | |
| CN116010586A (en) | Method, device, equipment and storage medium for generating health advice | |
| Karthiga et al. | Feature selection using multi-objective modified genetic algorithm in multimodal biometric system | |
| Rath et al. | Analysis of deep learning methods for detection of bird species | |
| Mahapatra et al. | MRMR-SSA: a hybrid approach for optimal feature selection | |
| CN114886383A (en) | Electroencephalogram signal emotional feature classification method based on transfer learning | |
| Parthasarathy et al. | Computer aided diagnosis using Harris Hawks optimizer with deep learning for pneumonia detection on chest X-ray images | |
| Fang et al. | Neural network-based automatic factor construction | |
| Kalaiselvi et al. | E-Tanh: a novel activation function for image processing neural network models | |
| Zhang et al. | Efficient history matching with dimensionality reduction methods for reservoir simulations | |
| Liu et al. | Evidence fusion theory in healthcare | |
| Liang et al. | Accurate image recognition of plant diseases based on multiple classifiers integration | |
| CN114121296B (en) | Data-driven clinical information rule extraction method, storage medium and equipment | |
| CN116089708A (en) | Agricultural knowledge recommendation method and device | |
| CN111221880B (en) | Feature combination method, device, medium, and electronic apparatus | |
| Ji et al. | A recognition method for Italian alphabet gestures based on convolutional neural network | |
| Yu et al. | Pyramidal and conditional convolution attention network for hyperspectral image classification using limited training samples | |
| CN114897057A (en) | Intelligent yin deficiency syndrome diagnosis system, method and equipment based on multi-mode data fusion | |
| Ramalakshmi et al. | Enhanced prediction using deep neural network-based image classification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |