CN116049639B - Selective migration learning method and device for electroencephalogram signals and storage medium - Google Patents

Selective migration learning method and device for electroencephalogram signals and storage medium Download PDF

Info

Publication number
CN116049639B
CN116049639B CN202310334701.8A CN202310334701A CN116049639B CN 116049639 B CN116049639 B CN 116049639B CN 202310334701 A CN202310334701 A CN 202310334701A CN 116049639 B CN116049639 B CN 116049639B
Authority
CN
China
Prior art keywords
matrix
domains
riemann
source
domain
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310334701.8A
Other languages
Chinese (zh)
Other versions
CN116049639A (en
Inventor
刘伟奇
马学升
陈金钢
左林雄
王肖玮
陈韵如
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tongxin Zhiyi Technology Beijing Co ltd
Original Assignee
Tongxin Zhiyi Technology Beijing Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tongxin Zhiyi Technology Beijing Co ltd filed Critical Tongxin Zhiyi Technology Beijing Co ltd
Priority to CN202310334701.8A priority Critical patent/CN116049639B/en
Publication of CN116049639A publication Critical patent/CN116049639A/en
Application granted granted Critical
Publication of CN116049639B publication Critical patent/CN116049639B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/24Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316Modalities, i.e. specific diagnostic methods
    • A61B5/369Electroencephalography [EEG]
    • A61B5/372Analysis of electroencephalograms
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/24Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316Modalities, i.e. specific diagnostic methods
    • A61B5/369Electroencephalography [EEG]
    • A61B5/372Analysis of electroencephalograms
    • A61B5/374Detecting the frequency distribution of signals, e.g. detecting delta, theta, alpha, beta or gamma waves
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235Details of waveform analysis
    • A61B5/7264Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00Reducing energy consumption in communication networks
    • Y02D30/70Reducing energy consumption in communication networks in wireless communication networks

Abstract

The embodiment of the application discloses a selective migration learning method and device of an electroencephalogram signal and a storage medium, wherein the selective migration learning method of the electroencephalogram signal comprises the following steps: preprocessing the acquired electroencephalogram signals of all source domains and target domains by common average references, and reserving electroencephalogram signals of selected channels in the source domains and the target domains; affine transformation is carried out on the covariance matrix by using the Riemann mean value of the covariance matrix of each preprocessed source domain and target domain as a reference matrix to obtain a pair Ji Xie variance matrix of each preprocessed source domain and target domain; using the Riemann mean value of the covariance matrix as a reference matrix, and respectively converting the corresponding pair Ji Xie variance matrix into corresponding tangential space vectors; and using a forward sequence floating search method to maximize the mobility of the selected tangent space vector set and obtain a final selected tangent space vector set so as to use the corresponding selected good source domain for classifier training.

Description

Selective migration learning method and device for electroencephalogram signals and storage medium
Technical Field
The application relates to the technical field of medical image processing, in particular to a selective migration learning method and device of electroencephalogram signals and a storage medium.
Background
A brain-computer interface (BCI) based on Motor Image (MI) can help a subject directly manipulate electronic devices using imagination-induced brain activities, and has potential application value in neurological rehabilitation training of stroke patients. Electroencephalogram (EEG) is an effective way to record MI signals due to its non-invasive operation and real-time transmission characteristics, which is critical for real-time control of BCI. However, due to the weak and unstable nature of the EEG signals, it is very susceptible to noise interference, making MI-based EEG signal analysis quite challenging. Furthermore, MI-based EEG signal features are less pronounced than other traditional EEG (such as event-related potentials and steady-state visual evoked potentials) because they are evoked by spontaneous motor imagery without external stimuli. Thus, MI-based EEG signals have a higher variability, which requires a lengthy classifier calibration time for each subject before performing real-time BCI tasks, which places great restrictions on the popularization and application of BCI.
Disclosure of Invention
An objective of the embodiments of the present application is to provide a selective migration learning method, apparatus and storage medium for electroencephalogram signals, which are used for solving the problem that in the prior art, since the EEG signals based on MI have higher variability, a lot of time is required to perform calibration when collecting a lot of labeled samples specific to a model of a subject.
In order to achieve the above object, an embodiment of the present application provides a selective migration learning method for electroencephalogram signals, including: preprocessing the acquired electroencephalogram signals of all source domains and target domains by common average references, and reserving electroencephalogram signals of selected channels in the source domains and the target domains;
affine transformation is carried out on the covariance matrix by using the Riemann mean value of the covariance matrix of each preprocessed source domain and target domain as a reference matrix to obtain a pair Ji Xie variance matrix of each preprocessed source domain and target domain;
using the Riemann mean value of the covariance matrix as a reference matrix, and respectively converting the corresponding pair Ji Xie variance matrix into corresponding tangential space vectors;
and using a forward sequence floating search method to maximize the mobility of the selected tangent space vector set and obtain a final selected tangent space vector set so as to use the corresponding selected good source domain for classifier training.
Optionally, the preprocessing the acquired electroencephalogram signals of all source domains and target domains by the common average reference includes:
and carrying out spectrum filtering on the acquired electroencephalogram signals of all the source domains and the target domains through a third-order Butterworth filter, and carrying out time segmentation after the instruction starts, so as to obtain the selected channel.
Optionally, the using the Riemann mean of the covariance matrix of each of the preprocessed source domain and target domain as a reference matrix includes:
and forming a Riemann manifold by using a symmetrical positive definite matrix corresponding to the covariance matrix, and then calculating to obtain a Riemann average value serving as a reference matrix.
Optionally, the Riemann mean is calculated using the Riemann distance:
Figure SMS_1
wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure SMS_2
representing the number of all symmetric positive definite matrices on the Riemann manifold, < >>
Figure SMS_3
Representing the Riemann mean.
Optionally, the affine transforming the covariance matrix includes:
using the formula:
Figure SMS_4
performing affine transformation,
wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure SMS_5
representing RiemannAnd (5) an average value.
Optionally, the maximizing the mobility of the selected tangential space vector set using the forward order floating search method includes:
defining the set of selected tangent space vectors
Figure SMS_6
To predict its +.>
Figure SMS_7
Namely: />
Figure SMS_8
Wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure SMS_10
is->
Figure SMS_12
Inter-class scatter matrix,/>
Figure SMS_14
Is->
Figure SMS_11
And->
Figure SMS_13
A spreading matrix between the two,
Figure SMS_15
for evaluating->
Figure SMS_16
Inter-class discernment between +.>
Figure SMS_9
For evaluating the difference between the selected source domain and target domain.
Optionally, the maximizing the mobility of the selected tangential space vector set using the forward order floating search method further comprises:
a set of labeled tangent space vectors from a good source domain selected from the source domains is iteratively selected using a forward-order floating search method to be added to or removed from the selected set of tangent space vectors to maximize the portability of the selected set of tangent space vectors.
Optionally, the training for classifier includes:
using a supervised contracted linear discriminant analysis classifier, after the selected good source domain corresponding to the final selected set of tangent space vectors is obtained, the set of labeled tangent space vectors from the selected good source and target domains are fed back into the classifier for training.
In order to achieve the above object, the present application further provides a selective migration learning device for electroencephalogram signals, including: a memory; and
a processor coupled to the memory, the processor configured to:
preprocessing the acquired electroencephalogram signals of all source domains and target domains by common average references, and reserving electroencephalogram signals of selected channels in the source domains and the target domains;
affine transformation is carried out on the covariance matrix by using the Riemann mean value of the covariance matrix of each preprocessed source domain and target domain as a reference matrix to obtain a pair Ji Xie variance matrix of each preprocessed source domain and target domain;
using the Riemann mean value of the covariance matrix as a reference matrix, and respectively converting the corresponding pair Ji Xie variance matrix into corresponding tangential space vectors;
and using a forward sequence floating search method to maximize the mobility of the selected tangent space vector set and obtain a final selected tangent space vector set so as to use the corresponding selected good source domain for classifier training.
To achieve the above object, the present application also provides a computer storage medium having stored thereon a computer program which, when executed by a machine, implements the steps of the method as described above.
The embodiment of the application has the following advantages:
the embodiment of the application provides a selective migration learning method of an electroencephalogram signal, which comprises the following steps: preprocessing the acquired electroencephalogram signals of all source domains and target domains by common average references, and reserving electroencephalogram signals of selected channels in the source domains and the target domains; affine transformation is carried out on the covariance matrix by using the Riemann mean value of the covariance matrix of each preprocessed source domain and target domain as a reference matrix to obtain a pair Ji Xie variance matrix of each preprocessed source domain and target domain; using the Riemann mean value of the covariance matrix as a reference matrix, and respectively converting the corresponding pair Ji Xie variance matrix into corresponding tangential space vectors; and using a forward sequence floating search method to maximize the mobility of the selected tangent space vector set and obtain a final selected tangent space vector set so as to use the corresponding selected good source domain for classifier training.
Through the method, a supervised TL (Transfer Learning ) algorithm based on the Riemann tangent space is used for analyzing the EEG-MI signals, and the marked samples can be transferred from different source domains to the target domain, so that the requirement for a large number of marked samples is reduced, the transferability of the source domain samples is improved, and the time required by the calibration work of the target object is shortened.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It will be apparent to those skilled in the art from this disclosure that the drawings described below are merely exemplary and that other embodiments may be derived from the drawings provided without undue effort.
Fig. 1 is a flowchart of a selective migration learning method for electroencephalogram signals according to an embodiment of the present application;
fig. 2 is a schematic diagram of a method framework of a selective migration learning method for electroencephalogram signals according to an embodiment of the present application;
fig. 3 is a schematic diagram of selected channels of a selective migration learning method for electroencephalogram signals according to an embodiment of the present disclosure;
fig. 4 is a schematic diagram of a Riemann manifold and a tangential space based on a certain point of a selective transfer learning method for electroencephalogram signals according to an embodiment of the present application;
fig. 5 is a block diagram of a selective migration learning device for electroencephalogram signals according to an embodiment of the present application.
Detailed Description
Other advantages and advantages of the present application will become apparent to those skilled in the art from the following description of specific embodiments, which is to be read in light of the present disclosure, wherein the present embodiments are described in some, but not all, of the several embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
In addition, the technical features described below in the different embodiments of the present application may be combined with each other as long as they do not collide with each other.
An embodiment of the present application provides a selective migration learning method for an electroencephalogram signal, referring to fig. 1 and fig. 2, fig. 1 is a flowchart of a selective migration learning method for an electroencephalogram signal provided in an embodiment of the present application, and fig. 2 is a method frame schematic diagram of a selective migration learning method for an electroencephalogram signal provided in an embodiment of the present application, where RA is aligned in parieman, and TSM refers to tangent space mapping. It should be understood that the method may also include additional blocks not shown and/or that blocks shown may be omitted, the scope of the application being not limited in this respect.
At step 101, the acquired electroencephalogram signals of all source and target domains are preprocessed by a common average reference, and electroencephalogram signals of selected channels in the source and target domains are retained.
In some embodiments, the preprocessing of the acquired electroencephalogram signals of all source and target domains with a common average reference includes:
and carrying out spectrum filtering on the acquired electroencephalogram signals of all the source domains and the target domains through a third-order Butterworth filter, and carrying out time segmentation after the instruction starts, so as to obtain the selected channel.
Specifically, data preprocessing:
the EEG signals (electroencephalogram signals) of all source and target domains are preprocessed by a common average reference, the EEG signals of the selected channels are preserved, spectrally filtered by a third order butterworth filter with cut-off frequencies of 8Hz and 30Hz, and time-segmented from 0.5 seconds to 2.5 seconds after the start of the instruction. The dimension of the tangential space vector is closely related to the number of channels, with selected channels marked green as shown in fig. 3.
At step 102, affine transformation is performed on the covariance matrix using the Riemann mean of the covariance matrix of each preprocessed source domain and target domain as a reference matrix, to obtain a pair Ji Xie variance matrix of each preprocessed source domain and target domain.
In some embodiments, the using the Riemann mean of the covariance matrices of each of the preprocessed source and target domains as a reference matrix comprises:
and forming a Riemann manifold by using a symmetrical positive definite matrix corresponding to the covariance matrix, and then calculating to obtain a Riemann average value serving as a reference matrix.
In some embodiments, the Riemann average is calculated using the Riemann distance:
Figure SMS_17
wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure SMS_18
representing the number of all symmetric positive definite matrices on the Riemann manifold, < >>
Figure SMS_19
Representing the Riemann mean.
In some embodiments, the affine transforming the covariance matrix comprises:
using the formula:
Figure SMS_20
performing affine transformation,
wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure SMS_21
representing the Riemann mean.
Specifically, riemann alignment is performed:
in MI-based BCI, supervised co-spatial mode (common spatial patterns, CSP) is the most commonly used feature extraction algorithm for a single subject. Covariance matrices from different classes are used to learn an optimal spatial filter that maximizes the variance difference between the two classes. Such spatial filters may be used to extract low-dimensional features of marked and unmarked samples. If the marked samples are noisy, the covariance matrix may generate an unstable spatial filter. In euclidean space, the covariance matrix is processed in the manner described above. However, its distribution is not Euclidean space, but rather a smooth Riemann manifold belonging to a symmetric forward (symmetric positive definite, SPD) matrix. Thus, the covariance matrix in the pariman manifold needs to be processed first. All SPD matrices may form a Riemann manifold. Is provided with
Figure SMS_22
And->
Figure SMS_26
Respectively +.>
Figure SMS_30
SPD matrix and->
Figure SMS_25
A SPD matrix, wherein->
Figure SMS_28
,/>
Figure SMS_32
Is the number of channels. Their manifold is->
Figure SMS_34
And (3) dimension. />
Figure SMS_23
And->
Figure SMS_27
Can be considered as points of a manifold. />
Figure SMS_31
And->
Figure SMS_33
Distance between Riemann->
Figure SMS_24
,/>
Figure SMS_29
Is the length of the minimum curve connecting them, and can be calculated by the following formula:
Figure SMS_35
wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure SMS_36
representing the Frobenius norm, +.>
Figure SMS_37
Is->
Figure SMS_38
Is a characteristic value of (a). The Riemann distance has the following important characteristics:
Figure SMS_39
,/>
Figure SMS_40
,/>
Figure SMS_41
Figure SMS_42
,/>
Figure SMS_43
,/>
Figure SMS_44
Figure SMS_45
,/>
Figure SMS_46
,/>
Figure SMS_47
,/>
Figure SMS_48
the third property, called congruence invariance, is critical to the environment in which the EEG signal is processed, meaning that the distance between the two SPD matrices is unchanged after affine transformation using the invertible matrices. Assume that
Figure SMS_49
All +.>
Figure SMS_50
A set of invertible matrices. The center of all SPD matrices is used for affine transformation and may be calculated using the riman distance.
Riemann mean value
Figure SMS_51
I.e. the geometric mean, is the center point of the manifold, and the Riemann distance can be used to calculate as follows:
Figure SMS_52
wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure SMS_53
representing the number of all SPD matrices on the manifold. There is no closed form method to calculate the Riemann mean. The iterative method can effectively obtain the Riemann mean value, and affine transformation is performed by using the centers of all SPD matrices as reference matrices. Riemann alignment uses Riemann mean +.>
Figure SMS_54
As a reference matrix and performing the following affine transformation:
Figure SMS_55
riemann alignment is performed on all covariance matrices of each source and target domain, and after affine transformation is performed using the Riemann mean of each domain (including source and target domains) as a reference matrix, all pairs Ji Xie of variance matrices from each domain are centered on an identity matrix. This feature makes the pair Ji Xie variance matrices from different domains comparable, thus initially reducing inter-domain differences.
At step 103, the corresponding pair Ji Xie of variance matrices are each converted to a respective tangential space vector using the Riemann mean of the covariance matrices as a reference matrix.
Specifically, tangential spatial mapping:
after performing the Riemann alignment, a tangential space mapping (tangent space mapping, TSM) is performed on all pairs of Ji Xie variance matrices. All pairs of Ji Xie variance matrices correspond to SPD matrices located in a pariman manifold, and their derivatives on the matrices on the manifold form a tangential space. The tangential space has the same dimensions as the manifold. The Riemann manifold and its tangential space at some point is shown in FIG. 4. As shown in figure 4 of the drawings,
Figure SMS_57
and->
Figure SMS_59
Belongs to the Riemann manifold. Based on->
Figure SMS_62
Tangential space of the dot>
Figure SMS_58
And->
Figure SMS_61
Respectively->
Figure SMS_64
And->
Figure SMS_65
Is a derivative of (a). />
Figure SMS_56
Can be regarded as->
Figure SMS_60
At->
Figure SMS_63
Logarithmic mapping of points as follows:
Figure SMS_66
in contrast to this,
Figure SMS_67
at->
Figure SMS_68
The exponential mapping of the dots is SPD matrix +.>
Figure SMS_69
The definition is:
Figure SMS_70
the Riemann distance may also be defined as:
Figure SMS_71
Figure SMS_72
wherein the method comprises the steps of
Figure SMS_75
Is a dot->
Figure SMS_79
In tangential space->
Figure SMS_83
Is (are) norms of->
Figure SMS_76
Vectorized symmetry matrix->
Figure SMS_80
. Let->
Figure SMS_84
And->
Figure SMS_87
Respectively->
Figure SMS_73
. Use->
Figure SMS_78
Coefficients, and will be modified +>
Figure SMS_82
Is converted into +.>
Figure SMS_86
Column vector->
Figure SMS_74
,/>
Figure SMS_77
Namely, point->
Figure SMS_81
SPD matrix>
Figure SMS_85
Is defined in the specification. The distance between the manifold and its tangential space is approximately as follows:
Figure SMS_88
,/>
Figure SMS_89
wherein the method comprises the steps of
Figure SMS_91
And->
Figure SMS_94
Locally distributed into the manifold. They are at the spot->
Figure SMS_96
The tangent space vectors at the positions are respectively +.>
Figure SMS_92
And->
Figure SMS_95
. Only when
Figure SMS_97
This approximation holds true when it is the Riemann mean of the manifold. The Riemann tangent space therefore belongs to Euclidean and is locally homomorphic to the Riemann manifold. In the tangential space mapping phase, +.>
Figure SMS_98
As reference matrix, all aligned covariance matrices
Figure SMS_90
Is converted into corresponding tangential space vector
Figure SMS_93
At step 104, the forward sequential floating search method is used to maximize the mobility of the selected set of tangent space vectors, resulting in a final selected set of tangent space vectors, to use the corresponding selected good source domain for classifier training.
In some embodiments, the maximizing the portability of the selected tangential space vector set using a forward sequential floating search method comprises:
defining the set of selected tangent space vectors
Figure SMS_99
To predict its +.>
Figure SMS_100
Namely: />
Figure SMS_101
Wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure SMS_104
is->
Figure SMS_106
Inter-class scatter matrix,/>
Figure SMS_108
Is->
Figure SMS_103
And->
Figure SMS_105
A spreading matrix between the two,
Figure SMS_107
for evaluating->
Figure SMS_109
Inter-class discernment between +.>
Figure SMS_102
For evaluating the selection between the source domain and the target domainIs a difference in (a) between the two.
In some embodiments, the maximizing the portability of the selected tangential space vector set using the forward sequential floating search method further comprises:
a set of labeled tangent space vectors from a good source domain selected from the source domains is iteratively selected using a forward-order floating search method to be added to or removed from the selected set of tangent space vectors to maximize the portability of the selected set of tangent space vectors.
Specifically, good source domain selection:
features from all source domains are not suitable to be exploited due to the high variability between domains and the expensive computational burden. Only the marked tangent space vector is migrated from the good source domain. Defining a set of selected tangent space vectors
Figure SMS_110
To predict the mobility (transferability) of the target domain to the tangential space vector set from the target domain>
Figure SMS_111
Namely:
Figure SMS_112
wherein the method comprises the steps of
Figure SMS_114
Is->
Figure SMS_116
Is a matrix of inter-class dispersion. Similarly, a->
Figure SMS_118
Is->
Figure SMS_115
And->
Figure SMS_117
A spreading matrix therebetween.
Figure SMS_119
For evaluating->
Figure SMS_120
Inter-class discernment between +.>
Figure SMS_113
For evaluating the difference between the selected source domain and the target domain.
A set of labeled tangent space vectors from a good source domain selected from the source domains is iteratively selected using a forward sequential floating search (Sequential forward floating search, SFFS) method to add to or remove from the selected set of tangent space vectors to maximize its portability. The specific algorithm is as follows:
input: from the target domain
Figure SMS_122
Marked tangential space vector->
Figure SMS_124
From->
Figure SMS_126
Personal good Source Domain->
Figure SMS_123
Marked tangential space vector->
Figure SMS_125
Wherein->
Figure SMS_127
Is the number of good source fields. />
Figure SMS_128
And->
Figure SMS_121
Representing the marker tangent space vectors from the target domain and the source domain, respectively.
And (3) outputting: finally selected tangent space vectorCollection set
Figure SMS_129
Initializing: initially selected set of tangent space vectors
Figure SMS_130
Its mobility->
Figure SMS_131
An initial set of residual tangent space vectors>
Figure SMS_132
Repeating;
step 1: selecting and adding a set of marker tangent space vectors from the most suitable good source domain
Figure SMS_133
Figure SMS_134
Step 2: selecting a set of marker tangent space vectors from the least suitable good source domain objects, and
Figure SMS_135
is removed from
Figure SMS_136
In some embodiments, the method for classifier training comprises:
using a supervised contracted linear discriminant analysis classifier, after the selected good source domain corresponding to the final selected set of tangent space vectors is obtained, the set of labeled tangent space vectors from the selected good source and target domains are fed back into the classifier for training.
Specifically, classifier training:
using a supervised contracted linear discriminant analysis (shrinkage linear discriminant analysis, sLDA) classifier, after SFFS-based good source domain selection, the marker tangent space vectors from the selected good source and target domains are fed back into the sLDA classifier. The method of the application is used for screening the good source domain firstly, and then the obtained selected good source domain is used for classifier training, so that the efficiency can be improved.
Through the method, a supervised TL (Transfer Learning ) algorithm based on the Riemann tangent space is used for analyzing the EEG-MI signals, and the marked samples can be transferred from different source domains to the target domain, so that the requirement for a large number of marked samples is reduced, the transferability of the source domain samples is improved, and the time required by the calibration work of the target object is shortened.
The application has the following advantages:
1. the method can simultaneously utilize the marked samples of the target domain and the source domain based on the Riemann tangent space, reduce variability among different source domains, improve the mobility of the samples, and accurately execute classification tasks under the condition that the marked samples are limited.
2. The SFFS method is adopted to select the source domain samples, the iterative algorithm can realize forward transmission and reduce the calculation cost, so that the most suitable good source domain samples are selected for the target domain, the time required by BCI calibration work can be saved, and the work efficiency is improved.
3. Based on the two characteristics, the algorithm classification accuracy of the method is 87.92% on average and is superior to that of the traditional non-migration learning algorithm because inter-domain variability is reduced and a forward iterative algorithm is used for selecting a good source domain.
Fig. 5 is a block diagram of a selective migration learning device for electroencephalogram signals according to an embodiment of the present application. The device comprises:
a memory 201; and a processor 202 connected to the memory 201, the processor 202 configured to: preprocessing the acquired electroencephalogram signals of all source domains and target domains by common average references, and reserving electroencephalogram signals of selected channels in the source domains and the target domains;
affine transformation is carried out on the covariance matrix by using the Riemann mean value of the covariance matrix of each preprocessed source domain and target domain as a reference matrix to obtain a pair Ji Xie variance matrix of each preprocessed source domain and target domain;
using the Riemann mean value of the covariance matrix as a reference matrix, and respectively converting the corresponding pair Ji Xie variance matrix into corresponding tangential space vectors;
and using a forward sequence floating search method to maximize the mobility of the selected tangent space vector set and obtain a final selected tangent space vector set so as to use the corresponding selected good source domain for classifier training.
In some embodiments, the processor 202 is further configured to: the preprocessing of the acquired electroencephalogram signals of all the source domains and the target domains through the common average reference comprises the following steps:
and carrying out spectrum filtering on the acquired electroencephalogram signals of all the source domains and the target domains through a third-order Butterworth filter, and carrying out time segmentation after the instruction starts, so as to obtain the selected channel.
In some embodiments, the processor 202 is further configured to: the using the Riemann mean of the covariance matrices of each of the preprocessed source and target domains as a reference matrix comprises:
and forming a Riemann manifold by using a symmetrical positive definite matrix corresponding to the covariance matrix, and then calculating to obtain a Riemann average value serving as a reference matrix.
In some embodiments, the processor 202 is further configured to: the Riemann average is calculated using the Riemann distance:
Figure SMS_137
wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure SMS_138
representing the number of all symmetric positive definite matrices on the Riemann manifold, < >>
Figure SMS_139
Representing LiAnd (5) a Manmean value.
In some embodiments, the processor 202 is further configured to: the affine transformation of the covariance matrix comprises:
using the formula:
Figure SMS_140
performing affine transformation,
wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure SMS_141
representing the Riemann mean.
In some embodiments, the processor 202 is further configured to: the using a forward order floating search method to maximize the portability of the selected tangential space vector set includes:
defining the set of selected tangent space vectors
Figure SMS_142
To predict its +.>
Figure SMS_143
Namely: />
Figure SMS_144
Wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure SMS_146
is->
Figure SMS_149
Inter-class scatter matrix,/>
Figure SMS_151
Is->
Figure SMS_147
And->
Figure SMS_148
A spreading matrix between the two,
Figure SMS_150
for evaluating->
Figure SMS_152
Inter-class discernment between +.>
Figure SMS_145
For evaluating the difference between the selected source domain and target domain.
In some embodiments, the processor 202 is further configured to: the using a forward order floating search method to maximize the portability of the selected tangential space vector set further comprises:
a set of labeled tangent space vectors from a good source domain selected from the source domains is iteratively selected using a forward-order floating search method to be added to or removed from the selected set of tangent space vectors to maximize the portability of the selected set of tangent space vectors.
In some embodiments, the processor 202 is further configured to: the training method for the classifier comprises the following steps:
using a supervised contracted linear discriminant analysis classifier, after the selected good source domain corresponding to the final selected set of tangent space vectors is obtained, the set of labeled tangent space vectors from the selected good source and target domains are fed back into the classifier for training.
Reference is made to the foregoing method embodiments for specific implementation methods, and details are not repeated here.
The present application may be a method, apparatus, system, and/or computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions embodied thereon for performing the various aspects of the present application.
The computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: portable computer disks, hard disks, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static Random Access Memory (SRAM), portable compact disk read-only memory (CD-ROM), digital Versatile Disks (DVD), memory sticks, floppy disks, mechanical coding devices, punch cards or in-groove structures such as punch cards or grooves having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media, as used herein, are not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses through fiber optic cables), or electrical signals transmitted through wires.
The computer readable program instructions described herein may be downloaded from a computer readable storage medium to a respective computing/processing device or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. The network interface card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in the respective computing/processing device.
Computer program instructions for performing the operations of the present application may be assembly instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, c++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present application are implemented by personalizing electronic circuitry, such as programmable logic circuitry, field Programmable Gate Arrays (FPGAs), or Programmable Logic Arrays (PLAs), with state information for computer readable program instructions, which may execute the computer readable program instructions.
Various aspects of the present application are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer readable program instructions may be provided to a processing unit of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Note that all features disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise. Thus, unless expressly stated otherwise, each feature disclosed is one example only of a generic set of equivalent or similar features. Where used, further, preferably, still further and preferably, the brief description of the other embodiment is provided on the basis of the foregoing embodiment, and further, preferably, further or more preferably, the combination of the contents of the rear band with the foregoing embodiment is provided as a complete construct of the other embodiment. A further embodiment is composed of several further, preferably, still further or preferably arrangements of the strips after the same embodiment, which may be combined arbitrarily.
While the application has been described in detail with respect to the general description and specific embodiments thereof, it will be apparent to those skilled in the art that certain modifications and improvements may be made thereto based upon the application. Accordingly, such modifications or improvements may be made without departing from the spirit of the application and are intended to be within the scope of the invention as claimed.

Claims (7)

1. A selective migration learning method of an electroencephalogram signal, comprising:
preprocessing the acquired electroencephalogram signals of all source domains and target domains by common average references, and reserving electroencephalogram signals of selected channels in the source domains and the target domains; wherein, the liquid crystal display device comprises a liquid crystal display device,
the preprocessing of the acquired electroencephalogram signals of all the source domains and the target domains through the common average reference comprises the following steps:
carrying out spectrum filtering on the acquired electroencephalogram signals of all source domains and all target domains through a third-order Butterworth filter, and carrying out time segmentation after an instruction starts, so as to obtain the selected channel;
affine transformation is carried out on the covariance matrix by using the Riemann mean value of the covariance matrix of each preprocessed source domain and target domain as a reference matrix to obtain a pair Ji Xie variance matrix of each preprocessed source domain and target domain;
using the Riemann mean value of the covariance matrix as a reference matrix, and respectively converting the corresponding pair Ji Xie variance matrix into corresponding tangential space vectors;
using a forward sequence floating search method to maximize the mobility of the selected tangent space vector set to obtain a final selected tangent space vector set so as to use the corresponding selected good source domain for classifier training; wherein, the liquid crystal display device comprises a liquid crystal display device,
the using a forward order floating search method to maximize the portability of the selected tangential space vector set includes:
defining the set of selected tangent space vectors
Figure QLYQS_1
To predict its +.>
Figure QLYQS_2
Namely: />
Figure QLYQS_3
Wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure QLYQS_5
is->
Figure QLYQS_7
Inter-class scatter matrix,/>
Figure QLYQS_9
Is->
Figure QLYQS_6
And->
Figure QLYQS_8
Scattering matrix between->
Figure QLYQS_10
For evaluating->
Figure QLYQS_11
Inter-class discernment between +.>
Figure QLYQS_4
For evaluating the difference between the selected source domain and target domain;
a set of labeled tangent space vectors from a good source domain selected from the source domains is iteratively selected using a forward-order floating search method to be added to or removed from the selected set of tangent space vectors to maximize the portability of the selected set of tangent space vectors.
2. The selective transfer learning method of an electroencephalogram signal according to claim 1, wherein the using the Riemann mean of the covariance matrices of the source and target domains after each preprocessing as a reference matrix includes:
and forming a Riemann manifold by using a symmetrical positive definite matrix corresponding to the covariance matrix, and then calculating to obtain a Riemann average value serving as a reference matrix.
3. The selective transfer learning method of an electroencephalogram signal according to claim 2, characterized by comprising:
the Riemann average is calculated using the Riemann distance:
Figure QLYQS_12
where k represents the kth point on the Riemann manifold,
Figure QLYQS_13
representing the Riemann distance, M is a fit value that minimizes the objective function,
Figure QLYQS_14
a value representing the kth point on the Riemann manifold, < >>
Figure QLYQS_15
Representing the number of all symmetric positive definite matrices on the Riemann manifold, < >>
Figure QLYQS_16
Representing the Riemann mean.
4. The selective transfer learning method of an electroencephalogram signal according to claim 3, characterized in that the affine transformation of the covariance matrix includes:
using the formula:
Figure QLYQS_17
performing affine transformation,
wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure QLYQS_18
representing Riemann mean, < >>
Figure QLYQS_19
Is the i-th symmetric positive definite matrix, +.>
Figure QLYQS_20
Is original->
Figure QLYQS_21
The new value after transformation.
5. The selective transfer learning method of an electroencephalogram signal according to claim 1, characterized in that the training for a classifier includes:
using a supervised contracted linear discriminant analysis classifier, after the selected good source domain corresponding to the final selected set of tangent space vectors is obtained, the set of labeled tangent space vectors from the selected good source and target domains are fed back into the classifier for training.
6. A selective migration learning device for electroencephalogram signals, comprising:
a memory; and
a processor coupled to the memory, the processor configured to:
preprocessing the acquired electroencephalogram signals of all source domains and target domains by common average references, and reserving electroencephalogram signals of selected channels in the source domains and the target domains; wherein, the liquid crystal display device comprises a liquid crystal display device,
the preprocessing of the acquired electroencephalogram signals of all the source domains and the target domains through the common average reference comprises the following steps:
carrying out spectrum filtering on the acquired electroencephalogram signals of all source domains and all target domains through a third-order Butterworth filter, and carrying out time segmentation after an instruction starts, so as to obtain the selected channel;
affine transformation is carried out on the covariance matrix by using the Riemann mean value of the covariance matrix of each preprocessed source domain and target domain as a reference matrix to obtain a pair Ji Xie variance matrix of each preprocessed source domain and target domain;
using the Riemann mean value of the covariance matrix as a reference matrix, and respectively converting the corresponding pair Ji Xie variance matrix into corresponding tangential space vectors;
using a forward sequence floating search method to maximize the mobility of the selected tangent space vector set to obtain a final selected tangent space vector set so as to use the corresponding selected good source domain for classifier training; wherein, the liquid crystal display device comprises a liquid crystal display device,
the using a forward order floating search method to maximize the portability of the selected tangential space vector set includes:
defining the set of selected tangent space vectors
Figure QLYQS_22
To predict its +.>
Figure QLYQS_23
Namely: />
Figure QLYQS_24
Wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure QLYQS_26
is->
Figure QLYQS_29
Inter-class scatter matrix,/>
Figure QLYQS_31
Is->
Figure QLYQS_27
And->
Figure QLYQS_28
Scattering matrix between->
Figure QLYQS_30
For evaluating->
Figure QLYQS_32
Inter-class discernment between +.>
Figure QLYQS_25
For evaluating the difference between the selected source domain and target domain;
a set of labeled tangent space vectors from a good source domain selected from the source domains is iteratively selected using a forward-order floating search method to be added to or removed from the selected set of tangent space vectors to maximize the portability of the selected set of tangent space vectors.
7. A computer storage medium having stored thereon a computer program, which when executed by a machine performs the steps of the method according to any of claims 1 to 5.
CN202310334701.8A 2023-03-31 2023-03-31 Selective migration learning method and device for electroencephalogram signals and storage medium Active CN116049639B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310334701.8A CN116049639B (en) 2023-03-31 2023-03-31 Selective migration learning method and device for electroencephalogram signals and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310334701.8A CN116049639B (en) 2023-03-31 2023-03-31 Selective migration learning method and device for electroencephalogram signals and storage medium

Publications (2)

Publication Number Publication Date
CN116049639A CN116049639A (en) 2023-05-02
CN116049639B true CN116049639B (en) 2023-06-23

Family

ID=86131677

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310334701.8A Active CN116049639B (en) 2023-03-31 2023-03-31 Selective migration learning method and device for electroencephalogram signals and storage medium

Country Status (1)

Country Link
CN (1) CN116049639B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116595437B (en) * 2023-05-17 2023-10-31 中国科学院自动化研究所 Training method, device and storage medium for zero calibration transfer learning classification model
CN116503679B (en) * 2023-06-28 2023-09-05 之江实验室 Image classification method, device, equipment and medium based on migration map

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114254676A (en) * 2021-12-20 2022-03-29 杭州电子科技大学 Source domain selection method for multi-source electroencephalogram migration
CN114305453A (en) * 2021-12-20 2022-04-12 杭州电子科技大学 Multi-source manifold electroencephalogram feature transfer learning method

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2962821B1 (en) * 2010-07-13 2013-02-22 Commissariat Energie Atomique METHOD AND SYSTEM FOR CLASSIFYING NEURAL SIGNALS, AND METHOD FOR SELECTING ELECTRODES FOR DIRECT NEURONAL CONTROL.
CN109199414B (en) * 2018-10-30 2020-11-17 武汉理工大学 Audio-visual evoked emotion recognition method and system based on electroencephalogram signals
CN112580436B (en) * 2020-11-25 2022-05-03 重庆邮电大学 Electroencephalogram signal domain adaptation method based on Riemann manifold coordinate alignment
CN113780169A (en) * 2021-09-10 2021-12-10 山东建筑大学 Riemann space electroencephalogram signal classification method and system based on brain-computer interface
CN114818824A (en) * 2022-05-11 2022-07-29 浙江工业大学 Motor imagery transfer learning method based on Pluker analysis

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114254676A (en) * 2021-12-20 2022-03-29 杭州电子科技大学 Source domain selection method for multi-source electroencephalogram migration
CN114305453A (en) * 2021-12-20 2022-04-12 杭州电子科技大学 Multi-source manifold electroencephalogram feature transfer learning method

Also Published As

Publication number Publication date
CN116049639A (en) 2023-05-02

Similar Documents

Publication Publication Date Title
CN116049639B (en) Selective migration learning method and device for electroencephalogram signals and storage medium
Yger et al. Riemannian approaches in brain-computer interfaces: a review
Agarwal et al. Transfer learning: Survey and classification
Ktena et al. Distance metric learning using graph convolutional networks: Application to functional brain networks
US20210182687A1 (en) Apparatus and method with neural network implementation of domain adaptation
CN105825192A (en) Facial expression identification method and system
EP3620982B1 (en) Sample processing method and device
CN110738660A (en) Spine CT image segmentation method and device based on improved U-net
Baig et al. Detecting malignant leukemia cells using microscopic blood smear images: a deep learning approach
Ayma et al. An adaptive filtering approach for segmentation of tuberculosis bacteria in Ziehl-Neelsen sputum stained images
Hatipoglu et al. A signal-to-image transformation approach for EEG and MEG signal classification
Li et al. Spectral-spatial sparse subspace clustering based on three-dimensional edge-preserving filtering for hyperspectral image
Li et al. A novel weighted anisotropic total variational model for image applications
CN115105094A (en) Attention and 3D dense connection neural network-based motor imagery classification method
Zhou et al. Adaptive weighted locality-constrained sparse coding for glaucoma diagnosis
Hong et al. Group testing for longitudinal data
CN113963193A (en) Method and device for generating vehicle body color classification model and storage medium
CN117520891A (en) Motor imagery electroencephalogram signal classification method and system
Mi et al. A comparative study and improvement of two ICA using reference signal methods
CN114533084B (en) Electroencephalogram feature extraction method and device, electronic equipment and storage medium
EP3588441B1 (en) Imagification of multivariate data sequences
Siuly et al. A new data mining scheme for analysis of big brain signal data
Suthar et al. Phase-stretch adaptive gradient-field extractor (page)
Brooks et al. Second-order networks in PyTorch
Acharya et al. Comparative analysis of feature extraction technique on EEG-based dataset

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant