CN115969392A - Cross-period brainprint recognition method based on tensor frequency space attention domain adaptive network - Google Patents

Cross-period brainprint recognition method based on tensor frequency space attention domain adaptive network Download PDF

Info

Publication number
CN115969392A
CN115969392A CN202310129985.7A CN202310129985A CN115969392A CN 115969392 A CN115969392 A CN 115969392A CN 202310129985 A CN202310129985 A CN 202310129985A CN 115969392 A CN115969392 A CN 115969392A
Authority
CN
China
Prior art keywords
domain
frequency
time
space
tensor
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310129985.7A
Other languages
Chinese (zh)
Inventor
孔万增
金宣妤
杨心宇
朱莉
唐佳佳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Dianzi University
Original Assignee
Hangzhou Dianzi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Dianzi University filed Critical Hangzhou Dianzi University
Priority to CN202310129985.7A priority Critical patent/CN115969392A/en
Publication of CN115969392A publication Critical patent/CN115969392A/en
Priority to US18/223,623 priority patent/US20240282439A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/117—Identification of persons
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/30—Input circuits therefor
    • A61B5/307—Input circuits therefor specially adapted for particular uses
    • A61B5/31—Input circuits therefor specially adapted for particular uses for electroencephalography [EEG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A61B5/372—Analysis of electroencephalograms
    • A61B5/374—Detecting the frequency distribution of signals, e.g. detecting delta, theta, alpha, beta or gamma waves
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7253—Details of waveform analysis characterised by using transforms
    • A61B5/7257—Details of waveform analysis characterised by using transforms using Fourier transforms
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/63—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for local operation

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Biomedical Technology (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Pathology (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Psychiatry (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Psychology (AREA)
  • Signal Processing (AREA)
  • Physiology (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Computational Linguistics (AREA)
  • Fuzzy Systems (AREA)
  • Image Analysis (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)

Abstract

The invention discloses a time-interval-crossing brain print identification method based on a tensor frequency-space attention domain adaptive network. Aiming at most of the existing multi-source domain self-adaptive methods, domain gaps between a plurality of source domains and a target domain are independently closed, and the relation between invariant features of all distribution alignment domains is ignored. The invention assists the performance of the target domain through the important relation of the modeling domain invariant feature without being influenced by the distribution difference between the source domains. A new tensorial frequency-space attention network (TSFAN) is employed to jointly merge appropriate common frequency-space features for both source and target pairs and across the source domain. In consideration of dimension, the TSFAN is further approximately expressed as a low rank Tucker format, so that the TSFAN linearly expands the number of domains, and expands the TSFAN to the condition of any time interval number. The invention can realize efficient cross-period task-independent brain print recognition, and is an effective method for portable brain print recognition in real life.

Description

Cross-period brain streak identification method based on tensor frequency-space attention domain adaptive network
Technical Field
The invention belongs to the field of electroencephalogram signal identification in the field of biological feature identification, and particularly relates to a time-interval-crossing electroencephalogram identification method based on a tensor frequency-space attention domain adaptive network.
Background
Biometric identification relies on personal characteristics and plays a key role in identity authentication systems. Although physical biometric identification, such as face recognition and fingerprint recognition, has been widely used in real life, the potential risk of elaborate counterfeiting or secret duplication remains unavoidable. In addition to physical biological characteristics, brain activities recorded by electroencephalogram (EEG) signals are proposed as a new cognitive biological characteristic, and the basic identity identification requirements are met. Furthermore, only living individuals can provide signals of brain activity, which are not controlled by the user. This means that the user's identity information cannot be intentionally leaked or stolen, making electroencephalogram-based biometric identification techniques suitable for security-critical applications.
Reliable and stable electroencephalogram identity characteristics are the basis of biological characteristic identification based on electroencephalogram. In fact, the traditional machine learning methods used in a large number of studies, which require significant expertise to extract features, are not always sufficient to have good performance. In recent years, deep learning has attracted considerable attention in decoding EEG recognition features, as it is able to capture high-level features and potential dependencies. In general, various types of deep learning methods, such as Convolutional Neural Networks (CNNs), recurrent Neural Networks (RNNs), and Graph Convolutional Neural Networks (GCNNs), have been demonstrated to extract temporal, frequency, and spatial identity discrimination features from brain electrical signals.
The electroencephalogram signals between different conversations are unstable under the influence of factors such as impedance, micro displacement of the electrode position, change of the tested state and the like. Thus, despite these significant advances, cross-period based biometric identification in real world scenarios remains challenging. Most previous researches focus on time-interval or mixed multi-time-interval data, and distribution differences among electroencephalogram data in a plurality of training time intervals are ignored. Intuitively, even between a single source domain (training session) and target domain (test session) data, the offset of domain-invariant representation extraction is not easily eliminated, and a greater degree of mismatch of multiple source domains may result in unsatisfactory performance.
In order to avoid the influence of domain deviation between multi-source domains, the electroencephalogram multi-source domain self-adaption method respectively minimizes the difference between a source domain and a target domain. In fact, the domain-invariant features captured with different source domains represent stable information from multiple views and deliver more appropriate information to the target domain. However, each domain-invariant feature computed by distributed alignment may be affected by the source domain involved and may not benefit from a common relationship of multiple source domains.
In order to solve the problems, the invention provides a cross-period brain print recognition method (TSFAN) based on a tensor frequency space attention domain adaptive network to capture EEG identity characteristics which are stable in a cross-period mode. Specifically, each pair of source domain and target domain data is mapped to a different temporal feature space, respectively. Then, the core idea of TSFAN is designed, namely based on tensor attention, the frequency-space attention of a source domain and a target domain is subjected to tensor quantization to obtain domain-invariant space-frequency characteristics, and the domain-invariant space-frequency characteristics naturally contribute to interaction between transferable information in a source and complex sources. Considering the dimensionality disaster, the tensor of the low-rank Tucker format is further adopted, so that the TSFAN can be linearly expanded in the number of domains.
Disclosure of Invention
The invention aims to provide a cross-period brain print identification method based on a tensor frequency space attention domain adaptive network aiming at the defects of the prior art. The method mainly constructs a tensor frequency space attention network based on multi-source domain adaptation, and fully utilizes the interaction correlation among different domains while relieving the data distribution difference of a source domain and a target domain in pairs.
A cross-period brain print recognition method based on a tensor frequency-space attention domain adaptive network comprises the following steps:
step (1), preprocessing original electroencephalogram data;
1-1, collecting electroencephalogram data generated by external stimulation of a plurality of subjects in different time periods under the same experimental paradigm;
1-2, filtering original electroencephalogram data by using a Butterworth filter for removing interference of factors such as external equipment and myoelectricity, and then performing Short-time Fourier Transform (STFT);
1-3, intercepting the electroencephalogram data obtained by the processing in the step 1-2, and marking the corresponding electroencephalogram sample data with a label of a tested object;
1-4, dividing the electroencephalogram sample data obtained after the processing of the step 1-3 into a training set and a testing set according to a proportion, wherein the training set data comprises K time period data, namely K source domains, and K is more than or equal to 2; taking the test set as a target domain;
constructing a tensor-based frequency-space attention domain adaptive network model, and training and testing the tensor-based frequency-space attention domain adaptive network model;
the tensor frequency space attention domain adaptive network model comprises K specific domain feature extraction networks with the same structure and 1 tensor frequency space attention network, wherein each specific domain feature extraction network comprises a multi-scale one-dimensional convolutional layer, a splicing layer, a maximum pooling layer, a fusion layer and a frequency space convolutional layer; wherein the multi-scale one-dimensional convolution layer comprises a plurality of parallel one-dimensional convolutions of different scales; the frequency-space convolution layer comprises a frequency domain one-dimensional convolution and a space domain one-dimensional convolution which are sequentially connected in series;
the input of the multi-scale one-dimensional convolutional layer is certain source domain data and target domain data, and the input is output to the splicing layer;
the splicing layer splices a plurality of received different-scale features to obtain a source domain brain print time domain feature Zt sj And target domain stria time domain feature Zt tj ，j∈[1,K]Then, the characteristics are respectively output to a maximum pooling layer and a fusion layer;
the maximum pooling layer performs dimensionality reduction on the received features in a time dimension and then outputs the features to a tensor frequency-space attention network;
the tensor frequency space attention network receives the features output by the maximum pooling layer of K specific domain feature extraction networks, and carries out interactive processing on the features to obtain the source domain frequency space attention Q containing the interactive correlation among the features sj And target domain frequency space attention Q tj Then outputting the above attention to the fusion layer; the method comprises the following steps:
the tensor frequency space attention network utilizes two full-connection layers to realize nonlinear mapping on the characteristics output by the K specific domain characteristic extraction networks to obtain the source domain frequency space attention Q sj And target domain frequency space attention Q tj ：
Q sj ＝F bj (Relu(F aj (P sj ；V j ))；u j ) Formula (1)
Q tj ＝F bj (Relu(F aj (P tj ；V j ))；u j )
Wherein, F aj And F bj Two fully-connected layers, V, representing the jth source domain space j And U j A parameter representing the two fully-connected layers, relu (), is the activation function,
Figure BDA0004083450000000031
representing the space frequency characteristics of a source domain and a target domain output by the maximum pooling layer; c is the number of original characteristic electroencephalogram channels, and s is the dimension of the original characteristic frequency domain;
the parameters of the full connection layer in the formula (1)
Figure BDA0004083450000000032
Higher-order tensor expressed in a tensorial order of (K +1 @)>
Figure BDA0004083450000000033
Figure BDA0004083450000000034
C' is a full junction layer F aj The number of the electroencephalogram channels of the processed characteristics, s', is the total connection layer F aj The frequency domain dimension size of the processed features is used for acquiring the interactive correlation among the features, and the high-order tensor/based on the consideration that dimension disaster may be caused along with the increase of the number of source domains, the high-order tensor/based on the low-rank Tucker form is adopted to express the high-order tensor/based on the relation between the source domains and the features>
Figure BDA0004083450000000035
Figure BDA0004083450000000036
/>
Wherein
Figure BDA0004083450000000037
{r 1 ...r K+1 Is a rank of the form Tucker, I 1 ＝I 2 ＝...＝I K ＝c′s′，I K+1 = cs; c is the number of original characteristic electroencephalogram channels, and s is the dimension of the original characteristic frequency domain;
the fusion layer receives the source domain brain print time domain characteristics Zt sj And target domain brain streak time domain feature Zt tj Respectively with the source domain frequency space attention Q sj And target domain frequency space attention Q tj Fusing to obtain a frequency-space enhanced source domain brain texture time domain feature Zt' sj And target domain brain texture time domain feature Zt' tj And output to the frequency space convolution layer;
the frequency-space convolutional layer converts the received time domain characteristic Zt' sj ，Zt’ tj Extracting to obtain source domain time-frequency space-brain-print characteristic Z through frequency domain one-dimensional convolution and space domain one-dimensional convolution operation sj And target domain time-frequency space-brain print characteristic Z tj ；
Step (3), constructing a classifier for identifying the brain prints, and training and testing the classifier;
the time-frequency space characteristics Z output in the step 2 sj ,Z tj Flattening, calculating sample by full connection layer and Softmax activation functionProbability of belonging to each category;
and (4) realizing cross-period brainprint recognition by using a tensegrity frequency-space attention domain-based adaptive network model which is well trained and tested and a classifier for brainprint recognition.
Preferably, in the step 1-2, the original electroencephalogram data are filtered by using a Butterworth filter, specifically, the electroencephalogram data are subjected to down-sampling to 250Hz, and the original electroencephalogram data are subjected to filtering processing of 0-75 Hz by using the Butterworth filter.
Preferably, the fast fourier transform in step 1-2 is specifically to perform short-time fourier transform on the filtered signal x to extract time-frequency features:
adopting a time-limited window function h (t), assuming that a non-stationary signal x is stationary in a time window, and analyzing the signal x section by section to obtain a group of local frequency spectrums of the signal by moving the window function h (t) on a time axis; the short-time fourier transform of the signal x (τ) is defined as:
Figure BDA0004083450000000041
where STFT (t, f) represents the short-time Fourier transform of the signal x (τ) at time t, h (τ -t) is a window function, and f represents frequency.
Preferably, the fusion layer is specifically:
Figure BDA0004083450000000042
Figure BDA0004083450000000043
preferably, the loss function of the classifier for brain print recognition
Figure BDA0004083450000000044
Comprises the following steps:
Figure BDA0004083450000000045
wherein theta is y Is the classifier parameter, N is the number of classes, θ f The parameters of the feature extractor are represented as,
Figure BDA0004083450000000046
represents the cross entropy loss of class i, and E () represents the cross entropy function.
Preferably, the total loss function of the classifier pair for brain print recognition is based on a tensor frequency-space attention domain adaptive network model
Figure BDA0004083450000000047
Comprises the following steps:
Figure BDA0004083450000000048
wherein
Figure BDA0004083450000000049
The representation is used to measure the classifier distance penalty function->
Figure BDA00040834500000000410
The data distribution difference loss function for measuring the source domain and the target domain is shown, and lambda and gamma are hyperparameters.
Another object of the present invention is to provide a cross-session brain print recognition apparatus, comprising:
the electroencephalogram data preprocessing module is used for filtering and fast Fourier transforming the acquired electroencephalogram data in different time periods;
the well-trained and tested network model based on the tensor frequency-space attention domain is used for extracting the characteristics of the electroencephalogram data output by the electroencephalogram data preprocessing module in different time periods and acquiring the source domain time-frequency-space electroencephalogram characteristics Z sj And target domain time-frequency space-brain print characteristic Z tj ；
Training the tested classifier for identifying the brain print, and performing source domain time-frequency space brain print characteristic Z sj And target domain time-frequency space-brain print characteristic Z tj And flattening, namely calculating the probability that the sample belongs to each category through a full connection layer and a Softmax activation function, and realizing the cross-period brain print recognition.
It is a further object of the present invention to provide a computer-readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform the above-mentioned method.
It is a further object of the present invention to provide a computing device comprising a memory having stored therein executable code and a processor that, when executing the executable code, implements the method described above.
The invention has the beneficial effects that:
the invention provides a method for jointly capturing intra-source transferable information of domain invariant features and cross-source interaction to relieve judgment capacity reduction caused by global distribution alignment, and provides an attention mechanism based on tensor. The method is expected to be applied to the biological identification technology with high confidentiality as the brain print identification.
Drawings
FIG. 1 is a flow chart of a brain print recognition model according to the present invention;
fig. 2 is a diagram of a tensor frequency-space attention domain adaptive network architecture according to the present invention.
Detailed Description
In order to make the implementation objects, technical solutions and advantages of the present invention clearer, the following detailed description is further described with reference to the technical solutions of the present invention and the accompanying drawings:
the invention relates to a time-interval-crossing brain print identification method of a tensor frequency-space attention domain adaptive network, and a flow chart of the method is shown in figure 1. The model architecture diagram is shown in fig. 2, and specifically consists of two modules: (1) In-source migratable feature learning, performing multi-source domain self-adaptation by using a time feature extractor and a spatial frequency feature extractor to obtain domain invariant features of each paired source domain and target domain; (2) Frequency space attention is tensioned to simulate complex source-to-source interactions. The overall architecture is carefully designed to explore stable electroencephalographic recognition features across sessions.
Step 1, preprocessing original electroencephalogram data
1) The noise frequency contained in the original electroencephalogram signal is usually lower than 0.5Hz or higher than 50Hz, in order to remove power frequency interference caused by electroencephalogram acquisition equipment and tested electromyogram interference, the electroencephalogram data are down-sampled to 250Hz, and the original electroencephalogram data are filtered by a Butterworth filter at 0-75 Hz;
2) And performing short-time Fourier transform on the signal x output in the operation 1) to extract time-frequency characteristics. A time-limited window function h (t) is adopted, a non-stationary signal x is assumed to be stationary in a time window, the signal x is analyzed section by section through the movement of the window function h (t) on a time axis to obtain a group of local frequency spectrums of the signal, and the specific window size of the scheme is 0.5s. The short-time fourier transform of the signal x (τ) is defined as:
Figure BDA0004083450000000051
where STFT (t, f) represents the short-time Fourier transform of the signal x (τ) at time t, where h (τ -t) is a window function and f represents frequency.
3) Intercepting the electroencephalogram data obtained by the processing of the step 2) by adopting a time window of 15s, and marking the corresponding electroencephalogram sample data with a tested label;
4) Dividing the EEG sample data obtained after the processing of the step 3) into training sets according to a proportion
Figure BDA0004083450000000061
And test set { X t ，Y t Where K is the number of time segments. Electroencephalogram sample based on or based on>
Figure BDA0004083450000000062
Wherein c is the number of electroencephalogram channels, s is the dimension of frequency domain, and t is the dimension of time domain. The specific scheme selects Fz, F7, F8, C3, C4, P7, P8 and O1,o2 nine channels, 1-30Hz, sample rate 250Hz, i.e. c =9,s =30,t =30.
Step 2, constructing a tensor-based frequency-space attention domain adaptive network model;
the tensor-based frequency-space attention domain adaptive network model comprises K specific domain feature extraction networks with the same structure and 1 tensor-based frequency-space attention network, wherein each specific domain feature extraction network comprises a multi-scale one-dimensional convolutional layer, a splicing layer, a maximum pooling layer, a fusion layer and a frequency-space convolutional layer; wherein the multi-scale one-dimensional convolution layer comprises a plurality of parallel one-dimensional convolutions of different scales; the frequency-space convolution layer comprises a frequency domain one-dimensional convolution and a space domain one-dimensional convolution which are sequentially connected in series;
the input of the multi-scale one-dimensional convolutional layer is certain source domain data and target domain data, and the input is output to the splicing layer;
the splicing layer splices a plurality of received different-scale features to obtain a source domain brain print time domain feature Zt sj And target domain stria time domain feature Zt tj ，j∈[1，K]Then, the characteristics are respectively output to a maximum pooling layer and a fusion layer;
the maximum pooling layer carries out dimensionality reduction on the received features in time dimensionality and then outputs the features to a tensorial frequency-space attention network;
the tensor frequency space attention network receives the features output by the maximum pooling layer of K specific domain feature extraction networks, and carries out interactive processing on the features to obtain the source domain frequency space attention Q containing the interactive correlation among the features sj And target domain frequency space attention Q tj Then outputting the above attention to the fusion layer; the method comprises the following steps:
the tensor frequency space attention network utilizes two full connection layers to realize nonlinear mapping on the characteristics output by the K specific domain characteristic extraction network to obtain the source domain frequency space attention Q sj And target domain frequency space attention Q tj ：
Q sj ＝F bj (Relu(F aj (P sj ；V j ))；u j ) Formula (1)
Q tj ＝F bj (Relu(F aj (P tj ；V j ))；u j )
Wherein, F aj And F bj Two fully-connected layers, V, representing the jth source domain space j And U j Parameters representing two fully connected layers, relu () being the activation function,
Figure BDA0004083450000000063
representing the space frequency characteristics of a source domain and a target domain output by the maximum pooling layer; c is the number of original characteristic electroencephalogram channels, and s is the dimension of the original characteristic frequency domain;
the parameters of the full connection layer in the formula (1)
Figure BDA0004083450000000064
Higher-order tensor expressed in a tensorial order of (K +1 @)>
Figure BDA0004083450000000065
Figure BDA0004083450000000066
C' is a full junction layer F aj The number of the electroencephalogram channels of the processed characteristics, s', is the total connection layer F aj The frequency domain dimension size of the processed features is used for acquiring the interactive correlation among the features, and the high-order tensor/based on the consideration that dimension disaster may be caused along with the increase of the number of source domains, the high-order tensor/based on the low-rank Tucker form is adopted to express the high-order tensor/based on the relation between the source domains and the features>
Figure BDA0004083450000000071
Figure BDA0004083450000000072
Wherein
Figure BDA0004083450000000073
{r 1 ...r K+1 Is a rank of the form Tucker, I 1 ＝I 2 ＝...＝I K ＝c′s′，I K+1 = cs; c is the number of original characteristic brain channels, s is original characteristicCharacterizing the dimension of a frequency domain;
the fusion layer receives the source domain brain print time domain characteristics Zt sj And target domain brain streak time domain feature Zt tj Respectively with the source domain frequency space attention Q sj And target domain frequency space attention Q tj Fusing to obtain a frequency-space enhanced source domain brain texture time domain feature Zt' sj And target domain brain texture time domain feature Zt' tj And output to the frequency space convolution layer; the method comprises the following steps:
Figure BDA0004083450000000074
Figure BDA0004083450000000075
/>
the frequency-space convolutional layer converts the received time domain feature Zt' sj ，Zt’ tj Extracting to obtain source domain time-frequency space-brain print characteristic Z through frequency domain one-dimensional convolution and space domain one-dimensional convolution operation sj And target domain time-frequency space-brain print characteristic Z tj ；
Step 3, constructing a classifier for identifying the brain print;
flattening the time-frequency space characteristics output in the step 2, calculating the probability that the sample belongs to each class through a full connection layer and a Softmax activation function, and defining the loss function of the classifier as
Figure BDA0004083450000000076
Wherein theta is y And N is the number of categories as classifier parameters.
Step 4, training the network model
And (3) performing gradient back propagation optimization loss function on the model constructed in the steps 2 to 3 by adopting the training set obtained in the step 1.4, and storing the optimal model for testing through the verification set obtained in the step 1.4. The loss function is expressed as:
Figure BDA0004083450000000077
wherein
Figure BDA0004083450000000078
For measuring the classifier distance>
Figure BDA0004083450000000079
For measuring the difference of data distribution of a source domain and a target domain, lambda and gamma are hyperreferences, and the data distribution is set to be 0.5. Using the SGD optimizer, the learning rate was 0.025 and the batch size was 64.
And 7, verifying the validity of the scheme on the multi-task identity recognition data set, wherein the scheme comprises 30 tested N =30, verifying the data in the first time interval and the data in the last time interval as two data division modes of testing, and performing a comparison experiment with the existing domain merging and multi-source domain method, wherein the result is shown in table 1. Verification results show that the model provided by the invention can effectively extract stable brain streak characteristics in different time periods.
TABLE 1 accuracy and equal error rate of models across time periods for identification datasets
Figure BDA00040834500000000710
Figure BDA0004083450000000081
/>

Claims (9)

1. A time-interval-crossing brain streak identification method based on a tensor frequency-space attention domain adaptive network is characterized by comprising the following steps of:
preprocessing original electroencephalogram data;
1-1, acquiring electroencephalogram data generated by external stimulation of a plurality of subjects in different time periods under the same experimental paradigm;
1-2, filtering original electroencephalogram data by using a Butterworth filter, and then performing fast Fourier transform;
1-3, intercepting the electroencephalogram data obtained by the processing in the step 1-2, and marking the corresponding electroencephalogram sample data with a label of a tested object;
1-4, dividing the electroencephalogram sample data obtained after the processing of the step 1-3 into a training set and a testing set according to a proportion, wherein the training set data comprises K time period data, namely K source domains, and K is more than or equal to 2; taking the test set as a target domain;
constructing a tensor-based frequency-space attention domain adaptive network model, and training and testing the tensor-based frequency-space attention domain adaptive network model;
the tensor-based frequency-space attention domain adaptive network model comprises K specific domain feature extraction networks with the same structure and 1 tensor-based frequency-space attention network, wherein each specific domain feature extraction network comprises a multi-scale one-dimensional convolutional layer, a splicing layer, a maximum pooling layer, a fusion layer and a frequency-space convolutional layer; wherein the multi-scale one-dimensional convolution layer comprises a plurality of parallel one-dimensional convolutions of different scales; the frequency-space convolution layer comprises a frequency domain one-dimensional convolution and a space domain one-dimensional convolution which are sequentially connected in series;
the input of the multi-scale one-dimensional convolutional layer is certain source domain data and target domain data, and the input is output to the splicing layer;
the splicing layer splices a plurality of received different-scale features to obtain a source domain brain print time domain feature Zt sj And target domain brain streak time domain feature Zt tj ，j∈[1，K]Then, the characteristics are respectively output to a maximum pooling layer and a fusion layer;
the maximum pooling layer performs dimensionality reduction on the received features in a time dimension and then outputs the features to a tensor frequency-space attention network;
the tensor frequency space attention network receives the features output by the maximum pooling layer of K specific domain feature extraction networks, and carries out interactive processing on the features to obtain the source domain frequency space attention Q containing the interactive correlation among the features sj And target domain frequency space attention Q tj Then outputting the above attention to the fusion layer; the method comprises the following steps:
the tensor frequency-space attention network utilizes two full connection layers to pair K charactersThe non-linear mapping is realized by extracting the characteristics output by the network through the localized characteristics to obtain the source domain frequency space attention Q sj And target domain frequency space attention Q tj ：
Q sj ＝F bj (Relu(F aj (P sj ；V j ))；U j ) Formula (1)
Q tj ＝F bj (Relu(F aj (P tj ；V j ))；U j )
Wherein, F aj And F bi Two fully-connected layers, V, representing the jth source domain space j And U j Parameters representing two fully connected layers, relu () being the activation function,
Figure FDA0004083449980000021
representing the source domain and target domain spatial frequency characteristics of the maximum pooling layer output; c is the number of original characteristic electroencephalogram channels, and s is the dimension of the original characteristic frequency domain;
the parameters of the full connection layer in the formula (1)
Figure FDA0004083449980000022
Higher-order tensor expressed in a tensorial order of (K +1 @)>
Figure FDA0004083449980000023
Figure FDA0004083449980000024
C' is a full junction layer F aj The number of the electroencephalogram channels of the processed characteristics, s', is the total connection layer F aj The frequency domain dimension size of the processed features is used for acquiring the interactive correlation among the features, and the high-order tensor/based on the consideration that dimension disaster may be caused along with the increase of the number of source domains, the high-order tensor/based on the low-rank Tucker form is adopted to express the high-order tensor/based on the relation between the source domains and the features>
Figure FDA0004083449980000025
Figure FDA0004083449980000026
Wherein
Figure FDA0004083449980000027
{r 1 ...r K+1 Is rank of the form Tucker, I 1 ＝I 2 ＝...＝I K ＝c′s′，I K+1 = cs; c is the number of original characteristic electroencephalogram channels, and s is the dimension of the original characteristic frequency domain;
the fusion layer receives the source domain brain print time domain characteristics Zt sj And target domain brain streak time domain feature Zt tj Respectively with the source domain frequency space attention Q sj And target domain frequency space attention Q tj Fusing to obtain a frequency-space enhanced source domain brain texture time domain feature Zt' sj And target domain brain texture time domain feature Zt' tj And output to the frequency-space convolution layer;
the frequency-space convolutional layer converts the received time domain feature Zt' sj ，Zt’ tj Extracting to obtain source domain time-frequency space-brain print characteristic Z through frequency domain one-dimensional convolution and space domain one-dimensional convolution operation sj And target domain time frequency space brain print characteristic Z tj ；
Step (3), constructing a classifier for identifying the brain prints, and training and testing the classifier;
the source domain time-frequency space-brain print characteristic Z output in the step 2 sj And target domain time-frequency space-brain print characteristic Z tj Flattening, namely calculating the probability of the sample belonging to each category through a full connection layer and a Softmax activation function;
and (4) realizing cross-period brain print recognition by utilizing a well-trained and tested tensor-based frequency-space attention domain adaptive network model and a classifier for brain print recognition.
2. The method of claim 1, wherein the step 1-2 of filtering the raw electroencephalogram data by using the butterworth filter is to down-sample the electroencephalogram data to 250Hz, and perform filtering processing of 0-75 Hz on the raw electroencephalogram data by using the butterworth filter.
3. The method according to claim 1, wherein the fast fourier transform of step 1-2 is specifically a short-time fourier transform of the filtered signal x to extract time-frequency features:
adopting a time-limited window function h (t), supposing that a non-stationary signal x is stationary in a time window, and analyzing the signal x section by section to obtain a group of local frequency spectrums of the signal by moving the window function h (t) on a time axis; the short-time fourier transform of the signal x (τ) is defined as:
Figure FDA0004083449980000028
where STFT (t, f) represents the short-time Fourier transform of the signal x (τ) at time t, h (τ -t) is a window function, and f represents frequency.
4. The method according to claim 1, characterized in that the fusion layer is in particular:
Figure FDA0004083449980000031
Figure FDA0004083449980000032
5. the method according to claim 1, characterized in that the loss function of the classifier for brain print recognition
Figure FDA0004083449980000033
Comprises the following steps:
Figure FDA0004083449980000034
whereinθ y Is the classifier parameter, N is the number of classes, θ f The parameters of the feature extractor are represented,
Figure FDA0004083449980000035
represents the cross entropy loss of class i, and E () represents the cross entropy function.
6. The method according to claim 5, wherein the total loss function is based on a tensor frequency-space attention domain adaptive network model and a classifier pair for brain print recognition
Figure FDA0004083449980000036
Comprises the following steps:
Figure FDA0004083449980000037
wherein
Figure FDA0004083449980000038
Representing a function for weighing classifier distance loss>
Figure FDA0004083449980000039
The data distribution difference loss function for measuring the source domain and the target domain is shown, and lambda and gamma are hyperparameters.
7. A cross-session brain print recognition apparatus for implementing the method of any one of claims 1-6, comprising:
the electroencephalogram data preprocessing module is used for filtering and fast Fourier transforming the acquired electroencephalogram data in different time periods;
the well-trained and tested network model based on the tensor frequency-space attention domain is used for extracting the characteristics of the electroencephalogram data output by the electroencephalogram data preprocessing module in different time periods and acquiring the source domain time-frequency-space electroencephalogram characteristics Z sj And target domain time-frequency space-brain print characteristic Z tj ；
Training testThe tested classifier for identifying the brain print is used for the source domain time-frequency space brain print characteristic Z sj And target domain time frequency space brain print characteristic Z tj And flattening, namely calculating the probability that the sample belongs to each category through a full connection layer and a Softmax activation function, and realizing the cross-period brain print recognition.
8. A computer-readable storage medium, on which a computer program is stored which, when executed in a computer, causes the computer to carry out the method of any one of claims 1-6.
9. A computing device comprising a memory having executable code stored therein and a processor that, when executing the executable code, implements the method of any of claims 1-6.
CN202310129985.7A 2023-02-17 2023-02-17 Cross-period brainprint recognition method based on tensor frequency space attention domain adaptive network Pending CN115969392A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202310129985.7A CN115969392A (en) 2023-02-17 2023-02-17 Cross-period brainprint recognition method based on tensor frequency space attention domain adaptive network
US18/223,623 US20240282439A1 (en) 2023-02-17 2023-07-19 Cross-session brainprint recognition method based on tensorized spatial-frequency attention network (tsfan) with domain adaptation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310129985.7A CN115969392A (en) 2023-02-17 2023-02-17 Cross-period brainprint recognition method based on tensor frequency space attention domain adaptive network

Publications (1)

Publication Number Publication Date
CN115969392A true CN115969392A (en) 2023-04-18

Family

ID=85976284

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310129985.7A Pending CN115969392A (en) 2023-02-17 2023-02-17 Cross-period brainprint recognition method based on tensor frequency space attention domain adaptive network

Country Status (2)

Country Link
US (1) US20240282439A1 (en)
CN (1) CN115969392A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117503057A (en) * 2023-11-09 2024-02-06 电子科技大学 Epileptic seizure detection device and medium for constructing brain network based on high-order tensor decomposition
CN118296357A (en) * 2024-05-30 2024-07-05 小舟科技有限公司 Electroencephalogram signal characteristic identification and representation method and device and computer equipment

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117503057A (en) * 2023-11-09 2024-02-06 电子科技大学 Epileptic seizure detection device and medium for constructing brain network based on high-order tensor decomposition
CN117503057B (en) * 2023-11-09 2024-05-31 电子科技大学 Epileptic seizure detection device and medium for constructing brain network based on high-order tensor decomposition
CN118296357A (en) * 2024-05-30 2024-07-05 小舟科技有限公司 Electroencephalogram signal characteristic identification and representation method and device and computer equipment
CN118296357B (en) * 2024-05-30 2024-08-09 小舟科技有限公司 Electroencephalogram signal characteristic identification and representation method and device and computer equipment

Also Published As

Publication number Publication date
US20240282439A1 (en) 2024-08-22

Similar Documents

Publication Publication Date Title
CN108776788B (en) Brain wave-based identification method
Chen et al. EEG-based biometric identification with convolutional neural network
CN112244873B (en) Electroencephalogram space-time feature learning and emotion classification method based on hybrid neural network
Kreuz et al. Measuring spike train synchrony
CN115969392A (en) Cross-period brainprint recognition method based on tensor frequency space attention domain adaptive network
US11232370B2 (en) Biometric data to facilitate learning
CN112200016A (en) Electroencephalogram signal emotion recognition based on ensemble learning method AdaBoost
CN109871831A (en) A kind of emotion identification method and system
CN115919330A (en) EEG Emotional State Classification Method Based on Multi-level SE Attention and Graph Convolution
CN116150670A (en) Task independent brain pattern recognition method based on feature decorrelation decoupling
CN117332300A (en) Motor imagery electroencephalogram classification method based on self-attention improved domain adaptation network
US20230315203A1 (en) Brain-Computer Interface Decoding Method and Apparatus Based on Point-Position Equivalent Augmentation
CN116595434A (en) Lie detection method based on dimension and classification algorithm
CN116821764A (en) Knowledge distillation-based multi-source domain adaptive EEG emotion state classification method
CN117503157A (en) Electroencephalogram signal emotion recognition method based on SGCRNN model
CN115758118A (en) Multi-source manifold embedding feature selection method based on electroencephalogram mutual information
CN115005839A (en) Electroencephalogram signal self-supervision representation learning method, system and storage medium
Kong et al. Task-free brainprint recognition based on low-rank and sparse decomposition model
Divya et al. Identification of epileptic seizures using autoencoders and convolutional neural network
Kolla et al. Comparative Analysis of VGG19, ResNet50, and GoogLeNet Inception Models for BCI
Hadiyoso et al. ECG based biometric using wavelet packet decomposition
Eskandari VAE-clustering of neural signals and their association to cytokines
Fang et al. Enhancing Medical Signal Processing and Diagnosis with AI-Generated Content Techniques
Nhu Deep Learning For Physiological Monitoring And Epilepsy Diagnosis
CN116702003A (en) Label symmetry-based cross-test electroencephalogram emotion recognition method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination