CN114209319B - fNIRS emotion recognition method and system based on graph network and self-adaptive denoising - Google Patents
fNIRS emotion recognition method and system based on graph network and self-adaptive denoising Download PDFInfo
- Publication number
- CN114209319B CN114209319B CN202111315105.2A CN202111315105A CN114209319B CN 114209319 B CN114209319 B CN 114209319B CN 202111315105 A CN202111315105 A CN 202111315105A CN 114209319 B CN114209319 B CN 114209319B
- Authority
- CN
- China
- Prior art keywords
- fnirs
- graph
- concentration
- emotion
- variation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 230000008909 emotion recognition Effects 0.000 title claims abstract description 38
- 230000008451 emotion Effects 0.000 claims abstract description 29
- INGWEZCOABYORO-UHFFFAOYSA-N 2-(furan-2-yl)-7-methyl-1h-1,8-naphthyridin-4-one Chemical compound N=1C2=NC(C)=CC=C2C(O)=CC=1C1=CC=CO1 INGWEZCOABYORO-UHFFFAOYSA-N 0.000 claims abstract description 16
- 108010064719 Oxyhemoglobins Proteins 0.000 claims abstract description 16
- 108010002255 deoxyhemoglobin Proteins 0.000 claims abstract description 16
- 238000002835 absorbance Methods 0.000 claims abstract description 13
- 238000013507 mapping Methods 0.000 claims abstract description 13
- 210000004556 brain Anatomy 0.000 claims abstract description 11
- 239000000523 sample Substances 0.000 claims abstract description 9
- 108010054147 Hemoglobins Proteins 0.000 claims abstract 8
- 102000001554 Hemoglobins Human genes 0.000 claims abstract 8
- 230000008859 change Effects 0.000 claims description 18
- 239000011159 matrix material Substances 0.000 claims description 14
- 230000003044 adaptive effect Effects 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 9
- 238000010586 diagram Methods 0.000 claims description 8
- 230000006870 function Effects 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 8
- 230000007246 mechanism Effects 0.000 claims description 6
- 230000015654 memory Effects 0.000 claims description 6
- 210000005013 brain tissue Anatomy 0.000 claims description 4
- 238000011176 pooling Methods 0.000 claims description 4
- 230000009467 reduction Effects 0.000 claims description 4
- 238000011423 initialization method Methods 0.000 claims description 2
- 230000010354 integration Effects 0.000 claims description 2
- 108091026922 FnrS RNA Proteins 0.000 description 7
- 238000001514 detection method Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000000605 extraction Methods 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 230000004913 activation Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 230000000638 stimulation Effects 0.000 description 3
- 206010063659 Aversion Diseases 0.000 description 2
- 101000872083 Danio rerio Delta-like protein C Proteins 0.000 description 2
- 230000008033 biological extinction Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000012512 characterization method Methods 0.000 description 2
- 125000004122 cyclic group Chemical group 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000002329 infrared spectrum Methods 0.000 description 2
- 238000004497 NIR spectroscopy Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000003542 behavioural effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000007274 generation of a signal involved in cell-cell signaling Effects 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000031700 light absorption Effects 0.000 description 1
- 230000007787 long-term memory Effects 0.000 description 1
- 238000002582 magnetoencephalography Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000004630 mental health Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/145—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue
- A61B5/1455—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue using optical sensors, e.g. spectral photometrical oximeters
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
Abstract
The invention discloses a method and a system for identifying fNIRS emotion based on graph network and self-adaptive denoising, wherein the method comprises the steps that fNIRS acquisition equipment continuously acquires the variation of light intensity before and after transmitting and receiving, converts the variation of light intensity into the variation of absorbance, and further obtains the relative variation of concentration of oxyhemoglobin and deoxyhemoglobin; denoising through a self-adaptive denoising network model to obtain a pure signal, wherein the input signal of the self-adaptive denoising network model is the relative variation of the concentration of the oxygenated hemoglobin and the concentration of the deoxygenated hemoglobin obtained in the previous step, and the output signal is the data of the relative variation of the concentration of the pure oxygenated hemoglobin and the concentration of the deoxygenated hemoglobin; and mapping graph nodes by combining the probe and channel characteristics, restoring brain topology by using a graph network, and classifying and outputting emotion labels through a dynamic graph attention emotion recognition network model. The invention solves the problems of complex wearing, difficult operation and the like of the brain-computer interface in practical application at present.
Description
Technical Field
The invention relates to the field of man-machine signal recognition, in particular to a fNIRS emotion recognition method and system based on a graph network and self-adaptive denoising.
Background
Emotion affects cognition and behavioral activity of a person and is also an important influencing factor of mental health. Emotion recognition is a hotspot in which studies can be classified into two types of non-physiological signals and physiological signals. As a traditional physiological index detection method, electroencephalogram, magnetoencephalography, functional magnetic resonance and the like have made certain progress in emotion recognition. At the same time, limitations of such methods are also gradually revealed, such as: low time or space resolution, high cost of acquisition equipment, easy interference, inconvenient carrying, etc.
In recent years, with the development of Near infrared technology and the technical upgrading of acquisition equipment, a Functional Near infrared spectroscopy (fNIRS) technology is used as an emerging non-invasive brain detection means, and has the advantages of high compliance, strong anti-interference capability, portability, easiness in implementation, low cost and the like, so that the method is suitable for all possible tested groups and experimental scenes. With the continuous development of technologies such as 5G technology, internet of things, man-machine interaction, machine learning and the like, the emotion analysis based on fNIRS has important significance and wide application prospect in the fields of medical care, media entertainment, information retrieval, education, intelligent wearable equipment and the like. Therefore, the emotion recognition method and system based on the functional near infrared spectrum technology have wide requirements and wide prospects.
Disclosure of Invention
In order to overcome the defects and shortcomings in the prior art, the invention provides an fNIRS emotion recognition method and system based on graph network and adaptive denoising.
The invention adopts the following technical scheme:
as shown in fig. 1, an fnrs emotion recognition method based on graph network and adaptive denoising includes the following steps:
s1 fNIRS acquisition equipment continuously acquires the change amount of light intensity before and after transmitting and receiving, converts the change amount of light intensity into the change of absorbance, and utilizes a beer-lambert law to calculate a relation equation of the change of absorbance and the change amount of concentration of a light absorption color group in brain tissue, mainly refers to oxyhemoglobin and deoxyhemoglobin, and further obtains the relative change amount of the concentration of the oxyhemoglobin and the deoxyhemoglobin through solving the equation.
The further process is as follows:
s1.1, acquiring the original near infrared light intensity changes of continuous waves with two different wavelengths from acquisition equipment, and recording the changes asAnd->
S1.2 converting the original light intensity information into absorbanceAnd->
S1.3, solving a relation equation of absorbance change and relative change quantity of concentration of light-absorbing color groups in brain tissues according to a beer-lambert law;
wherein epsilon is a molar extinction coefficient, d is a detection depth, and DPF is a differential path factor;
s1.4, obtaining the relative change quantity of the concentration of the oxyhemoglobin and the deoxyhemoglobin through solving an equation, and marking the relative change quantity as delta C HbO (t) and ΔC HbR (t)。
S2, denoising through a self-adaptive denoising network model to obtain a pure signal, wherein an input signal of the self-adaptive denoising network model is the relative variation of the concentration of the oxyhemoglobin and the deoxyhemoglobin obtained in the previous step, and an output signal is the data of the relative variation of the concentration of the pure oxyhemoglobin and the deoxyhemoglobin.
The method is characterized by further comprising the following steps:
as shown in FIG. 2, the adaptive denoising network model comprises a plurality of convolution and deconvolution block depth convolution countermeasure pairs with different sizes, which are marked as G p Wherein the input is noisy data ΔC HbO (t) and ΔC HbR (t) generating network output as relative variation data of pure oxyhemoglobin and deoxyhemoglobin concentration, noted asAnd
the difference between the output and the input of the denoising network model is the generated pure noise signal, which is recorded as
Pure noise signal to be generatedAnd clean data P HbO (t) and P HbR (t) adding to obtain the generated noisy data, denoted +.>And->
Note deltac HbO (t),ΔC HbR (t) is DeltaC, P HbO (t),P HbR (t) is P, then the total loss of the model is defined as:
wherein,loss as noise discriminant:
loss as pure discriminant:
for cycle consistency loss:
after iterative training, the denoising algorithm is as follows:
P HbO (t),P HbR (t)＝G p (ΔC HbO (t),ΔC HbR (t)) (8)
can obtain the relative variation data P of pure oxyhemoglobin and deoxyhemoglobin concentration HbO (t) and P HbR (t)。
And S3, mapping graph nodes by combining the probe and channel characteristics, restoring brain topology by using a graph network, and classifying and outputting emotion labels through a dynamic graph attention emotion recognition network model. The dynamic graph attention emotion recognition network model comprises graph convolution and an attention mechanism, and the probe is an optode of fNIRS and comprises a transmitter and a receiver.
As shown in fig. 3, the specific process is as follows:
s3.1 first defines a graph, denoted G (V, E, W), where V represents a combination of graph nodes, |v|=n represents a total of n nodes, corresponding to the data sequence Δc of n channels of fNIRS HbO (t),ΔC HbR (t) is denoted as X; e represents a set of edges in the graph, and a set of different channels in the fNIRS; w is an adjacency matrix, the connection relation of the nodes is defined, namely the relevance of different channels in the fNIRS is defined, wherein the values in the adjacency matrix describe the relation importance among the nodes, and the value W ij The initialization method uses a gaussian kernel function method:
wherein dist is the Gaussian distance between nodes, and θ and τ are fixed parameters in the Gaussian distance algorithm.
S3.2, in order to draw attention of the graph, calculating the similarity coefficient e of each node and the adjacent nodes ij ：
Where a is the global mapping matrix,and->Respectively a weight matrix of a node i and a weight matrix of a node j;
s3.3 calculating and normalizing the attention coefficients among the nodes of the graph, and marking the attention coefficients as alpha ij ：
Wherein, leakyReLU () is a nonlinear activation function;
s3.4, in graph convolution, weighting and summing by utilizing multiple heads of attention to perform parameter integration to obtain new feature X' i ：
In the formula, sigma is a nonlinear mapping relation, K is the number of attention heads, and I represents splicing;
s3.5, pooling dimension reduction is carried out, emotion categories are output through a classifier after flattening and fully connecting layers, and a model frame is shown in figure 2;
s3.6, the model adopts cross entropy plus a regularization term as a loss function, and is expressed as:
where costentropy () represents the cross entropy calculation, l,respectively a real label and a predicted value, +.>For learning rate->For representing all parameters of the model;
s3.7, adopting a back propagation algorithm to realize dynamic change of the adjacent matrix, calculating a loss function, and carrying out iterative update of the network on partial differentiation of the adjacent matrix:
the iterative update formula is:
an fnrs emotion recognition system based on graph network and adaptive denoising, comprising:
the fNIRS acquisition module: continuously collecting the variation of the light intensity before and after transmitting and receiving by using an fNIRS collecting device, converting the variation of the light intensity into the variation of absorbance, and further obtaining the relative variation of the concentration of oxyhemoglobin and deoxyhemoglobin;
the fNIRS adaptive denoising network module: for obtaining a clean signal;
the fNIRS dynamic diagram attention emotion recognition network module: and outputting the emotion label according to the clean signal.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the fNIRS emotion recognition method when the computer program is executed.
A storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of an fNIRS emotion recognition method.
The invention has the beneficial effects that:
(1) In the denoising process of the data, an adaptive denoising model based on the generation of an countermeasure network is adopted. Compared with the traditional data denoising method based on machine learning, the method can avoid manual participation and experience analysis in the data denoising process to a great extent, overcomes the defect of strong task dependence of the traditional method, and has high self-adaptability under the multitasking condition. Meanwhile, by generating an countermeasure learning mode, the noise difference problem of the fNIRS in the static task and the dynamic task can be solved without specific assumption, and the denoising model has stronger generalization capability.
(2) In the emotion characterization extraction of the fnrs signal, a deep-learned network model is used. Compared with the traditional manual feature extraction method, the method can realize the extraction of the emotion features in the fNIRS through the network learning in a data driving mode, and overcomes the problems of limited dimensionality, uncertainty of effectiveness and the like in the calculation of the fixed features in the traditional method. Features are learned and extracted through a deep learning network, emotion characterizations with different dimensions can be effectively obtained, and extraction and utilization of the emotion features in the fNIRS data are enhanced.
(3) The dynamic graph convolutional neural network is used to efficiently model fnrs data with probe location information and channel signals. Compared with the traditional method that data is simply regarded as a time series signal and is analyzed by using a machine learning method based on a support vector machine and a Bayesian classifier or a cyclic neural network based on long-term and short-term memory, the invention provides a method for topological mapping fNIRS data by using a graph, different probes are mapped into nodes in the graph, the time series data are characteristics of the nodes, and different passing relations are represented as edges in the graph through an adjacency matrix. The method fully utilizes the characteristics of the data, has reducibility on the topology of the brain structure, simultaneously characterizes the data relevance of different channels, and can improve the accuracy of the network model in emotion recognition of fNIRS brain signal detection.
(4) The method introduces a graph attention mechanism, obtains attention coefficients among different nodes by calculating similarity coefficients of different probe nodes and adjacent nodes, and updates node characteristics by using weighted summation of a multi-head attention mechanism in the graph convolution process. According to the method, the feature connection of the adjacent nodes of the nodes can be introduced in the training process of the model, so that the model can better extract the associated features of different channel data in the fNIRS, and meanwhile, the activation response of different brain areas to different emotions can be obtained, so that the method has a remarkable effect in emotion recognition of brain signals.
Drawings
FIG. 1 is a workflow diagram of the present invention;
FIG. 2 is a block diagram of an fNIRS adaptive denoising network model of the present invention;
FIG. 3 is a diagram of the structure of the fNIRS dynamic diagram attention emotion recognition network model of the present invention;
FIG. 4 is a schematic diagram of an fNIRS acquisition module of the invention.
Detailed Description
The present invention will be described in further detail with reference to examples and drawings, but embodiments of the present invention are not limited thereto.
Example 1
The fNIRS emotion recognition method based on the graph network and the adaptive denoising is suitable for emotion recognition tasks of fNIRS acquisition equipment and mainly comprises an external emotion stimulation step, an fNIRS acquisition step, an fNIRS adaptive denoising step and an fNIRS emotion recognition step.
And an external emotion stimulation step, wherein the external emotion stimulation material adopts video of six emotion label types, namely anger, aversion, fear, pleasure, sadness and surprise, and a user carries out emotion induction by watching the video.
As shown in fig. 4, the fnrs acquisition step, the fnrs acquisition device consists of a multi-channel dual-wavelength near infrared continuous wave transmitting-receiving source.
Firstly, a user wears the fNIRS acquisition device to acquire and record the variation of the light intensity of different light poles before and after transmitting and receiving in real time.
Let the single channel light intensity variation at any moment be:and->
The calculated conversion to absorbance was:
according to the beer-lambert law: Δa=ε×d×dpf, where ε is a molar extinction coefficient, d is a detection depth, and DPF is a differential path factor.
Obtaining the relative change of the concentration of oxyhemoglobin and deoxyhemoglobin, which is marked as delta C, by solving a relation equation of the absorbance change and the relative change of the concentration of the absorbance chromophore in brain tissue HbO And DeltaC HbR 。
fNIRS adaptive denoising step
Inputting the data into an fNIRS self-adaptive denoising module for denoising enhancement, and obtaining pure relative variation data P of oxyhemoglobin and deoxyhemoglobin concentration through a trained generation network HbO And P HbR 。P HbO ,PHbR＝G p (ΔC HbO ,ΔC HbR )。
The fNIRS self-adaptive denoising module is based on a generating countermeasure network, performs feature extraction on fNIRS signals through convolution, and performs pure signal generation through deconvolution. In the training process of the network, paired noisy signals and clean signals are input, training is carried out through the countermeasures of the two discriminators, meanwhile, constraint of space mapping is carried out by introducing the cyclic consistency loss, and training efficiency of the model is improved.
fNIRS emotion recognition step
Map node mapping is carried out on pure data of all channels: p (P) HbO ,P HbR V, V is a graphA collection of nodes. Initializing the adjacency matrix W by using a Gaussian kernel function, wherein the Gaussian kernel function is:
wherein dist is the Gaussian distance between nodes, and θ and τ are fixed parameters in the Gaussian distance algorithm.
Calculating similarity coefficient of each node and adjacent nodesa is a global mapping matrix,>and->The weight matrix of node i and node j, respectively.
Normalizing graph node attention coefficients to alpha using a LeakyReLU nonlinear activation function ij ，
Obtaining new characteristics by weighting and splicing multi-head attention parameters through graph convolution σ is a nonlinear mapping relationship, K is the number of attention headers, || represents stitching.
And carrying out pooling dimension reduction, and outputting emotion recognition probability through a classifier after flattening and fully connecting layers.
In this embodiment, the emotion recognition probability is a probability value of anger, aversion, happiness, sadness, surprise six Ekerman emotion classification labels, and
the emotion recognition method based on the physiological signals is mainly based on brain electrical signals and functional magnetic resonance, and the emotion recognition method based on the functional near infrared spectrum technology fully exploits the effect and potential of the novel non-invasive brain detection method in emotion research, has great significance in practical application, and simultaneously opens up a novel emotion analysis method of the physiological signals.
The signal denoising method adopted by the method can realize end-to-end self-adaptive denoising of the multichannel fNIRS signals, and the algorithm has higher generalization capability and universality.
The invention provides a dynamic graph-based attention model, which adopts a dynamic graph convolution method to construct brain topology and extract features, introduces an attention mechanism to extract relevant features among fNIRS channels, improves the learning ability of the model and obtains higher emotion recognition accuracy.
The invention adopts a deep learning method, extracts the characteristics by data driving, improves the expression capability of the emotion characteristics and simultaneously avoids artificial participation.
Example 2
An fnrs emotion recognition system based on graph network and adaptive denoising, comprising:
the fNIRS acquisition module: continuously collecting the variation of the light intensity before and after transmitting and receiving by using an fNIRS collecting device, converting the variation of the light intensity into the variation of absorbance, and further obtaining the relative variation of the concentration of oxyhemoglobin and deoxyhemoglobin;
the fNIRS adaptive denoising network module: for obtaining a clean signal;
the fNIRS dynamic diagram attention emotion recognition network module: and outputting the emotion label according to the clean signal.
Example 3
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the fNIRS emotion recognition method when the computer program is executed.
Example 4
A storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of an fNIRS emotion recognition method.
The embodiments described above are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the embodiments described above, and any other changes, modifications, substitutions, combinations, and simplifications that do not depart from the spirit and principles of the present invention should be made in the equivalent manner, and are included in the scope of the present invention.
Claims (5)
1. The method for identifying the fNIRS emotion based on the graph network and the self-adaptive denoising is characterized by comprising the following steps of:
the fNIRS acquisition equipment continuously acquires the variation of the light intensity before and after transmitting and receiving, converts the variation of the light intensity into the variation of absorbance, and further obtains the relative variation of the concentration of oxyhemoglobin and deoxyhemoglobin;
denoising through a self-adaptive denoising network model to obtain a pure signal, wherein the input signal of the self-adaptive denoising network model is the relative variation of the concentration of the oxygenated hemoglobin and the concentration of the deoxygenated hemoglobin obtained in the previous step, and the output signal is the data of the relative variation of the concentration of the pure oxygenated hemoglobin and the concentration of the deoxygenated hemoglobin;
mapping graph nodes by combining the probe and channel characteristics, restoring brain topology by using a graph network, and classifying and outputting emotion labels by using a dynamic graph attention emotion recognition network model;
the probe is an optode of fNIRS and comprises a transmitting party and a receiving party;
the self-adaptive denoising network model comprises a plurality of depth convolution countermeasure pairs formed by convolution and deconvolution blocks with different sizes;
performing convolution to extract features of the fNIRS signal, and generating a pure signal through a deconvolution block;
the dynamic graph attention emotion recognition network model comprises graph convolution and an attention mechanism;
the identification process of the dynamic graph attention emotion identification network model is as follows: constructing a graph network, mapping data to a graph, mapping a pure fNIRS signal to a node of the graph, mapping a probe and channel characteristic feature to an edge of the graph, extracting the feature by a dynamic graph convolution method, introducing an attention mechanism to learn the channel relevance, and finally classifying and outputting by dimension reduction and flattening splicing of the graph pooling, so that the emotion is accurately identified;
the emotion labels are classified and output through the dynamic graph attention emotion recognition network model, specifically:
definition of the graph, denoted G (V, E, W), where V represents a set of graph nodes, |v|=n represents a total of n nodes, corresponding to a data sequence Δc of n lanes of fnigs HbO (t)，ΔC HbR (t)，ΔC HbO (t)，ΔC HbR (t) each represents a change in concentration of oxyhemoglobin and deoxyhemoglobin at time t; e represents a set of edges in the graph, i.e., a set of different channels in the fNIRS; w is an adjacency matrix, defining connection relation of nodes, namely relevance of different channels in the fNIRS, wherein the value W in the adjacency matrix ij Describing the importance of the relationship between nodes, the value w in the adjacency matrix ij The initialization method uses a gaussian kernel function method, wherein i and j represent nodes;
drawing attention to the graph, and calculating similarity coefficients of each node and adjacent nodes;
calculating attention coefficients among the nodes of the graph and normalizing;
in graph convolution, weighting and summing are carried out by utilizing multiple heads of attention to carry out parameter integration, so that new characteristics are obtained;
and carrying out pooling dimension reduction, and outputting emotion categories through a classifier after flattening and fully connecting layers.
2. The method for recognizing fNIRS according to claim, wherein a relational equation of absorbance change and concentration change of light absorbing color group in brain tissue is obtained by using beer-lambert law, and the relative change of oxyhemoglobin and deoxyhemoglobin concentration is obtained by solving the equation.
3. A system for implementing the fNIRS emotion recognition method of any one of claims 1 to 2, comprising:
the fNIRS acquisition module: continuously collecting the variation of the light intensity before and after transmitting and receiving by using an fNIRS collecting device, converting the variation of the light intensity into the variation of absorbance, and further obtaining the relative variation of the concentration of oxyhemoglobin and deoxyhemoglobin;
the fNIRS adaptive denoising network module: for obtaining a clean signal;
the fNIRS dynamic diagram attention emotion recognition network module: and outputting the emotion label according to the clean signal.
4. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the fNIRS emotion recognition method of any one of claims 1 to 2 when the computer program is executed by the processor.
5. A storage medium having stored thereon a computer program, which when executed by a processor, implements the steps of the fNIRS emotion recognition method of any one of claims 1 to 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111315105.2A CN114209319B (en) | 2021-11-08 | 2021-11-08 | fNIRS emotion recognition method and system based on graph network and self-adaptive denoising |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111315105.2A CN114209319B (en) | 2021-11-08 | 2021-11-08 | fNIRS emotion recognition method and system based on graph network and self-adaptive denoising |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114209319A CN114209319A (en) | 2022-03-22 |
| CN114209319B true CN114209319B (en) | 2024-03-29 |
Family
ID=80696655
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111315105.2A Active CN114209319B (en) | 2021-11-08 | 2021-11-08 | fNIRS emotion recognition method and system based on graph network and self-adaptive denoising |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114209319B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114782449B (en) * | 2022-06-23 | 2022-11-22 | 中国科学技术大学 | Method, system, equipment and storage medium for extracting key points in lower limb X-ray image |
| CN117156072B (en) * | 2023-11-01 | 2024-02-13 | 慧创科仪(北京)科技有限公司 | Device for processing near infrared data of multiple persons, processing equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107280685A (en) * | 2017-07-21 | 2017-10-24 | 国家康复辅具研究中心 | Top layer physiological noise minimizing technology and system |
| CN111466876A (en) * | 2020-03-24 | 2020-07-31 | 山东大学 | Alzheimer's disease auxiliary diagnosis system based on fNIRS and graph neural network |
| WO2020166091A1 (en) * | 2019-02-15 | 2020-08-20 | 俊徳 加藤 | Biological function measurement device, and biological function measurement method, and program |
| WO2021067464A1 (en) * | 2019-10-01 | 2021-04-08 | The Board Of Trustees Of The Leland Stanford Junior University | Joint dynamic causal modeling and biophysics modeling to enable multi-scale brain network function modeling |
| CN113180650A (en) * | 2021-01-25 | 2021-07-30 | 北京不器科技发展有限公司 | Near-infrared brain imaging atlas identification method |
| KR102288267B1 (en) * | 2020-07-22 | 2021-08-11 | 액티브레인바이오(주) | AI(Artificial Intelligence) BASED METHOD OF PROVIDING BRAIN INFORMATION |
| CN113598774A (en) * | 2021-07-16 | 2021-11-05 | 中国科学院软件研究所 | Active emotion multi-label classification method and device based on multi-channel electroencephalogram data |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4590555B2 (en) * | 2004-09-02 | 2010-12-01 | 国立大学法人長岡技術科学大学 | Sensitive state discrimination method and apparatus |
| US10682081B2 (en) * | 2015-12-21 | 2020-06-16 | Outerfacing Technology LLC | Acquiring and processing non-contact functional near-infrared spectroscopy data |
| JP7080657B2 (en) * | 2018-02-07 | 2022-06-06 | 株式会社デンソー | Emotion identification device |
-
2021
- 2021-11-08 CN CN202111315105.2A patent/CN114209319B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107280685A (en) * | 2017-07-21 | 2017-10-24 | 国家康复辅具研究中心 | Top layer physiological noise minimizing technology and system |
| WO2020166091A1 (en) * | 2019-02-15 | 2020-08-20 | 俊徳 加藤 | Biological function measurement device, and biological function measurement method, and program |
| WO2021067464A1 (en) * | 2019-10-01 | 2021-04-08 | The Board Of Trustees Of The Leland Stanford Junior University | Joint dynamic causal modeling and biophysics modeling to enable multi-scale brain network function modeling |
| CN111466876A (en) * | 2020-03-24 | 2020-07-31 | 山东大学 | Alzheimer's disease auxiliary diagnosis system based on fNIRS and graph neural network |
| KR102288267B1 (en) * | 2020-07-22 | 2021-08-11 | 액티브레인바이오(주) | AI(Artificial Intelligence) BASED METHOD OF PROVIDING BRAIN INFORMATION |
| CN113180650A (en) * | 2021-01-25 | 2021-07-30 | 北京不器科技发展有限公司 | Near-infrared brain imaging atlas identification method |
| CN113598774A (en) * | 2021-07-16 | 2021-11-05 | 中国科学院软件研究所 | Active emotion multi-label classification method and device based on multi-channel electroencephalogram data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114209319A (en) | 2022-03-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Chen et al. | Accurate EEG-based emotion recognition on combined features using deep convolutional neural networks | |
| CN107491726B (en) | Real-time expression recognition method based on multichannel parallel convolutional neural network | |
| CN114209319B (en) | fNIRS emotion recognition method and system based on graph network and self-adaptive denoising | |
| CN112800998B (en) | Multi-mode emotion recognition method and system integrating attention mechanism and DMCCA | |
| CN112244873A (en) | Electroencephalogram time-space feature learning and emotion classification method based on hybrid neural network | |
| CN110367967A (en) | A kind of pocket lightweight human brain condition detection method based on data fusion | |
| CN114533086B (en) | Motor imagery brain electrolysis code method based on airspace characteristic time-frequency transformation | |
| Zhong et al. | Cross-scene deep transfer learning with spectral feature adaptation for hyperspectral image classification | |
| CN111523601A (en) | Latent emotion recognition method based on knowledge guidance and generation counterstudy | |
| CN113343860A (en) | Bimodal fusion emotion recognition method based on video image and voice | |
| CN111067507A (en) | Electrocardiosignal denoising method based on generation of countermeasure network and strategy gradient | |
| CN111297380A (en) | Emotion recognition method based on space-time convolution core block | |
| Rayatdoost et al. | Subject-invariant EEG representation learning for emotion recognition | |
| Jinliang et al. | EEG emotion recognition based on granger causality and capsnet neural network | |
| Jayasekara et al. | Timecaps: Capturing time series data with capsule networks | |
| CN115169386A (en) | Weak supervision increasing activity identification method based on meta-attention mechanism | |
| Tang et al. | Eye movement prediction based on adaptive BP neural network | |
| CN115444420A (en) | CCNN and stacked-BilSTM-based network emotion recognition method | |
| Truong et al. | Assessing learned features of Deep Learning applied to EEG | |
| Serkan et al. | VarioGram–A colorful time-graph representation for time series | |
| Jamal et al. | Cloud-Based Human Emotion Classification Model from EEG Signals | |
| Vibhute et al. | Color image processing approach for nitrogen estimation of vineyard | |
| Kuang et al. | Remote photoplethysmography signals enhancement based on generative adversarial networks | |
| Özel | A compass for using deep learning in emotion recognition via brain signals | |
| Du et al. | Multivariate time series classification based on fusion features |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |