CN114676720A - Psychological state identification method and system based on graph neural network - Google Patents

Psychological state identification method and system based on graph neural network Download PDF

Info

Publication number
CN114676720A
CN114676720A CN202210187337.2A CN202210187337A CN114676720A CN 114676720 A CN114676720 A CN 114676720A CN 202210187337 A CN202210187337 A CN 202210187337A CN 114676720 A CN114676720 A CN 114676720A
Authority
CN
China
Prior art keywords
neural network
signals
graph
electroencephalogram
graph neural
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210187337.2A
Other languages
Chinese (zh)
Inventor
杨阳
李洋
王洪君
刘云霞
张志风
李泽洲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong University
Original Assignee
Shandong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong University filed Critical Shandong University
Priority to CN202210187337.2A priority Critical patent/CN114676720A/en
Publication of CN114676720A publication Critical patent/CN114676720A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/08—Feature extraction
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Theoretical Computer Science (AREA)
  • Psychiatry (AREA)
  • Biophysics (AREA)
  • Surgery (AREA)
  • Public Health (AREA)
  • Pathology (AREA)
  • Veterinary Medicine (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Animal Behavior & Ethology (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Psychology (AREA)
  • Fuzzy Systems (AREA)
  • Physiology (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Child & Adolescent Psychology (AREA)
  • Developmental Disabilities (AREA)
  • Educational Technology (AREA)
  • Hospice & Palliative Care (AREA)
  • Social Psychology (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

The invention provides a mental state identification method and a system based on a graph neural network, which comprises the following steps: preprocessing the acquired electroencephalogram signals, and decomposing the extracted electroencephalogram signals into a plurality of electroencephalogram signals by using an empirical mode decomposition algorithm; extracting multidimensional statistical characteristics from the decomposed electroencephalogram signals; processing the multidimensional statistical characteristics by using a graph neural network, and extracting the connection relation between graph characteristics and signals among the multidimensional statistical characteristics; and compressing the connection relation between the multi-dimensional image characteristic signals and the multi-dimensional channel signals to obtain the classification result of the electroencephalogram signals. The invention uses the dynamic graph neural network method, can better depict the connection relation of the electroencephalogram signals, and more truly reflects the change characteristics of the electroencephalogram signals based on space.

Description

Psychological state identification method and system based on graph neural network
Technical Field
The invention belongs to the technical field of biological signal processing, and particularly relates to a mental state identification method and system based on a graph neural network.
Background
The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.
Psychological state classification is the manifestation of the attitude of a subject when confronted with different objective things, which produces an emotion in response to the subject's cognition and assessment of the situation when confronted with stimuli of emotion-inducing material. The research on the emotion can enable a researcher to know the emotion state change of a testee in time, and carry out multi-directional assessment to obtain the emotion state and feed back the emotion state in time. Identification of mental states can be applied to a number of aspects, such as: network user emotion description, patient psychological state assessment, psychological state analysis of supervisors in key supervision places and the like. The psychological state is a phenomenon of psychology and physiology which changes instantly, and timely and accurately analyzing the change of emotion has important significance for the survival of human bodies and the stable development of society.
Identification of mental states can be divided into two categories: one is a method of emotion recognition based on non-biological signals, and the other is a method of recognizing signals based on psychological states of physiological signals.
The psychological state recognition method based on the non-biological signals is mainly based on non-biological signals such as voice tone, facial expression and posture. With the development of computer vision technology and artificial intelligence algorithm technology, facial expressions are often used for emotion recognition algorithm, firstly, images are preprocessed, hidden features in the images are extracted, and the images are input into a support vector machine or a deep learning model to obtain classification results. The non-biological signals when the psychological state of the testee is induced are recorded, the emotion change is obtained through an artificial intelligence algorithm, but the characteristics of human expression, posture and the like can be subjectively controlled by the testee, the characteristics are concealed, the emotional state of the testee cannot be comprehensively expressed based on the non-biological signals, and the detection accuracy is extremely low.
The physiological signal based mental state recognition algorithm is based on brain signals and peripheral nervous system and brain signals like respiration, blood pressure, pulses, etc. Physiological signals are less likely to be controlled by the subject than non-physiological signals, and are more reliable than non-physiological signals. Lie detectors like those used in criminal investigation mainly determine whether a subject lies by the change of physiological signals and obtain considerable effects. Generally, the collected signals are preprocessed, and features are extracted, and the extracted features are classified using a relevant classification algorithm and a classification result of a psychological state is output. However, there is no theoretic problem in the current psychological state recognition and classification detection of signals due to the change rule between sub-signals.
With the increasing attention of psychological state recognition, no consideration is given to information interaction among the sub-signals during the psychological state recognition, and therefore, the psychological state recognition cannot be accurately realized.
Disclosure of Invention
To overcome the above-mentioned deficiencies of the prior art, the present invention provides a mental state recognition method based on a graph neural network for describing emotional changes of a subject when the subject is stimulated.
To achieve the above object, one or more embodiments of the present invention provide the following technical solutions:
in a first aspect, a mental state recognition method based on a graph neural network is disclosed, which comprises the following steps:
preprocessing the acquired electroencephalogram signals, and decomposing the extracted electroencephalogram signals into a plurality of electroencephalogram signals by using an empirical mode decomposition algorithm;
extracting multidimensional statistical characteristics from the decomposed electroencephalogram signals;
processing the multidimensional statistical characteristics by using a graph neural network, and extracting the connection relation between graph characteristics and signals among the multidimensional statistical characteristics;
and compressing the connection relation between the multi-dimensional image characteristic signal and the multi-dimensional channel signal to obtain a classification result of the electroencephalogram signal.
According to the further technical scheme, the preprocessing is carried out on the acquired electroencephalogram signals, and the preprocessing specifically comprises the following steps:
decomposing the electroencephalogram signals into a limited number of intrinsic mode functions by using an empirical mode decomposition algorithm;
each eigenmode function should satisfy two conditions:
firstly, the number of extreme points and zero crossing points is equal, or the difference between the two is not more than 1;
second, the average of the local maximum envelope and the local minimum envelope is zero.
In the further technical scheme, in the representation of the graph, a plurality of intrinsic mode functions are selected as nodes;
In constructing the graph, the number of eigenmode functions selected is denoted as N, and is expressed as:
N＝min(Si)
wherein SiIs the number of IMFs obtained using empirical mode decomposition on the ith signal.
According to the further technical scheme, each intrinsic mode function component is divided into m clusters, and the size of each cluster can be automatically modified along with the change of a sequence;
defining the cluster data satisfies the following formula:
Figure BDA0003523241850000031
wherein T ismDuration of each cluster, TxRepresenting the duration of the segmentation of the pre-electroencephalogram signal.
According to the further technical scheme, the decomposed electroencephalogram signals are subjected to multi-dimensional statistical feature extraction, and the method specifically comprises the following steps:
and extracting 12 statistical characteristics from each cluster data, wherein the characteristics are key characteristics representing the whole data set, and the characteristic set is { maximum value, average value, median, upper quartile, variation, kurtosis, minimum value, modulus, value range, standard deviation, skewness and lower quartile }.
In the further technical scheme, in the neural network of the graph, the graph is represented as
Figure BDA0003523241850000032
Wherein
Figure BDA0003523241850000033
And epsilon is defined as a set of nodes and a set of edges,
Figure BDA0003523241850000034
can use a feature matrix
Figure BDA0003523241850000035
Represents;
Figure BDA0003523241850000036
the number of nodes in the graph is shown, and d is the input feature dimension;
a is a adjacency matrix describing the structure of the graph,
Figure BDA0003523241850000037
if the node i is connected with the node j, A (i, j) ≠ 0, and A (i, j) represents the functional relation between the node i and the node j.
In the further technical scheme, the adjacency matrix in the graph neural network is obtained by training the whole network, the connection mode of the adjacency matrix is not set in advance, but a limiting condition needs to be set for the adjacency matrix in the training process;
the formula of the graph neural network is as follows:
Figure BDA0003523241850000038
wherein, sigma is an activation function,
Figure BDA0003523241850000039
is a contiguous matrix of the neighbors,
Figure BDA00035232418500000310
is a degree matrix.
In a further technical scheme, the input of the graph neural network is represented as:
Figure BDA0003523241850000041
in a second aspect, a mental state recognition system based on a graph neural network is disclosed, comprising:
a brain electrical signal pre-processing module configured to: preprocessing the acquired electroencephalogram signals, and decomposing the extracted electroencephalogram signals into a plurality of electroencephalogram signals by using an empirical mode decomposition algorithm;
a signal feature extraction module configured to: extracting multidimensional statistical characteristics from the decomposed electroencephalogram signals;
a graph neural network module configured to: processing the multidimensional statistical characteristics by using a graph neural network, and extracting the connection relation between graph characteristics and signals among the multidimensional statistical characteristics;
a feature dimension reduction module configured to: and compressing the connection relation between the multi-dimensional image characteristic signals and the multi-dimensional channel signals to obtain the classification result of the electroencephalogram signals.
The above one or more technical solutions have the following beneficial effects:
the method of the invention uses the graph neural network to classify and predict the psychological state. In the prior algorithm, multi-dimensional electroencephalogram signals are regarded as multi-dimensional parallel data, and machine learning algorithm is used for electroencephalogram analysis, so that the multi-dimensional electroencephalogram signals are subjected to rough fusion. However, in this way, the connection relation and the spatial relation among the multi-dimensional electroencephalogram signals are not analyzed, and by the method, the adjacent matrix describing the connection relation among the signals is obtained by the training of the neural network of the image, so that the change characteristics of the multi-dimensional electroencephalogram signals based on the space can be accurately described.
The data preprocessing and feature extraction part of the invention further expands the dimensionality of electroencephalogram data by using an empirical mode decomposition algorithm, and describes the difference and regularity between different electroencephalograms by extracting multi-dimensional statistical features, and the statistical features can better describe the change rule of the multi-dimensional electroencephalograms in different time periods.
The invention can reduce the dimension of the graph characteristics generated by the graph neural network and output the result of generating the psychological state by characteristic dimension reduction, has simple characteristic dimension reduction and can be converted based on tasks, and can be suitable for analysis of various psychological states.
Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and together with the description serve to explain the invention and not to limit the invention.
FIG. 1 is a schematic diagram of a signal preprocessing and feature extraction module segmenting data according to the present invention;
FIG. 2 is a schematic diagram showing the connection relationship between modules in the system;
FIG. 3 is a schematic diagram of data preprocessing;
FIG. 4 is a schematic diagram of the neural network computation;
FIG. 5 is a schematic diagram of a feature dimension reduction process;
Detailed Description
It is to be understood that the following detailed description is exemplary and is intended to provide further explanation of the invention as claimed. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the invention.
The embodiments and features of the embodiments of the invention may be combined with each other without conflict.
Interpretation of terms:
1. empirical mode decomposition: the data decomposition is based on the time scale characteristics of the signals, and different from Fourier decomposition and wavelet decomposition, a basis function needs to be established in advance, so that the method can be applied to the decomposition of any type of signals, and has wide application in various fields.
2. Graph neural network: with the rise of deep learning, the graph neural network is proposed to be applied to the processing of non-Euclidean spatial data, and the research of the graph neural network is closely related to the development of graph embedding and graph theory.
Example one
The embodiment discloses a mental state recognition method based on a graph neural network, which is used for carrying out mental state classification and prediction and is used for describing emotional changes of a subject when the subject is stimulated, and the method comprises the following steps:
step 1: and (4) analyzing the mental state based on the electroencephalogram signals. The electroencephalogram signals need to be preprocessed, and empirical mode decomposition is used for decomposing the signals. The number of IMFs after decomposition is determined by the characteristics of the signals, the number of sub-signals generated by different signals is different, signals with the same dimensionality are selected in the subsequent process, and different signals are decomposed by using an empirical mode.
And 2, step: and (3) carrying out data division on the decomposed multi-dimensional electroencephalogram signals, and extracting 12 statistical features from each divided data, wherein the features are key features representing the whole data set.
And step 3: and (3) using a graph neural network model to obtain an adjacency matrix through training so as to represent the relevance between the multidimensional signals. Regarding each signal as a node in the graph, edges between each signal are expressed as connection relations between the nodes, and the range of each edge is [0,1 ].
The outputs of the multi-layer graph neural networks are connected together and are used as the output of the graph neural network module, high-dimensional graph features are obtained from the high-layer graph neural network, and low-dimensional graph features are obtained from the bottom-layer graph neural network.
And 4, step 4: and through multi-layer MLP, extracting features from the multi-dimensional graph features output by the computation graph neural network module, and outputting a psychological state through dimension reduction.
The connection relation between the brain electrical signals is not set in advance, but is obtained by training as a part of the parameters of the neural network of the image, so the neural network of the dynamic image is called.
In step 1, signal preprocessing is performed
This step is the process of preprocessing the electroencephalogram signal, and the disclosed mental state data set is used for training a mental state analysis model. Empirical mode decomposition algorithms provide a new mathematical tool for nonlinear and non-stationary signal analysis. The electroencephalogram signals are decomposed into a limited number of Intrinsic Mode Functions (IMFs) by using an empirical mode decomposition algorithm, and the number of the electroencephalogram signals is expanded.
Each IMF should satisfy two conditions. First, the number of extreme points and zero-crossing points is equal, or the difference between the two is not more than 1. Second, the average of the local maximum envelope and the local minimum envelope is zero.
The number of the decomposed intrinsic mode functions IMF is determined by the characteristics of the electroencephalogram signals, the number of the intrinsic mode functions generated by different electroencephalogram signals is different, and the signals with the same dimensionality are selected in the subsequent process.
Each IMF is considered as a node of the graph, and the connection relationship between the nodes is obtained by training the graph neural network.
In the representation of the graph, several IMFs are selected as nodes. When selecting the number of IMFS, the amount of unwanted information is minimized and the original signal information is retained to the maximum. In constructing the graph, the number of IMFs selected is denoted as N, and is represented as:
N＝min(Si)
wherein S isiIs the number of IMFs obtained by using an empirical mode decomposition algorithm on the ith electroencephalogram signal.
In step 2, feature extraction is performed
Each intrinsic mode function is IMF, a plurality of intrinsic mode functions can be generated through EMD decomposition, data segmentation and statistical feature extraction are carried out in different IMFs, statistical features contain most useful information in original electroencephalogram data, and the original electroencephalogram data are reduced to a lower dimensional number. The invention extracts 12 statistical features from each cluster data, which are key features representing the entire data set. The feature set is { maximum, mean, median, upper quartile, variation, kurtosis, minimum, modulus, range, standard deviation, skewness, lower quartile }.
It should be noted that, in order to reduce the dimensionality of each IMF component, the present invention improves the clustering technique for feature extraction. Each eigenmode function is segmented into a plurality of clusters, from each of which a number of statistical features are calculated in order to describe brain activity from the acquired brain electrical data. The electroencephalogram data are divided into a plurality of clusters, so that subsequent feature extraction is facilitated, and the features represent the input of corresponding nodes. In fig. 1, a schematic diagram of the segmentation and statistical feature extraction process from different IMFs components is shown. Each IMF component is empirically divided into m clusters. The size of each cluster is modified according to the length of the extracted brain electricity, and the length of each cluster is not strictly limited. In order to realize the real-time performance and the accuracy of electroencephalogram detection, clustering data are defined to meet the following formula:
Figure BDA0003523241850000081
wherein, TmFor the duration of each cluster. T isxRepresenting the duration of the segmentation of the pre-electroencephalogram signal. H is the number of maxima in the time series, and the above formula is used to describe the range of the corresponding time length of each cluster after dividing the data.
Feature extraction plays an important role in extracting special patterns from raw data to achieve reliable classification. The statistical features contain most of the useful information in the raw data and reduce the raw data to a lower dimension. The present invention extracts 12 statistical features from each cluster data, which are key features representing the entire data set. The invention extracts a plurality of statistical characteristics from the multi-dimensional electroencephalogram signals so as to better represent the difference between the signals. The schematic diagram of the electroencephalogram signal preprocessing and feature extraction segmentation data is shown in fig. 3.
In step 3, the neural network is related to the graph
In order to obtain the correlation between the multi-dimensional signals, a graph neural network model is used for obtaining an adjacency matrix through training so as to represent the correlation between the multi-dimensional signals. Regarding each signal as a node in the graph, the edges between each signal are represented as the connection relationship between the nodes, and the range of each edge is [0,1 ].
The whole graph neural network is shown in figure 4 and is composed of multiple layers of graph convolutional neural network layers, and the output of each layer is used as the output of a module after the Concat operation is carried out on the output of each layer.
In order to obtain the difference between different psychological states, the outputs of the multilayer graph neural network are linked through concat operation and are used as the outputs of the graph neural network together, the high-dimensional graph features are obtained from the high-level graph neural network, and the low-dimensional graph features are obtained from the bottom-level graph neural network.
The invention uses the neural network of the graph to process the electroencephalogram for decomposed electroencephalogram signals, thereby improving the classification accuracy of the electroencephalogram. The development of the graph neural network is based on the fusion of deep learning and graph theory. Is shown as
Figure BDA0003523241850000082
Wherein
Figure BDA0003523241850000083
And ε is defined as the set of nodes and the set of edges.
Figure BDA0003523241850000084
Can use a feature matrix
Figure BDA0003523241850000085
And (4) showing.
Figure BDA0003523241850000086
The number of nodes in the graph, and d is the input feature dimension. A is a adjacency matrix describing the structure of the graph,
Figure BDA0003523241850000087
If node i is connected to node j, A (i, j) ≠ 0. A (i, j) characterisationThe functional relationship between node i and node j is shown.
The input of the graph neural network is graph data, and the following g is the graph structure of the graph data.
It is worth mentioning that the adjacency matrix in the graph neural network is obtained by training the whole network, the connection mode of the adjacency matrix is not set in advance, but the limitation condition still needs to be set for the adjacency matrix in the training process. The training process of the graph neural network can be summarized as an optimal description method of generating similarity measures of the respective subsignals. The formula of the graph neural network is as follows:
Figure BDA0003523241850000091
where σ is the activation function.
Figure BDA0003523241850000092
Is a contiguous matrix.
Figure BDA0003523241850000093
Is a degree matrix. The structure of the dynamic graph convolution neural network is shown in fig. 4.
The training of the neural network of the graph is the same as the training of the common neural network model, and the forward propagation and the backward propagation exist, and the details are not repeated here.
Having the concat operation merge each layer of outputs of the multi-layer graph neural network, as a whole output, the output of the graph neural network may be represented as:
Figure BDA0003523241850000094
where { x1,x2,…xi,…,xtAnd the input of the neural network layer of the ith layer diagram is input. { theta ]1,θ2,…,θtIs the network training parameter.
The invention obtains the optimal connection relation among the multi-dimensional electroencephalogram signals extracted according to the psychological state by using the graph neural network training, and extracts the high-dimensional and low-dimensional graph characteristics for the adjacency matrix obtained by the graph neural network training. The extracted features can represent not only linear relationships between signals but also spatial relationships between signals.
In step 4, dimension reduction is performed on the features
Through multi-layer MLP (referred to as multi-layer perceptron), features are extracted from multi-dimensional graph features output by the computational graph neural network, and a mental state is output through dimension reduction, as shown in fig. 5.
The invention adopts a processing mode based on a graph neural network algorithm, thus introducing the relationship among the sub-signals of a plurality of electroencephalograms to process the signals, learning through the graph neural network to obtain the similarity and the connection relationship among a plurality of signals, not introducing a fixed adjacency matrix, reversely improving the adjacency matrix by a prediction result, establishing the required connection relationship, improving the operation speed and the sensitivity of the algorithm on the basis of ensuring the accuracy of a training sample, saving manpower and material resources and having more robustness.
In order to extract the differentiated deep local features, feature dimension reduction is adopted to obtain a final emotion classification result. The MLP branch comprises two parts. The first part is to extract the structure represented between the different features in each node. The second part is a fully connected layer that outputs the mental states.
And reducing the feature dimension on each node by using a feature dimension reduction model, wherein the feature dimension reduction part consists of an input layer, two hidden layers and an output layer, and each layer comprises a learned filter and a nonlinear activation function. To reduce the occurrence of overfitting, dropout is employed to reduce the mutual application between neuron nodes. And outputting the emotion classification result through the full connection layer by the reconstructed node characteristics after dimensionality reduction.
The invention uses multilayer MLPs to reduce and fuse the extracted multi-dimensional graph characteristics and outputs the classification result of the psychological state. The method has better effect in classification and prediction tasks of the electroencephalogram signal data sets in different psychological states.
Example two
It is an object of the present embodiment to provide a computing device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the above method when executing the program.
EXAMPLE III
An object of the present embodiment is to provide a computer-readable storage medium.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method.
Example four
The present embodiment aims to provide a mental state recognition system based on a graph neural network, which includes:
a brain electrical signal pre-processing module configured to: preprocessing the electroencephalogram signals, and decomposing the extracted electroencephalogram signals into a plurality of electroencephalogram signals by using an empirical mode decomposition algorithm. The input is as follows: acquiring an electroencephalogram signal on electroencephalogram acquisition equipment; the output is: and decomposing the multi-dimensional electroencephalogram signals.
A signal feature extraction module configured to: and extracting multi-dimensional statistical characteristics from the decomposed electroencephalogram signals. The input is as follows: multi-dimensional electroencephalogram signals. The output is: a multi-dimensional signature.
A graph neural network module configured to: and extracting the connection relation between the graph characteristics and the signals among the multi-dimensional electroencephalogram signals. The input is as follows: a multi-dimensional signature. The output is: and connecting the multi-dimensional graph characteristic signal and the multi-dimensional channel signal.
A feature dimension reduction module configured to: compressing the characteristics in the multi-dimensional image signals and obtaining the classification result of the extracted electroencephalogram signals. The input is as follows: and connecting the multi-dimensional graph characteristic signal and the multi-dimensional channel signal. The output is: and (5) classifying the psychological states of the electroencephalogram signals.
The detailed connection relationship of the modules is shown in the attached figure 2 in the specification.
The steps involved in the apparatuses of the above second, third and fourth embodiments correspond to the first embodiment of the method, and the detailed description thereof can be found in the relevant description of the first embodiment. The term "computer-readable storage medium" should be taken to include a single medium or multiple media containing one or more sets of instructions; it should also be understood to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by a processor and that cause the processor to perform any of the methods of the present invention.
It will be understood by those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computer device, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by the computing device, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps thereof may be fabricated into a single integrated circuit module. The present invention is not limited to any specific combination of hardware and software.
Although the embodiments of the present invention have been described with reference to the accompanying drawings, it is not intended to limit the scope of the invention, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive changes in the technical solutions of the present invention.

Claims (10)

1. The mental state recognition method based on the graph neural network is characterized by comprising the following steps of:
preprocessing the acquired electroencephalogram signals, and decomposing the extracted electroencephalogram signals into a plurality of electroencephalogram signals by using an empirical mode decomposition algorithm;
extracting multidimensional statistical characteristics from the decomposed electroencephalogram signals;
Processing the multidimensional statistical characteristics by using a graph neural network, and extracting the connection relation between graph characteristics and signals among the multidimensional statistical characteristics;
and compressing the connection relation between the multi-dimensional image characteristic signals and the multi-dimensional channel signals to obtain classification results of the electroencephalogram signals.
2. The mental state recognition method based on the neural network of the figure as claimed in claim 1, wherein the preprocessing of the acquired brain electrical signals comprises:
decomposing the electroencephalogram signal into a limited number of intrinsic mode functions by using an empirical mode decomposition algorithm;
each eigenmode function should satisfy two conditions:
firstly, the number of extreme points and zero crossing points is equal, or the difference between the two is not more than 1;
second, the average of the local maximum envelope and the local minimum envelope is zero.
3. The method according to claim 2, wherein a plurality of eigenmode functions are selected as nodes in the representation of the graph;
in constructing the graph, the number of eigenmode functions selected is denoted as N, and is expressed as:
N＝min(Si)
wherein SiIs the number of IMFs obtained using empirical mode decomposition on the ith signal.
4. The mental state recognition method based on the graph neural network according to claim 2, wherein each eigenmode function component is divided into m clusters, and the size of each cluster can be automatically modified according to the change of the sequence;
defining the cluster data satisfies the following formula:
Figure FDA0003523241840000021
wherein T ismDuration of each cluster, TxRepresenting the duration of the pre-segmentation electroencephalographic signal.
5. The mental state recognition method based on the graph neural network as claimed in claim 1, wherein the decomposed electroencephalogram signal is subjected to multi-dimensional statistical feature extraction, specifically:
and extracting 12 statistical characteristics from each cluster data, wherein the characteristics are key characteristics representing the whole data set, and the characteristic set is { maximum value, average value, median, upper quartile, variation, kurtosis, minimum value, modulus, value range, standard deviation, skewness and lower quartile }.
6. The mental state recognition method of claim 1, wherein in the neural network, the figure is represented as
Figure FDA0003523241840000022
Wherein
Figure FDA0003523241840000023
And epsilon is defined as a set of nodes and a set of edges,
Figure FDA0003523241840000024
can use a feature matrix
Figure FDA0003523241840000025
Representing;
Figure FDA0003523241840000026
the number of nodes in the graph is shown, and d is the input feature dimension;
a is a adjacency matrix describing the structure of the graph,
Figure FDA0003523241840000027
If the node i is connected with the node j, A (i, j) ≠ 0, and A (i, j) represents the functional relation between the node i and the node j.
7. The mental state recognition method based on the graph neural network as claimed in claim 1, wherein the adjacency matrix in the graph neural network is obtained by training the whole network, the connection mode of the adjacency matrix is not set in advance, but a restriction condition needs to be set for the adjacency matrix in the training process;
the formula of the graph neural network is as follows:
Figure FDA0003523241840000028
where, σ is the activation function,
Figure FDA0003523241840000029
is a contiguous matrix of the neighbors,
Figure FDA00035232418400000210
is a degree matrix;
in a further preferred embodiment, the input of the neural network of the graph is represented as:
Figure FDA00035232418400000211
8. psychological state recognition system based on graph neural network, characterized by comprising:
a brain electrical signal pre-processing module configured to: preprocessing the acquired electroencephalogram signals, and decomposing the extracted electroencephalogram signals into a plurality of electroencephalogram signals by using an empirical mode decomposition algorithm;
a signal feature extraction module configured to: extracting multidimensional statistical characteristics from the decomposed electroencephalogram signals;
a graph neural network module configured to: processing the multidimensional statistical characteristics by using a graph neural network, and extracting the connection relation between graph characteristics and signals among the multidimensional statistical characteristics;
A feature dimension reduction module configured to: and compressing the connection relation between the multi-dimensional image characteristic signals and the multi-dimensional channel signals to obtain classification results of the electroencephalogram signals.
9. A computing device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program performs the steps of the method of any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, is adapted to carry out the steps of the method according to any one of the preceding claims 1 to 7.
CN202210187337.2A 2022-02-28 2022-02-28 Psychological state identification method and system based on graph neural network Pending CN114676720A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210187337.2A CN114676720A (en) 2022-02-28 2022-02-28 Psychological state identification method and system based on graph neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210187337.2A CN114676720A (en) 2022-02-28 2022-02-28 Psychological state identification method and system based on graph neural network

Publications (1)

Publication Number Publication Date
CN114676720A true CN114676720A (en) 2022-06-28

Family

ID=82073221

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210187337.2A Pending CN114676720A (en) 2022-02-28 2022-02-28 Psychological state identification method and system based on graph neural network

Country Status (1)

Country Link
CN (1) CN114676720A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115081558A (en) * 2022-08-22 2022-09-20 北京元亦科技有限公司 Brain network data classification method and system based on graph neural network
CN116671881A (en) * 2023-08-03 2023-09-01 北京九叁有方物联网科技有限公司 Head-wearing brain body operation capability assessment device and method based on graph neural network

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106955112A (en) * 2017-03-17 2017-07-18 泉州装备制造研究所 Brain wave Emotion recognition method based on Quantum wavelet neural networks model
CN109299751A (en) * 2018-11-26 2019-02-01 南开大学 The SSVEP brain electricity classification method of convolutional Neural model based on the enhancing of EMD data
CN110399857A (en) * 2019-08-01 2019-11-01 西安邮电大学 A kind of brain electricity emotion identification method based on figure convolutional neural networks
CN111493822A (en) * 2020-03-23 2020-08-07 济南国科医工科技发展有限公司 Sleep electroencephalogram based rapid eye movement period sleep behavior disorder classification method
CN113693563A (en) * 2021-08-27 2021-11-26 北京工业大学 Brain function network classification method based on hypergraph attention network

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106955112A (en) * 2017-03-17 2017-07-18 泉州装备制造研究所 Brain wave Emotion recognition method based on Quantum wavelet neural networks model
CN109299751A (en) * 2018-11-26 2019-02-01 南开大学 The SSVEP brain electricity classification method of convolutional Neural model based on the enhancing of EMD data
CN110399857A (en) * 2019-08-01 2019-11-01 西安邮电大学 A kind of brain electricity emotion identification method based on figure convolutional neural networks
CN111493822A (en) * 2020-03-23 2020-08-07 济南国科医工科技发展有限公司 Sleep electroencephalogram based rapid eye movement period sleep behavior disorder classification method
CN113693563A (en) * 2021-08-27 2021-11-26 北京工业大学 Brain function network classification method based on hypergraph attention network

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
TENGFEI SONG ET AL.: "EEG Emotion Recognition Using Dynamical Graph Convolutional Neural Networks", 《IEEE TRANSACTIONS ON AFFECTIVE COMPUTING》, 21 March 2018 (2018-03-21), pages 1 - 7 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115081558A (en) * 2022-08-22 2022-09-20 北京元亦科技有限公司 Brain network data classification method and system based on graph neural network
CN115081558B (en) * 2022-08-22 2022-11-15 北京元亦科技有限公司 Brain network data classification method and system based on graph neural network
CN116671881A (en) * 2023-08-03 2023-09-01 北京九叁有方物联网科技有限公司 Head-wearing brain body operation capability assessment device and method based on graph neural network

Similar Documents

Publication Publication Date Title
Liu et al. Epileptic seizure prediction with multi-view convolutional neural networks
CN114052735B (en) Deep field self-adaption-based electroencephalogram emotion recognition method and system
CN111134666A (en) Emotion recognition method of multi-channel electroencephalogram data and electronic device
CN112244873A (en) Electroencephalogram time-space feature learning and emotion classification method based on hybrid neural network
CN114676720A (en) Psychological state identification method and system based on graph neural network
US8595164B2 (en) Wavelet modeling paradigms for cardiovascular physiological signal interpretation
CN114595725B (en) Electroencephalogram signal classification method based on addition network and supervised contrast learning
Rajaguru et al. A framework for epilepsy classification using modified sparse representation classifiers and naive Bayesian classifier from electroencephalogram signals
Shen et al. Parallel sequence-channel projection convolutional neural network for EEG-based emotion recognition
Wankhade et al. Deep learning of empirical mean curve decomposition-wavelet decomposed EEG signal for emotion recognition
Pan et al. Recognition of human inner emotion based on two-stage FCA-ReliefF feature optimization
Zaid et al. Pre-processed and combined EEG data for epileptic seizure classification using deep learning
Yulita et al. Multichannel electroencephalography-based emotion recognition using machine learning
Lee et al. Emotion recognition with short-period physiological signals using bimodal sparse autoencoders.
Ji et al. 3DSleepNet: A multi-channel bio-signal based sleep stages classification method using deep learning
Hu et al. An features extraction and recognition method for underwater acoustic target based on ATCNN
Qazi et al. Automatic emotion recognition (AER) system based on two-level ensemble of lightweight deep CNN models
Immanuel et al. ANALYSIS OF DIFFERENT EMOTIONS WITH BIO-SIGNALS (EEG) USING DEEP CNN
Yu et al. Encoding physiological signals as images for affective state recognition using convolutional neural networks
Hassan et al. Review of EEG Signals Classification Using Machine Learning and Deep-Learning Techniques
Mampitiya et al. Classification of human emotions using ensemble classifier by analysing EEG signals
Polat et al. Automatic detection of cursor movements from the eeg signals via deep learning approach
Guodong et al. Multi feature fusion EEG emotion recognition
CN114626408A (en) Electroencephalogram signal classification method and device, electronic equipment, medium and product
Purnamasari et al. EEG based emotion recognition system induced by video music using a wavelet feature vectors and an artificial neural networks

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination