CN113610853B - Emotional state display method, device and system based on resting brain function image - Google Patents
Emotional state display method, device and system based on resting brain function image Download PDFInfo
- Publication number
- CN113610853B CN113610853B CN202111178892.0A CN202111178892A CN113610853B CN 113610853 B CN113610853 B CN 113610853B CN 202111178892 A CN202111178892 A CN 202111178892A CN 113610853 B CN113610853 B CN 113610853B
- Authority
- CN
- China
- Prior art keywords
- image
- brain function
- resting
- function image
- emotion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10088—Magnetic resonance imaging [MRI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30016—Brain
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Psychiatry (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computational Linguistics (AREA)
- Evolutionary Biology (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Developmental Disabilities (AREA)
- Psychology (AREA)
- Radiology & Medical Imaging (AREA)
- Child & Adolescent Psychology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Educational Technology (AREA)
- Hospice & Palliative Care (AREA)
- Quality & Reliability (AREA)
- Social Psychology (AREA)
- Pathology (AREA)
- Heart & Thoracic Surgery (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a resting state brain function image-based emotional state display method, device and system, wherein the method comprises the following steps: obtaining a resting brain function image of a tested person; inputting the resting brain function image into the first network model to obtain an emotion index; performing feature extraction on the resting brain function image to obtain an initial feature image; enhancing an interested area in the initial characteristic image according to the emotion index to obtain a target characteristic image; and superposing the target characteristic image on the resting brain function image to obtain a brain mode image for displaying the emotional state. The method can preliminarily confirm the abnormal risk of the emotional state of the tested person by obtaining the emotional index through the first network model, can further obtain image information related to the emotional state by carrying out feature extraction and partial enhancement on the resting state brain function image, and can intuitively display the emotional state of the tested person through a brain mode image obtained by superimposing the target feature image on the resting state brain function image.
Description
Technical Field
The disclosure relates to the technical field of artificial intelligence, in particular to a resting state brain function image-based emotional state display method, device and system.
Background
Whether psychological stress, anxiety or depression, will manifest as abnormal emotional states, which may lead to the development of anxiety or depression if the severity cannot be accurately checked and evaluated in a timely manner and psychological interventions are performed in a timely manner.
In the related art, the emotional state of a subject is generally analyzed from a functional magnetic resonance brain image. The existing functional magnetic resonance brain image can not provide a specific brain image related to the emotional state like a brain tumor image, and is not beneficial to a psychiatrist to accurately diagnose the emotional state of a tested person. The analysis of the functional magnetic resonance brain image requires professional knowledge, and the testee cannot intuitively know the emotional state of the testee according to the functional magnetic resonance brain image.
Disclosure of Invention
The embodiment of the disclosure provides an emotional state display method, device and system based on resting brain function images, which can intuitively display the emotional state.
Therefore, the embodiment of the disclosure provides the following technical scheme:
in a first aspect, an embodiment of the present disclosure provides an emotional state display method based on a resting brain function image, including:
obtaining a resting brain function image of a tested person;
inputting the resting brain function image into a first network model to obtain an emotion index;
performing feature extraction on the resting brain function image to obtain an initial feature image;
enhancing the interested area in the initial characteristic image according to the emotion index to obtain a target characteristic image;
and superposing the target characteristic image on the resting brain function image to obtain a brain mode image for displaying an emotional state.
Optionally, the enhancing the region of interest in the initial feature image according to the emotion index to obtain a target feature image includes:
calculating a weight coefficient of the initial characteristic image according to the emotion index;
multiplying each pixel point in the initial characteristic image by the weight coefficient to obtain the target characteristic image;
wherein the higher the sentiment index, the larger the weighting factor.
Optionally, calculating a weight coefficient of the initial feature image according to the emotion index includes:
judging whether the emotion index is larger than a set value or not;
if yes, inputting the resting brain function image into a second network model to obtain a first prediction vector, obtaining a first gradient map of the initial feature image according to the first prediction vector, and calculating an average gradient value of the first gradient map to serve as a weight coefficient of the initial feature image;
if not, inputting the resting brain function image into a third network model to obtain a second prediction vector, obtaining a second gradient map of the initial feature image according to the second prediction vector, and calculating the average gradient value of the second gradient map as the weight coefficient of the initial feature image.
Optionally, the performing feature extraction on the resting brain function image to obtain an initial feature image includes:
inputting the resting state brain function image into a fourth network model comprising a plurality of convolution kernels to obtain sub-feature maps corresponding to the convolution kernels one by one;
generating weight vectors corresponding to the convolution kernels one by one according to the distribution of emotion in the resting brain function image;
and carrying out weighted fusion on the sub-feature graphs according to the weight vectors corresponding to the sub-feature graphs to obtain the initial feature image.
Optionally, before inputting the resting brain function image into the first network model to obtain the emotion index, the method further includes:
down-sampling the resting brain function image;
and performing noise reduction processing on the resting state brain function image subjected to the down-sampling processing.
Optionally, the resting brain function image is multiple;
and superposing a plurality of target characteristic images corresponding to the resting state brain function images on the resting state brain function images to obtain brain mode images for displaying emotional states.
Optionally, the training method of the first network model includes:
obtaining sample data corresponding to a plurality of individuals, wherein the sample data comprises resting brain function images;
dividing the sample data into a training set, a verification set and a test set according to the ratio of 6:2: 2;
training the deep learning model by using a training set, verifying the deep learning model by using a verification set, and testing the deep learning model by using a test set;
and taking the trained deep learning model as a first network model.
In a second aspect, an embodiment of the present disclosure provides an emotional state display device based on a resting brain function image, including:
the acquisition module is used for acquiring a resting brain function image of the tested person;
the data processing module is used for inputting the resting brain function image into a first network model to obtain an emotion index;
the characteristic extraction module is used for extracting the characteristics of the resting brain function image to obtain an initial characteristic image;
the enhancement module is used for enhancing the interested area in the initial characteristic image according to the emotion index to obtain a target characteristic image;
and the superposition module is used for superposing the target characteristic image on the resting brain function image to obtain a brain mode image for displaying the emotional state.
In a third aspect, an embodiment of the present disclosure provides a resting state brain function image-based emotional state display system, including:
the data acquisition module is used for acquiring a resting state brain function image to be analyzed;
the brain depression analysis module is used for analyzing the resting state brain function image to be analyzed according to the pre-trained deep learning model to obtain an emotion index;
and the depressed brain image mode generating module is used for generating a brain mode image for displaying the emotional state according to the emotional index, the resting state brain function image to be analyzed and the feature extraction model.
Optionally, a feedback module is further included;
the feedback module comprises an emotion abnormity judging module and a display module;
the emotion abnormity judging module is used for generating an emotion abnormity risk level according to the emotion index and sending the risk level to the display module;
the display module is used for displaying the brain mode image and displaying an early warning image according to the emotional abnormality risk level.
One or more technical solutions provided in the embodiments of the present disclosure have the following advantages:
according to the emotion state display method based on the resting state brain function image, the emotion index is obtained through the first network model, abnormal risks of the emotion state of the tested person can be confirmed preliminarily, image information related to the emotion state can be further obtained by performing feature extraction and partial enhancement on the resting state brain function image, and the emotion state of the tested person can be displayed visually through the brain mode image obtained by superimposing the target feature image on the resting state brain function image.
Drawings
Fig. 1 is a flowchart of an emotional state presentation method based on resting brain function images according to an embodiment of the present disclosure;
fig. 2 is a block diagram of a resting brain function image-based emotional state display device according to an embodiment of the present disclosure;
fig. 3 is an emotional state presentation system based on resting brain function images according to an embodiment of the present disclosure.
Reference numerals:
21: an acquisition module; 22: a data processing module; 23: a feature extraction module; 24: a boost module; 25: a superposition module;
31: a data acquisition module; 32: a brain depression analysis module; 33: a depressed brain image pattern generation module; 34: and a feedback module.
Detailed Description
To make the objects, technical solutions and advantages of the present disclosure more apparent, the present disclosure will be described in further detail below with reference to the accompanying drawings in conjunction with the detailed description. It should be understood that the description is intended to be exemplary only, and is not intended to limit the scope of the present disclosure. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.
The described embodiments of the present disclosure are only some, and not all, embodiments of the present disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.
In the description of the present disclosure, it should be noted that the terms "first", "second", and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In addition, technical features involved in different embodiments of the present disclosure described below may be combined with each other as long as they do not conflict with each other.
Fig. 1 is a flowchart of an emotional state display method based on resting brain function images according to an embodiment of the present disclosure. As shown in fig. 1, an embodiment of the present disclosure provides an emotional state display method based on a resting brain function image, including the following steps:
s101: obtaining a resting brain function image of the tested person. The brain function image in resting state is obtained by functional magnetic resonance scanning.
S102: and inputting the resting brain function image into the first network model to obtain the emotion index. The first network model comprises an input layer, thirteen convolutional layers, six pooling layers, three full-connection layers and an output layer; and performing feature extraction and type recognition on the resting brain function image through a two-dimensional convolutional neural network model to finally obtain the emotion index. The mood index may be selected from the group consisting of depressed mood index, anxious mood index and stress mood index. Before the resting state brain function image is input into the first network model, the resting state brain function image can be selected to be subjected to down-sampling processing, then the resting state brain function image subjected to down-sampling processing is subjected to noise reduction processing, and finally the resting state brain function image subjected to noise reduction processing is cut, so that the influence of noise on the subsequent data processing process is reduced. In some embodiments, the higher the emotional index, the greater the risk of abnormality of the emotional state. In some embodiments, the training process of the first network model includes obtaining sample data corresponding to a plurality of individuals, the sample data including resting brain function images; dividing the sample data into a training set, a verification set and a test set according to the ratio of 6:2: 2; training the deep learning model by using a training set, verifying the deep learning model by using a verification set, and testing the deep learning model by using a test set; and taking the trained deep learning model as a first network model. The first network model may be selected as a deep learning model.
S103: and performing feature extraction on the resting brain function image by using the first network model to obtain an initial feature image and an emotion index. The initial feature image includes features related to emotional states in the resting brain function image and an emotional index.
S104: and enhancing the interested region in the initial characteristic image according to the emotion index to obtain a target characteristic image. The region of interest may be selected as the region most affected by mood.
S105: and superposing the target characteristic image on the resting brain function image to obtain a brain mode image for displaying the emotional state. Brain mode images can highlight brain features related to emotional states.
According to the emotion state display method based on the resting state brain function image, the emotion index is obtained through the first network model, abnormal risks of the emotion state of the tested person can be confirmed preliminarily, image information related to the emotion state can be further obtained by performing feature extraction and partial enhancement on the resting state brain function image, and the emotion state of the tested person can be displayed visually through the brain mode image obtained by superimposing the target feature image on the resting state brain function image.
In some embodiments, determining whether the sentiment index is greater than a set value; if yes, inputting the resting brain function image into a second network model to obtain a first prediction vector; the second network model is a deep learning network structure, the network structure comprises a convolution layer, a pooling layer, an activation layer and a full-connection layer, and the convolution layer, the pooling layer and the activation layer are sequentially connected into an assembly. And inputting the resting brain function image into the second network model to obtain a first prediction vector. Acquiring a first gradient map of the initial feature image according to the first prediction vector; calculating the average gradient value of the first gradient map as a weight coefficient of the initial characteristic image; if not, inputting the resting brain function image into a third network model to obtain a second prediction vector; the third network model is a deep learning network structure, the network structure comprises a convolution layer, a pooling layer, an activation layer and a full-connection layer, and the convolution layer, the pooling layer and the activation layer are sequentially connected into a component. And inputting the resting brain function image into a third network model to obtain a second prediction vector. Acquiring a second gradient map of the initial feature image according to the second prediction vector; and calculating the average gradient value of the second gradient map as a weight coefficient of the initial characteristic image. The second network model and the third network model are deep learning network models.
Inputting the resting brain function image into a fourth network model comprising a plurality of convolution kernels to obtain sub-feature maps corresponding to the convolution kernels one by one; generating weight vectors corresponding to the convolution kernels one by one according to the distribution of emotion in the resting brain function image; and performing weighted fusion on the sub-feature maps according to the weight vectors corresponding to the sub-feature maps to obtain an initial feature image.
And multiplying each pixel point in the initial characteristic image by the weight coefficient to obtain a target characteristic image.
In some embodiments, the resting brain function image of the subject is obtained based on the resting state (eye closure, relaxation, nothing is wanted). And superposing a plurality of target characteristic images corresponding to the resting state brain function images on the resting state brain function images to obtain brain mode images for displaying emotional states.
Fig. 2 is a block diagram of a resting brain function image-based emotional state display device according to an embodiment of the present disclosure. As shown in fig. 2, an embodiment of the present disclosure provides an emotional state display device based on a resting brain function image, including:
the acquisition module 21 is used for acquiring a resting brain function image of the tested person;
the data processing module 22 is used for inputting the resting brain function image into the first network model to obtain an emotion index;
the feature extraction module 23 is configured to perform feature extraction on the resting brain function image to obtain an initial feature image;
the enhancement module 24 is used for enhancing the interested region in the initial characteristic image according to the emotion index to obtain a target characteristic image;
and the superposition module 25 is used for superposing the target characteristic image on the resting brain function image to obtain a brain mode image for displaying the emotional state.
The emotion state display device based on the resting state brain function image, provided by the embodiment of the disclosure, can preliminarily confirm the abnormal risk of the emotion state of the tested person by obtaining the emotion index through the first network model, can further acquire image information related to the emotion state by performing feature extraction and partial enhancement on the resting state brain function image, and can visually display the emotion state of the tested person through a brain mode image obtained by superimposing the target feature image on the resting state brain function image.
Fig. 3 is an emotional state presentation system based on resting brain function images according to an embodiment of the present disclosure. As shown in fig. 3, an embodiment of the present disclosure provides an emotional state display system based on resting brain function images, including:
and the data acquisition module 31 is used for acquiring a resting state brain function image to be analyzed.
And the brain depression analysis module 32 is used for analyzing the resting state brain function image to be analyzed according to the pre-trained deep learning model to obtain the emotion index.
And a depressed brain image pattern generating module 33, configured to generate a brain pattern image for displaying an emotional state according to the emotional index, the resting state brain function image to be analyzed, and the feature extraction model.
A feedback module 34 including an emotional anomaly determination module and a display module; the emotion anomaly judging module is used for generating an emotion anomaly risk level according to the emotion index and sending the risk level to the display module; the display module is used for displaying the brain mode image and displaying the early warning image according to the emotional abnormality risk level.
The emotional anomaly risk level can be divided into a first level, a second level, a third level and a fourth level from small to large according to the threshold interval of the emotional index. The emotion state of the first level is normal, the early warning image corresponding to the first level can be selected to be a white bar, the emotion state of the second level is slightly abnormal, the early warning image corresponding to the second level can be selected to be a green bar, the emotion state of the third level is moderate abnormal, the early warning image corresponding to the third level can be selected to be a blue bar, the emotion state of the fourth level is severely abnormal, and the early warning image corresponding to the fourth level can be selected to be a red bar.
The system provided by the embodiment of the disclosure can judge whether the emotional state is abnormal or not and the severity of the abnormality and visually display the emotional state to the testee, and the display form of the image is easier to understand. The invention can more objectively measure and evaluate the emotional state of the testee, so the invention has important value for realizing self health management and improving the life quality.
It is to be understood that the above-described specific embodiments of the present disclosure are merely illustrative of or illustrative of the principles of the present disclosure and are not to be construed as limiting the present disclosure. Accordingly, any modification, equivalent replacement, improvement or the like made without departing from the spirit and scope of the present disclosure should be included in the protection scope of the present disclosure. Further, it is intended that the following claims cover all such variations and modifications that fall within the scope and bounds of the appended claims, or equivalents of such scope and bounds.
Claims (9)
1. A resting brain function image-based emotional state display method comprises the following steps:
obtaining a resting brain function image of a tested person;
inputting the resting brain function image into a first network model to obtain an emotion index;
performing feature extraction on the resting brain function image to obtain an initial feature image;
enhancing the interested area in the initial characteristic image according to the emotion index to obtain a target characteristic image;
superposing the target characteristic image on the resting brain function image to obtain a brain mode image for displaying an emotional state;
the characteristic extraction of the resting brain function image to obtain an initial characteristic image comprises the following steps:
inputting the resting state brain function image into a fourth network model comprising a plurality of convolution kernels to obtain sub-feature maps corresponding to the convolution kernels one by one;
generating weight vectors corresponding to the convolution kernels one by one according to the distribution of emotion in the resting brain function image;
and carrying out weighted fusion on the sub-feature graphs according to the weight vectors corresponding to the sub-feature graphs to obtain the initial feature image.
2. The resting brain function image-based emotional state display method according to claim 1, wherein the enhancing the region of interest in the initial feature image according to the emotion index to obtain a target feature image comprises:
calculating a weight coefficient of the initial characteristic image according to the emotion index;
multiplying each pixel point in the initial characteristic image by the weight coefficient to obtain the target characteristic image;
wherein the higher the sentiment index, the larger the weighting factor.
3. The resting brain function image-based emotional state presentation method according to claim 2, wherein calculating the weight coefficient of the initial feature image according to the emotion index comprises:
judging whether the emotion index is larger than a set value or not;
if yes, inputting the resting brain function image into a second network model to obtain a first prediction vector, obtaining a first gradient map of the initial feature image according to the first prediction vector, and calculating an average gradient value of the first gradient map to serve as a weight coefficient of the initial feature image;
if not, inputting the resting brain function image into a third network model to obtain a second prediction vector, obtaining a second gradient map of the initial feature image according to the second prediction vector, and calculating the average gradient value of the second gradient map as the weight coefficient of the initial feature image.
4. The emotional state display method based on the resting brain function image according to claim 1, wherein before inputting the resting brain function image into the first network model to obtain the emotional index, the method further comprises:
down-sampling the resting brain function image;
and performing noise reduction processing on the resting state brain function image subjected to the down-sampling processing.
5. The emotional state presentation method based on resting brain function images according to claim 1, wherein the resting brain function images are plural;
and superposing a plurality of target characteristic images corresponding to the resting state brain function images on the resting state brain function images to obtain brain mode images for displaying emotional states.
6. The emotional state display method based on resting brain function images according to any one of claims 1 to 5, wherein the training method of the first network model comprises:
obtaining sample data corresponding to a plurality of individuals, wherein the sample data comprises resting brain function images and emotion indexes;
dividing the sample data into a training set, a verification set and a test set according to the ratio of 6:2: 2;
training the deep learning model by using a training set, verifying the deep learning model by using a verification set, and testing the deep learning model by using a test set;
and taking the trained deep learning model as a first network model.
7. An emotional state display device based on resting brain function images, comprising:
the acquisition module is used for acquiring a resting brain function image of the tested person;
the data processing module is used for inputting the resting brain function image into a first network model to obtain an emotion index;
the characteristic extraction module is used for extracting the characteristics of the resting brain function image to obtain an initial characteristic image; the characteristic extraction of the resting brain function image to obtain an initial characteristic image comprises the following steps:
inputting the resting state brain function image into a fourth network model comprising a plurality of convolution kernels to obtain sub-feature maps corresponding to the convolution kernels one by one;
generating weight vectors corresponding to the convolution kernels one by one according to the distribution of emotion in the resting brain function image;
carrying out weighted fusion on the sub-feature maps according to the weight vectors corresponding to the sub-feature maps to obtain the initial feature image;
the enhancement module is used for enhancing the interested area in the initial characteristic image according to the emotion index to obtain a target characteristic image;
and the superposition module is used for superposing the target characteristic image on the resting brain function image to obtain a brain mode image for displaying the emotional state.
8. An emotional state display system based on resting brain function images, comprising:
the data acquisition module is used for acquiring a resting state brain function image to be analyzed;
the brain depression analysis module is used for analyzing the resting state brain function image to be analyzed according to the pre-trained deep learning model to obtain an emotion index;
the depression brain image mode generating module is used for generating a brain mode image for displaying the emotional state based on the resting state brain function image according to the emotion index, the resting state brain function image to be analyzed and the feature extraction model; the depression brain image mode generating module is used for carrying out feature extraction on the resting brain function image to obtain an initial feature image;
enhancing the interested area in the initial characteristic image according to the emotion index to obtain a target characteristic image;
superposing the target characteristic image on the resting brain function image to obtain a brain mode image for displaying an emotional state;
the characteristic extraction of the resting brain function image to obtain an initial characteristic image comprises the following steps:
inputting the resting state brain function image into a fourth network model comprising a plurality of convolution kernels to obtain sub-feature maps corresponding to the convolution kernels one by one;
generating weight vectors corresponding to the convolution kernels one by one according to the distribution of emotion in the resting brain function image;
and carrying out weighted fusion on the sub-feature graphs according to the weight vectors corresponding to the sub-feature graphs to obtain the initial feature image.
9. The resting state brain function image based emotional state display system of claim 8, further comprising a feedback module; the feedback module comprises an emotion abnormity judging module and a display module;
the emotion abnormity judging module is used for generating an emotion abnormity risk level according to the emotion index and sending the risk level to the display module;
the display module is used for displaying the brain mode image and displaying an early warning image according to the emotional abnormality risk level.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111178892.0A CN113610853B (en) | 2021-10-11 | 2021-10-11 | Emotional state display method, device and system based on resting brain function image |
| PCT/CN2021/133534 WO2023060721A1 (en) | 2021-10-11 | 2021-11-26 | Emotional state displaying method, apparatus and system based on resting-state cerebral functional image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111178892.0A CN113610853B (en) | 2021-10-11 | 2021-10-11 | Emotional state display method, device and system based on resting brain function image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113610853A CN113610853A (en) | 2021-11-05 |
| CN113610853B true CN113610853B (en) | 2022-01-28 |
Family
ID=78343489
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111178892.0A Active CN113610853B (en) | 2021-10-11 | 2021-10-11 | Emotional state display method, device and system based on resting brain function image |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113610853B (en) |
| WO (1) | WO2023060721A1 (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113610853B (en) * | 2021-10-11 | 2022-01-28 | 北京工业大学 | Emotional state display method, device and system based on resting brain function image |
| CN113610067B (en) * | 2021-10-11 | 2021-12-28 | 北京工业大学 | Emotional state display method, device and system |
| CN116861211B (en) * | 2023-06-08 | 2024-09-06 | 苏州大学 | Electroencephalogram emotion recognition method and system integrating space-time interaction neural network |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101912263A (en) * | 2010-09-14 | 2010-12-15 | 北京师范大学 | Real-time functional magnetic resonance data processing system based on brain functional network component detection |
| CN102293656A (en) * | 2011-05-25 | 2011-12-28 | 四川大学华西医院 | Emotional stability evaluation system based on magnetic resonance imaging and evaluation method thereof |
| CN104605853A (en) * | 2015-01-21 | 2015-05-13 | 北京师范大学 | Real-time nerve decoding system based on brain function features |
| CN110604572A (en) * | 2019-10-08 | 2019-12-24 | 江苏海洋大学 | Brain activity state identification method based on human brain characteristic map |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8926511B2 (en) * | 2008-02-29 | 2015-01-06 | Biosense Webster, Inc. | Location system with virtual touch screen |
| US8478379B2 (en) * | 2008-11-12 | 2013-07-02 | Biosense Webster, Inc. | Probe visualization based on mechanical properties |
| US10610109B2 (en) * | 2018-01-12 | 2020-04-07 | Futurewei Technologies, Inc. | Emotion representative image to derive health rating |
| CN113610853B (en) * | 2021-10-11 | 2022-01-28 | 北京工业大学 | Emotional state display method, device and system based on resting brain function image |
-
2021
- 2021-10-11 CN CN202111178892.0A patent/CN113610853B/en active Active
- 2021-11-26 WO PCT/CN2021/133534 patent/WO2023060721A1/en active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101912263A (en) * | 2010-09-14 | 2010-12-15 | 北京师范大学 | Real-time functional magnetic resonance data processing system based on brain functional network component detection |
| CN102293656A (en) * | 2011-05-25 | 2011-12-28 | 四川大学华西医院 | Emotional stability evaluation system based on magnetic resonance imaging and evaluation method thereof |
| CN104605853A (en) * | 2015-01-21 | 2015-05-13 | 北京师范大学 | Real-time nerve decoding system based on brain function features |
| CN110604572A (en) * | 2019-10-08 | 2019-12-24 | 江苏海洋大学 | Brain activity state identification method based on human brain characteristic map |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113610853A (en) | 2021-11-05 |
| WO2023060721A1 (en) | 2023-04-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113610853B (en) | Emotional state display method, device and system based on resting brain function image | |
| KR101000492B1 (en) | Apparatus and method for diagnosis of psychological symptoms corresponding to analysis of coloring patterns in art therapy assessment and medium of recording its program | |
| JP5058064B2 (en) | Picture therapy pattern selection and arrangement, color painting analysis method, psychological symptom diagnosis method based on picture therapy pattern analysis, and recording medium on which a program for doing this is recorded | |
| CN111887867A (en) | Method and system for analyzing character formation based on expression recognition and psychological test | |
| CN109891519A (en) | Information processing unit, information processing method and program | |
| CN112489029B (en) | Medical image segmentation method and device based on convolutional neural network | |
| CN113298780A (en) | Child bone age assessment method and system based on deep learning | |
| CN117557941A (en) | Video intelligent analysis system and method based on multi-mode data fusion | |
| US11393087B2 (en) | Objective analysis of medical image data for white matter abnormality | |
| CN112561848A (en) | Method, non-transitory computer-readable medium, and apparatus for regional brain age prediction | |
| Tan et al. | Automated detection of congenital heart disease in fetal ultrasound screening | |
| CN118552504A (en) | Ultrasonic image detection method and system based on artificial intelligence | |
| CN116738352B (en) | Method and device for classifying abnormal rod cells of retinal vascular occlusion disease | |
| CN117137488B (en) | Auxiliary identification method for depression symptoms based on electroencephalogram data and facial expression images | |
| CN114298189A (en) | Fatigue driving detection method, device, equipment and storage medium | |
| CN113658697A (en) | Psychological assessment system based on video fixation difference | |
| CN116452592B (en) | Method, device and system for constructing brain vascular disease AI cognitive function evaluation model | |
| CN117786600A (en) | Cognitive evaluation method, device, electronic equipment and storage medium | |
| CN113610067B (en) | Emotional state display method, device and system | |
| CN116469579A (en) | Brain state monitoring method, equipment, storage medium and device | |
| CN113469274B (en) | Method, device and equipment for evaluating reading capability | |
| CN113611416B (en) | Psychological scene assessment method and system based on virtual reality technology | |
| CN115457029A (en) | Underwater image quality measuring method based on perception characteristics | |
| CN113537407B (en) | Image data evaluation processing method and device based on machine learning | |
| Raghav et al. | Autism Spectrum Disorder Detection in Children Using Transfer Learning Techniques |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |