CN113610853B - Emotional state display method, device and system based on resting brain function image - Google Patents

Emotional state display method, device and system based on resting brain function image Download PDF

Info

Publication number
CN113610853B
CN113610853B CN202111178892.0A CN202111178892A CN113610853B CN 113610853 B CN113610853 B CN 113610853B CN 202111178892 A CN202111178892 A CN 202111178892A CN 113610853 B CN113610853 B CN 113610853B
Authority
CN
China
Prior art keywords
image
resting
brain function
state
emotional
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111178892.0A
Other languages
Chinese (zh)
Other versions
CN113610853A (en
Inventor
栗觅
胡斌
吕胜富
康嘉明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing University of Technology
Original Assignee
Beijing University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing University of Technology filed Critical Beijing University of Technology
Priority to CN202111178892.0A priority Critical patent/CN113610853B/en
Publication of CN113610853A publication Critical patent/CN113610853A/en
Priority to PCT/CN2021/133534 priority patent/WO2023060721A1/en
Application granted granted Critical
Publication of CN113610853B publication Critical patent/CN113610853B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/16Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/165Evaluating the state of mind, e.g. depression, anxiety
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/25Fusion techniques
    • G06F18/253Fusion techniques of extracted features
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10072Tomographic images
    • G06T2207/10088Magnetic resonance imaging [MRI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30016Brain

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Psychiatry (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Biology (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Developmental Disabilities (AREA)
  • Psychology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Child & Adolescent Psychology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Educational Technology (AREA)
  • Hospice & Palliative Care (AREA)
  • Quality & Reliability (AREA)
  • Social Psychology (AREA)
  • Pathology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a resting state brain function image-based emotional state display method, device and system, wherein the method comprises the following steps: obtaining a resting brain function image of a tested person; inputting the resting brain function image into the first network model to obtain an emotion index; performing feature extraction on the resting brain function image to obtain an initial feature image; enhancing an interested area in the initial characteristic image according to the emotion index to obtain a target characteristic image; and superposing the target characteristic image on the resting brain function image to obtain a brain mode image for displaying the emotional state. The method can preliminarily confirm the abnormal risk of the emotional state of the tested person by obtaining the emotional index through the first network model, can further obtain image information related to the emotional state by carrying out feature extraction and partial enhancement on the resting state brain function image, and can intuitively display the emotional state of the tested person through a brain mode image obtained by superimposing the target feature image on the resting state brain function image.

Description

Emotional state display method, device and system based on resting brain function image
Technical Field
The disclosure relates to the technical field of artificial intelligence, in particular to a resting state brain function image-based emotional state display method, device and system.
Background
Whether psychological stress, anxiety or depression, will manifest as abnormal emotional states, which may lead to the development of anxiety or depression if the severity cannot be accurately checked and evaluated in a timely manner and psychological interventions are performed in a timely manner.
In the related art, the emotional state of a subject is generally analyzed from a functional magnetic resonance brain image. The existing functional magnetic resonance brain image can not provide a specific brain image related to the emotional state like a brain tumor image, and is not beneficial to a psychiatrist to accurately diagnose the emotional state of a tested person. The analysis of the functional magnetic resonance brain image requires professional knowledge, and the testee cannot intuitively know the emotional state of the testee according to the functional magnetic resonance brain image.
Disclosure of Invention
The embodiment of the disclosure provides an emotional state display method, device and system based on resting brain function images, which can intuitively display the emotional state.
Therefore, the embodiment of the disclosure provides the following technical scheme:
in a first aspect, an embodiment of the present disclosure provides an emotional state display method based on a resting brain function image, including:
obtaining a resting brain function image of a tested person;
inputting the resting brain function image into a first network model to obtain an emotion index;
performing feature extraction on the resting brain function image to obtain an initial feature image;
enhancing the interested area in the initial characteristic image according to the emotion index to obtain a target characteristic image;
and superposing the target characteristic image on the resting brain function image to obtain a brain mode image for displaying an emotional state.
Optionally, the enhancing the region of interest in the initial feature image according to the emotion index to obtain a target feature image includes:
calculating a weight coefficient of the initial characteristic image according to the emotion index;
multiplying each pixel point in the initial characteristic image by the weight coefficient to obtain the target characteristic image;
wherein the higher the sentiment index, the larger the weighting factor.
Optionally, calculating a weight coefficient of the initial feature image according to the emotion index includes:
judging whether the emotion index is larger than a set value or not;
if yes, inputting the resting brain function image into a second network model to obtain a first prediction vector, obtaining a first gradient map of the initial feature image according to the first prediction vector, and calculating an average gradient value of the first gradient map to serve as a weight coefficient of the initial feature image;
if not, inputting the resting brain function image into a third network model to obtain a second prediction vector, obtaining a second gradient map of the initial feature image according to the second prediction vector, and calculating the average gradient value of the second gradient map as the weight coefficient of the initial feature image.
Optionally, the performing feature extraction on the resting brain function image to obtain an initial feature image includes:
inputting the resting state brain function image into a fourth network model comprising a plurality of convolution kernels to obtain sub-feature maps corresponding to the convolution kernels one by one;
generating weight vectors corresponding to the convolution kernels one by one according to the distribution of emotion in the resting brain function image;
and carrying out weighted fusion on the sub-feature graphs according to the weight vectors corresponding to the sub-feature graphs to obtain the initial feature image.
Optionally, before inputting the resting brain function image into the first network model to obtain the emotion index, the method further includes:
down-sampling the resting brain function image;
and performing noise reduction processing on the resting state brain function image subjected to the down-sampling processing.
Optionally, the resting brain function image is multiple;
and superposing a plurality of target characteristic images corresponding to the resting state brain function images on the resting state brain function images to obtain brain mode images for displaying emotional states.
Optionally, the training method of the first network model includes:
obtaining sample data corresponding to a plurality of individuals, wherein the sample data comprises resting brain function images;
dividing the sample data into a training set, a verification set and a test set according to the ratio of 6:2: 2;
training the deep learning model by using a training set, verifying the deep learning model by using a verification set, and testing the deep learning model by using a test set;
and taking the trained deep learning model as a first network model.
In a second aspect, an embodiment of the present disclosure provides an emotional state display device based on a resting brain function image, including:
the acquisition module is used for acquiring a resting brain function image of the tested person;
the data processing module is used for inputting the resting brain function image into a first network model to obtain an emotion index;
the characteristic extraction module is used for extracting the characteristics of the resting brain function image to obtain an initial characteristic image;
the enhancement module is used for enhancing the interested area in the initial characteristic image according to the emotion index to obtain a target characteristic image;
and the superposition module is used for superposing the target characteristic image on the resting brain function image to obtain a brain mode image for displaying the emotional state.
In a third aspect, an embodiment of the present disclosure provides a resting state brain function image-based emotional state display system, including:
the data acquisition module is used for acquiring a resting state brain function image to be analyzed;
the brain depression analysis module is used for analyzing the resting state brain function image to be analyzed according to the pre-trained deep learning model to obtain an emotion index;
and the depressed brain image mode generating module is used for generating a brain mode image for displaying the emotional state according to the emotional index, the resting state brain function image to be analyzed and the feature extraction model.
Optionally, a feedback module is further included;
the feedback module comprises an emotion abnormity judging module and a display module;
the emotion abnormity judging module is used for generating an emotion abnormity risk level according to the emotion index and sending the risk level to the display module;
the display module is used for displaying the brain mode image and displaying an early warning image according to the emotional abnormality risk level.
One or more technical solutions provided in the embodiments of the present disclosure have the following advantages:
according to the emotion state display method based on the resting state brain function image, the emotion index is obtained through the first network model, abnormal risks of the emotion state of the tested person can be confirmed preliminarily, image information related to the emotion state can be further obtained by performing feature extraction and partial enhancement on the resting state brain function image, and the emotion state of the tested person can be displayed visually through the brain mode image obtained by superimposing the target feature image on the resting state brain function image.
Drawings
Fig. 1 is a flowchart of an emotional state presentation method based on resting brain function images according to an embodiment of the present disclosure;
fig. 2 is a block diagram of a resting brain function image-based emotional state display device according to an embodiment of the present disclosure;
fig. 3 is an emotional state presentation system based on resting brain function images according to an embodiment of the present disclosure.
Reference numerals:
21: an acquisition module; 22: a data processing module; 23: a feature extraction module; 24: a boost module; 25: a superposition module;
31: a data acquisition module; 32: a brain depression analysis module; 33: a depressed brain image pattern generation module; 34: and a feedback module.
Detailed Description
To make the objects, technical solutions and advantages of the present disclosure more apparent, the present disclosure will be described in further detail below with reference to the accompanying drawings in conjunction with the detailed description. It should be understood that the description is intended to be exemplary only, and is not intended to limit the scope of the present disclosure. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.
The described embodiments of the present disclosure are only some, and not all, embodiments of the present disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.
In the description of the present disclosure, it should be noted that the terms "first", "second", and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In addition, technical features involved in different embodiments of the present disclosure described below may be combined with each other as long as they do not conflict with each other.
Fig. 1 is a flowchart of an emotional state display method based on resting brain function images according to an embodiment of the present disclosure. As shown in fig. 1, an embodiment of the present disclosure provides an emotional state display method based on a resting brain function image, including the following steps:
s101: obtaining a resting brain function image of the tested person. The brain function image in resting state is obtained by functional magnetic resonance scanning.
S102: and inputting the resting brain function image into the first network model to obtain the emotion index. The first network model comprises an input layer, thirteen convolutional layers, six pooling layers, three full-connection layers and an output layer; and performing feature extraction and type recognition on the resting brain function image through a two-dimensional convolutional neural network model to finally obtain the emotion index. The mood index may be selected from the group consisting of depressed mood index, anxious mood index and stress mood index. Before the resting state brain function image is input into the first network model, the resting state brain function image can be selected to be subjected to down-sampling processing, then the resting state brain function image subjected to down-sampling processing is subjected to noise reduction processing, and finally the resting state brain function image subjected to noise reduction processing is cut, so that the influence of noise on the subsequent data processing process is reduced. In some embodiments, the higher the emotional index, the greater the risk of abnormality of the emotional state. In some embodiments, the training process of the first network model includes obtaining sample data corresponding to a plurality of individuals, the sample data including resting brain function images; dividing the sample data into a training set, a verification set and a test set according to the ratio of 6:2: 2; training the deep learning model by using a training set, verifying the deep learning model by using a verification set, and testing the deep learning model by using a test set; and taking the trained deep learning model as a first network model. The first network model may be selected as a deep learning model.
S103: and performing feature extraction on the resting brain function image by using the first network model to obtain an initial feature image and an emotion index. The initial feature image includes features related to emotional states in the resting brain function image and an emotional index.
S104: and enhancing the interested region in the initial characteristic image according to the emotion index to obtain a target characteristic image. The region of interest may be selected as the region most affected by mood.
S105: and superposing the target characteristic image on the resting brain function image to obtain a brain mode image for displaying the emotional state. Brain mode images can highlight brain features related to emotional states.
According to the emotion state display method based on the resting state brain function image, the emotion index is obtained through the first network model, abnormal risks of the emotion state of the tested person can be confirmed preliminarily, image information related to the emotion state can be further obtained by performing feature extraction and partial enhancement on the resting state brain function image, and the emotion state of the tested person can be displayed visually through the brain mode image obtained by superimposing the target feature image on the resting state brain function image.
In some embodiments, determining whether the sentiment index is greater than a set value; if yes, inputting the resting brain function image into a second network model to obtain a first prediction vector; the second network model is a deep learning network structure, the network structure comprises a convolution layer, a pooling layer, an activation layer and a full-connection layer, and the convolution layer, the pooling layer and the activation layer are sequentially connected into an assembly. And inputting the resting brain function image into the second network model to obtain a first prediction vector. Acquiring a first gradient map of the initial feature image according to the first prediction vector; calculating the average gradient value of the first gradient map as a weight coefficient of the initial characteristic image; if not, inputting the resting brain function image into a third network model to obtain a second prediction vector; the third network model is a deep learning network structure, the network structure comprises a convolution layer, a pooling layer, an activation layer and a full-connection layer, and the convolution layer, the pooling layer and the activation layer are sequentially connected into a component. And inputting the resting brain function image into a third network model to obtain a second prediction vector. Acquiring a second gradient map of the initial feature image according to the second prediction vector; and calculating the average gradient value of the second gradient map as a weight coefficient of the initial characteristic image. The second network model and the third network model are deep learning network models.
Inputting the resting brain function image into a fourth network model comprising a plurality of convolution kernels to obtain sub-feature maps corresponding to the convolution kernels one by one; generating weight vectors corresponding to the convolution kernels one by one according to the distribution of emotion in the resting brain function image; and performing weighted fusion on the sub-feature maps according to the weight vectors corresponding to the sub-feature maps to obtain an initial feature image.
And multiplying each pixel point in the initial characteristic image by the weight coefficient to obtain a target characteristic image.
In some embodiments, the resting brain function image of the subject is obtained based on the resting state (eye closure, relaxation, nothing is wanted). And superposing a plurality of target characteristic images corresponding to the resting state brain function images on the resting state brain function images to obtain brain mode images for displaying emotional states.
Fig. 2 is a block diagram of a resting brain function image-based emotional state display device according to an embodiment of the present disclosure. As shown in fig. 2, an embodiment of the present disclosure provides an emotional state display device based on a resting brain function image, including:
the acquisition module 21 is used for acquiring a resting brain function image of the tested person;
the data processing module 22 is used for inputting the resting brain function image into the first network model to obtain an emotion index;
the feature extraction module 23 is configured to perform feature extraction on the resting brain function image to obtain an initial feature image;
the enhancement module 24 is used for enhancing the interested region in the initial characteristic image according to the emotion index to obtain a target characteristic image;
and the superposition module 25 is used for superposing the target characteristic image on the resting brain function image to obtain a brain mode image for displaying the emotional state.
The emotion state display device based on the resting state brain function image, provided by the embodiment of the disclosure, can preliminarily confirm the abnormal risk of the emotion state of the tested person by obtaining the emotion index through the first network model, can further acquire image information related to the emotion state by performing feature extraction and partial enhancement on the resting state brain function image, and can visually display the emotion state of the tested person through a brain mode image obtained by superimposing the target feature image on the resting state brain function image.
Fig. 3 is an emotional state presentation system based on resting brain function images according to an embodiment of the present disclosure. As shown in fig. 3, an embodiment of the present disclosure provides an emotional state display system based on resting brain function images, including:
and the data acquisition module 31 is used for acquiring a resting state brain function image to be analyzed.
And the brain depression analysis module 32 is used for analyzing the resting state brain function image to be analyzed according to the pre-trained deep learning model to obtain the emotion index.
And a depressed brain image pattern generating module 33, configured to generate a brain pattern image for displaying an emotional state according to the emotional index, the resting state brain function image to be analyzed, and the feature extraction model.
A feedback module 34 including an emotional anomaly determination module and a display module; the emotion anomaly judging module is used for generating an emotion anomaly risk level according to the emotion index and sending the risk level to the display module; the display module is used for displaying the brain mode image and displaying the early warning image according to the emotional abnormality risk level.
The emotional anomaly risk level can be divided into a first level, a second level, a third level and a fourth level from small to large according to the threshold interval of the emotional index. The emotion state of the first level is normal, the early warning image corresponding to the first level can be selected to be a white bar, the emotion state of the second level is slightly abnormal, the early warning image corresponding to the second level can be selected to be a green bar, the emotion state of the third level is moderate abnormal, the early warning image corresponding to the third level can be selected to be a blue bar, the emotion state of the fourth level is severely abnormal, and the early warning image corresponding to the fourth level can be selected to be a red bar.
The system provided by the embodiment of the disclosure can judge whether the emotional state is abnormal or not and the severity of the abnormality and visually display the emotional state to the testee, and the display form of the image is easier to understand. The invention can more objectively measure and evaluate the emotional state of the testee, so the invention has important value for realizing self health management and improving the life quality.
It is to be understood that the above-described specific embodiments of the present disclosure are merely illustrative of or illustrative of the principles of the present disclosure and are not to be construed as limiting the present disclosure. Accordingly, any modification, equivalent replacement, improvement or the like made without departing from the spirit and scope of the present disclosure should be included in the protection scope of the present disclosure. Further, it is intended that the following claims cover all such variations and modifications that fall within the scope and bounds of the appended claims, or equivalents of such scope and bounds.

Claims (9)

1.一种基于静息态脑功能图像的情绪状态展示方法,其中,包括:1. A method for displaying emotional states based on resting-state brain function images, comprising: 获取被测者的静息态脑功能图像;Obtain the resting-state brain function images of the subjects; 将所述静息态脑功能图像输入第一网络模型得到情绪指数;Inputting the resting-state brain function image into the first network model to obtain an emotional index; 对所述静息态脑功能图像进行特征提取得到初始特征图像;performing feature extraction on the resting-state brain function image to obtain an initial feature image; 根据所述情绪指数对所述初始特征图像中感兴趣的区域进行增强得到目标特征图像;The target feature image is obtained by enhancing the region of interest in the initial feature image according to the emotion index; 在所述静息态脑功能图像上叠加所述目标特征图像得到用于展示情绪状态的脑模式图像;superimposing the target feature image on the resting state brain function image to obtain a brain pattern image for displaying emotional state; 对所述静息态脑功能图像进行特征提取得到初始特征图像包括:The initial feature image obtained by performing feature extraction on the resting state brain function image includes: 将所述静息态脑功能图像输入包括多个卷积核的第四网络模型得到与所述卷积核一一对应的子特征图;Inputting the resting-state brain function image into a fourth network model including multiple convolution kernels to obtain sub-feature maps corresponding to the convolution kernels one-to-one; 根据情绪在静息态脑功能图像中的分布生成与所述卷积核一一对应的权重向量;Generate a weight vector corresponding to the convolution kernel one-to-one according to the distribution of emotion in the resting-state brain function image; 根据与所述子特征图对应的权重向量对所述子特征图进行加权融合得到所述初始特征图像。The initial feature image is obtained by weighted fusion of the sub-feature maps according to the weight vector corresponding to the sub-feature map. 2.根据权利要求1所述的基于静息态脑功能图像的情绪状态展示方法,根据所述情绪指数对所述初始特征图像中感兴趣的区域进行增强得到目标特征图像包括:2. The emotional state display method based on resting-state brain function images according to claim 1, the region of interest in the initial feature image is enhanced to obtain a target feature image according to the emotional index, comprising: 根据所述情绪指数计算所述初始特征图像的权重系数;Calculate the weight coefficient of the initial feature image according to the emotion index; 将所述初始特征图像中的每个像素点与所述权重系数相乘得到所述目标特征图像;Multiplying each pixel in the initial feature image by the weight coefficient to obtain the target feature image; 其中,所述情绪指数越高所述权重系数越大。Wherein, the higher the sentiment index is, the higher the weight coefficient is. 3.根据权利要求2所述的基于静息态脑功能图像的情绪状态展示方法,根据所述情绪指数计算所述初始特征图像的权重系数包括:3. The emotional state display method based on resting-state brain function images according to claim 2, wherein calculating the weight coefficient of the initial feature image according to the emotional index comprises: 判断所述情绪指数是否大于设定值;Determine whether the emotional index is greater than the set value; 若是,则将所述静息态脑功能图像输入第二网络模型得到第一预测向量,根据所述第一预测向量获取所述初始特征图像的第一梯度图,计算所述第一梯度图的平均梯度值作为所述初始特征图像的权重系数;If so, input the resting-state brain function image into the second network model to obtain a first prediction vector, obtain the first gradient map of the initial feature image according to the first prediction vector, and calculate the value of the first gradient map. The average gradient value is used as the weight coefficient of the initial feature image; 若否,则将所述静息态脑功能图像输入第三网络模型得到第二预测向量,根据所述第二预测向量获取所述初始特征图像的第二梯度图,计算所述第二梯度图的平均梯度值作为所述初始特征图像的权重系数。If not, input the resting-state brain function image into a third network model to obtain a second prediction vector, obtain a second gradient map of the initial feature image according to the second prediction vector, and calculate the second gradient map The average gradient value of is used as the weight coefficient of the initial feature image. 4.根据权利要求1所述的基于静息态脑功能图像的情绪状态展示方法,将所述静息态脑功能图像输入第一网络模型得到情绪指数之前,还包括:4. The emotional state display method based on the resting state brain function image according to claim 1, before the resting state brain function image is input into the first network model to obtain the emotion index, further comprising: 对所述静息态脑功能图像进行下采样处理;performing down-sampling processing on the resting-state brain function image; 对下采样处理后的静息态脑功能图像进行降噪处理。Noise reduction processing is performed on the down-sampled resting-state functional brain images. 5.根据权利要求1所述的基于静息态脑功能图像的情绪状态展示方法,所述静息态脑功能图像为多个;5. The emotional state display method based on resting-state brain function images according to claim 1, wherein the resting-state brain function images are multiple; 在所述静息态脑功能图像上叠加与所述静息态脑功能图像对应的多个目标特征图像得到用于展示情绪状态的脑模式图像。A plurality of target feature images corresponding to the resting state brain function image are superimposed on the resting state brain function image to obtain a brain pattern image for displaying an emotional state. 6.根据权利要求1-5任一项所述的基于静息态脑功能图像的情绪状态展示方法,所述第一网络模型的训练方法包括:6. The emotional state display method based on resting-state brain function images according to any one of claims 1-5, the training method of the first network model comprises: 获取多个个体对应的样本数据,所述样本数据包括静息态脑功能图像和情绪指数;obtaining sample data corresponding to a plurality of individuals, the sample data including resting-state brain function images and emotional indexes; 将样本数据按照6:2:2的比例划分为训练集、验证集和测试集;Divide the sample data into training set, validation set and test set according to the ratio of 6:2:2; 使用训练集对深度学习模型进行训练,使用验证集对深度学习模型进行验证,使用测试集对深度学习模型进行测试;Use the training set to train the deep learning model, use the validation set to verify the deep learning model, and use the test set to test the deep learning model; 将训练好的深度学习模型作为第一网络模型。Use the trained deep learning model as the first network model. 7.一种基于静息态脑功能图像的情绪状态展示装置,其中,包括:7. An emotional state display device based on resting-state brain function images, comprising: 获取模块,用于获取被测者的静息态脑功能图像;The acquisition module is used to acquire the resting-state brain function image of the subject; 数据处理模块,用于将所述静息态脑功能图像输入第一网络模型得到情绪指数;a data processing module for inputting the resting-state brain function image into the first network model to obtain an emotional index; 特征提取模块,用于对所述静息态脑功能图像进行特征提取得到初始特征图像;对所述静息态脑功能图像进行特征提取得到初始特征图像包括:A feature extraction module, configured to perform feature extraction on the resting-state brain function image to obtain an initial feature image; perform feature extraction on the resting-state brain function image to obtain an initial feature image including: 将所述静息态脑功能图像输入包括多个卷积核的第四网络模型得到与所述卷积核一一对应的子特征图;Inputting the resting-state brain function image into a fourth network model including multiple convolution kernels to obtain sub-feature maps corresponding to the convolution kernels one-to-one; 根据情绪在静息态脑功能图像中的分布生成与所述卷积核一一对应的权重向量;Generate a weight vector corresponding to the convolution kernel one-to-one according to the distribution of emotion in the resting-state brain function image; 根据与所述子特征图对应的权重向量对所述子特征图进行加权融合得到所述初始特征图像;The initial feature image is obtained by weighting and fusing the sub-feature map according to the weight vector corresponding to the sub-feature map; 增强模块,用于根据所述情绪指数对所述初始特征图像中感兴趣的区域进行增强得到目标特征图像;an enhancement module, configured to enhance the region of interest in the initial feature image according to the emotion index to obtain a target feature image; 叠加模块,用于在所述静息态脑功能图像上叠加所述目标特征图像得到用于展示情绪状态的脑模式图像。A superimposing module, configured to superimpose the target feature image on the resting-state brain function image to obtain a brain pattern image for displaying an emotional state. 8.一种基于静息态脑功能图像的情绪状态展示系统,包括:8. An emotional state display system based on resting-state brain function images, comprising: 数据获取模块,用于获取待分析的静息态脑功能图像;a data acquisition module for acquiring resting-state brain function images to be analyzed; 脑抑郁分析模块,用于根据预训练的深度学习模型分析待分析的静息态脑功能图像,得到情绪指数;The brain depression analysis module is used to analyze the resting-state brain function images to be analyzed according to the pre-trained deep learning model, and obtain the emotional index; 抑郁脑图像模式生成模块,用于根据情绪指数、待分析的静息态脑功能图像和特征提取模型生成用于展示基于静息态脑功能图像的情绪状态的脑模式图像;抑郁脑图像模式生成模块,用于对所述静息态脑功能图像进行特征提取得到初始特征图像;Depressed brain image pattern generation module, which is used to generate a brain pattern image for displaying the emotional state based on the resting-state brain function image according to the emotion index, the resting-state brain function image to be analyzed, and the feature extraction model; Depressed brain image pattern generation a module for performing feature extraction on the resting-state brain function image to obtain an initial feature image; 根据所述情绪指数对所述初始特征图像中感兴趣的区域进行增强得到目标特征图像;The target feature image is obtained by enhancing the region of interest in the initial feature image according to the emotion index; 在所述静息态脑功能图像上叠加所述目标特征图像得到用于展示情绪状态的脑模式图像;superimposing the target feature image on the resting state brain function image to obtain a brain pattern image for displaying emotional state; 对所述静息态脑功能图像进行特征提取得到初始特征图像包括:The initial feature image obtained by performing feature extraction on the resting state brain function image includes: 将所述静息态脑功能图像输入包括多个卷积核的第四网络模型得到与所述卷积核一一对应的子特征图;Inputting the resting-state brain function image into a fourth network model including multiple convolution kernels to obtain sub-feature maps corresponding to the convolution kernels one-to-one; 根据情绪在静息态脑功能图像中的分布生成与所述卷积核一一对应的权重向量;Generate a weight vector corresponding to the convolution kernel one-to-one according to the distribution of emotion in the resting-state brain function image; 根据与所述子特征图对应的权重向量对所述子特征图进行加权融合得到所述初始特征图像。The initial feature image is obtained by weighted fusion of the sub-feature maps according to the weight vector corresponding to the sub-feature map. 9.根据权利要求8所述的基于静息态脑功能图像的情绪状态展示系统,还包括反馈模块;所述反馈模块包括情绪异常判别模块和显示模块;9. The emotional state display system based on resting state brain function images according to claim 8, further comprising a feedback module; the feedback module comprises an abnormal emotion discrimination module and a display module; 所述情绪异常判别模块用于根据所述情绪指数生成情绪异常风险等级,并将所述风险等级发送给所述显示模块;The emotional abnormality discrimination module is configured to generate abnormal emotional risk level according to the emotional index, and send the risk level to the display module; 所述显示模块用于显示所述脑模式图像,并根据所述情绪异常风险等级显示预警图像。The display module is used for displaying the brain pattern image, and displaying an early warning image according to the emotional abnormality risk level.
CN202111178892.0A 2021-10-11 2021-10-11 Emotional state display method, device and system based on resting brain function image Active CN113610853B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202111178892.0A CN113610853B (en) 2021-10-11 2021-10-11 Emotional state display method, device and system based on resting brain function image
PCT/CN2021/133534 WO2023060721A1 (en) 2021-10-11 2021-11-26 Emotional state displaying method, apparatus and system based on resting-state cerebral functional image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111178892.0A CN113610853B (en) 2021-10-11 2021-10-11 Emotional state display method, device and system based on resting brain function image

Publications (2)

Publication Number Publication Date
CN113610853A CN113610853A (en) 2021-11-05
CN113610853B true CN113610853B (en) 2022-01-28

Family

ID=78343489

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111178892.0A Active CN113610853B (en) 2021-10-11 2021-10-11 Emotional state display method, device and system based on resting brain function image

Country Status (2)

Country Link
CN (1) CN113610853B (en)
WO (1) WO2023060721A1 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113610067B (en) * 2021-10-11 2021-12-28 北京工业大学 Emotional state display method, device and system
CN113610853B (en) * 2021-10-11 2022-01-28 北京工业大学 Emotional state display method, device and system based on resting brain function image
CN116861211B (en) * 2023-06-08 2024-09-06 苏州大学 A method and system for EEG emotion recognition integrating spatiotemporal interactive neural network
CN119249107B (en) * 2024-12-04 2025-04-08 虎刺帕数智科技(浙江)有限公司 Risk assessment management system and method based on nerve recognition technology

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101912263A (en) * 2010-09-14 2010-12-15 北京师范大学 Real-time fMRI data processing system based on detection of brain functional network components
CN102293656A (en) * 2011-05-25 2011-12-28 四川大学华西医院 Emotional stability evaluation system based on magnetic resonance imaging and evaluation method thereof
CN104605853A (en) * 2015-01-21 2015-05-13 北京师范大学 Real-time nerve decoding system based on brain function features
CN110604572A (en) * 2019-10-08 2019-12-24 江苏海洋大学 Brain Activity State Recognition Method Based on Human Brain Feature Atlas

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8926511B2 (en) * 2008-02-29 2015-01-06 Biosense Webster, Inc. Location system with virtual touch screen
US8478379B2 (en) * 2008-11-12 2013-07-02 Biosense Webster, Inc. Probe visualization based on mechanical properties
US10610109B2 (en) * 2018-01-12 2020-04-07 Futurewei Technologies, Inc. Emotion representative image to derive health rating
CN113610853B (en) * 2021-10-11 2022-01-28 北京工业大学 Emotional state display method, device and system based on resting brain function image

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101912263A (en) * 2010-09-14 2010-12-15 北京师范大学 Real-time fMRI data processing system based on detection of brain functional network components
CN102293656A (en) * 2011-05-25 2011-12-28 四川大学华西医院 Emotional stability evaluation system based on magnetic resonance imaging and evaluation method thereof
CN104605853A (en) * 2015-01-21 2015-05-13 北京师范大学 Real-time nerve decoding system based on brain function features
CN110604572A (en) * 2019-10-08 2019-12-24 江苏海洋大学 Brain Activity State Recognition Method Based on Human Brain Feature Atlas

Also Published As

Publication number Publication date
WO2023060721A1 (en) 2023-04-20
CN113610853A (en) 2021-11-05

Similar Documents

Publication Publication Date Title
CN113610853B (en) Emotional state display method, device and system based on resting brain function image
Low et al. Peak width of skeletonized mean diffusivity as a marker of diffuse cerebrovascular damage
KR20210013830A (en) Medical image analyzing apparatus and method based on medical use artificial neural network evaluating analysis result thereof
JP6433632B1 (en) Computer system, subject diagnosis support method and program
CN117557941A (en) Video intelligent analysis system and method based on multi-mode data fusion
JP2009178538A (en) Picture therapy pattern selection and arrangement, color painting analysis method, psychological symptom diagnosis method based on picture therapy pattern analysis, and recording medium on which a program for doing this is recorded
CN111048202A (en) Intelligent traditional Chinese medicine diagnosis system and method thereof
US11393087B2 (en) Objective analysis of medical image data for white matter abnormality
CN111429547A (en) Abnormal color vision test chart synthesis method based on false homochromy search
CN113611416B (en) Psychological scene assessment method and system based on virtual reality technology
CN114847877A (en) Alzheimer disease risk assessment method based on visual memory assessment
CN113610067B (en) Emotional state display method, device and system
CN117994216A (en) Multi-scale feature fusion lesion detection method for tumor habitat analysis of medical image
TW202111617A (en) Method, non-transitory computer-readable media and apparatus for age prediction of regional brain area
KR101878359B1 (en) System and method for detecting mutiple-intelligence using information technology
CN113469274B (en) Method, device and equipment for evaluating reading capability
CN113842145B (en) Method, device and system for calculating emotion index based on pupil wave
CN109886378A (en) Method and apparatus for pushed information
CN116452592A (en) Method, device and system for constructing brain vascular disease AI cognitive function evaluation model
CN109965873A (en) Biological function measuring and analysis system, method and recording medium
Gende et al. High/low quality style transfer for mutual conversion of OCT images using contrastive unpaired translation generative adversarial networks
CN115497621A (en) Old person cognitive status evaluation system
KR20190067069A (en) Method for Enhancing Reliability of BCI System
CN112885432A (en) Emotion analysis and management system
JP7313165B2 (en) Alzheimer's Disease Survival Analyzer and Alzheimer's Disease Survival Analysis Program

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant