CN113558603B - Multi-modal cognitive disorder recognition method based on deep learning - Google Patents
Multi-modal cognitive disorder recognition method based on deep learning Download PDFInfo
- Publication number
- CN113558603B CN113558603B CN202110654421.6A CN202110654421A CN113558603B CN 113558603 B CN113558603 B CN 113558603B CN 202110654421 A CN202110654421 A CN 202110654421A CN 113558603 B CN113558603 B CN 113558603B
- Authority
- CN
- China
- Prior art keywords
- brain
- data
- mode
- functional
- deep learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013135 deep learning Methods 0.000 title claims abstract description 31
- 238000000034 method Methods 0.000 title claims abstract description 30
- 208000010877 cognitive disease Diseases 0.000 title claims abstract description 29
- 210000004556 brain Anatomy 0.000 claims abstract description 103
- 238000005259 measurement Methods 0.000 claims abstract description 16
- 239000011159 matrix material Substances 0.000 claims abstract description 15
- 238000012512 characterization method Methods 0.000 claims abstract description 13
- 238000005192 partition Methods 0.000 claims abstract description 13
- 238000012549 training Methods 0.000 claims abstract description 13
- 238000012545 processing Methods 0.000 claims abstract description 10
- 238000007781 pre-processing Methods 0.000 claims abstract description 7
- 238000004364 calculation method Methods 0.000 claims abstract description 6
- 238000010276 construction Methods 0.000 claims abstract description 5
- 230000002776 aggregation Effects 0.000 claims description 11
- 238000004220 aggregation Methods 0.000 claims description 11
- 239000013598 vector Substances 0.000 claims description 11
- 208000028698 Cognitive impairment Diseases 0.000 claims description 7
- 238000003384 imaging method Methods 0.000 claims description 6
- 238000010219 correlation analysis Methods 0.000 claims description 4
- 238000000638 solvent extraction Methods 0.000 claims description 4
- 238000013145 classification model Methods 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 8
- 238000012804 iterative process Methods 0.000 abstract description 4
- 206010012289 Dementia Diseases 0.000 description 6
- 210000003710 cerebral cortex Anatomy 0.000 description 4
- 230000006996 mental state Effects 0.000 description 4
- 208000024827 Alzheimer disease Diseases 0.000 description 3
- 238000013136 deep learning model Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 208000027061 mild cognitive impairment Diseases 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 230000003925 brain function Effects 0.000 description 2
- 230000002490 cerebral effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 210000004326 gyrus cinguli Anatomy 0.000 description 2
- 230000001771 impaired effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000000284 resting effect Effects 0.000 description 2
- 208000024891 symptom Diseases 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001684 chronic effect Effects 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 230000003930 cognitive ability Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000000875 corresponding effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 210000001061 forehead Anatomy 0.000 description 1
- 238000002599 functional magnetic resonance imaging Methods 0.000 description 1
- 208000018879 impaired coordination Diseases 0.000 description 1
- 230000002427 irreversible effect Effects 0.000 description 1
- 238000002595 magnetic resonance imaging Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000004770 neurodegeneration Effects 0.000 description 1
- 208000015122 neurodegenerative disease Diseases 0.000 description 1
- 210000002442 prefrontal cortex Anatomy 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves
- A61B5/055—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/40—Detecting, measuring or recording for evaluating the nervous system
- A61B5/4076—Diagnosing or monitoring particular conditions of the nervous system
- A61B5/4088—Diagnosing of monitoring cognitive diseases, e.g. Alzheimer, prion diseases or dementia
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Public Health (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Animal Behavior & Ethology (AREA)
- Veterinary Medicine (AREA)
- Surgery (AREA)
- Molecular Biology (AREA)
- Neurology (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Physiology (AREA)
- Artificial Intelligence (AREA)
- Psychiatry (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Hospice & Palliative Care (AREA)
- Fuzzy Systems (AREA)
- Evolutionary Computation (AREA)
- Child & Adolescent Psychology (AREA)
- Developmental Disabilities (AREA)
- Signal Processing (AREA)
- Mathematical Physics (AREA)
- Psychology (AREA)
- Neurosurgery (AREA)
- High Energy & Nuclear Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Radiology & Medical Imaging (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
Abstract
The invention discloses a multi-mode cognitive disorder recognition method based on deep learning, which comprises the following steps of: preprocessing the construction of a brain connectivity matrix; performing multi-mode brain characterization data calculation and complex brain network measurement; constructing a multi-mode data processing deep learning frame; and performing multi-class recognition model training. According to the technical scheme, a deep learning technology is adopted, a two-layer cascade structure is designed and realized, a GoogLeNet framework is integrated, three mode data of structural, functional connectivity and functional topology are input into the model for training, optimal weights of various feature combinations are continuously sought in an iterative process through a multi-node multi-level network, a judgment system of unified features of cognition disorder patients with different degrees is finally established, weights of 360 brain areas in HCPHMMP brain partitions are directly distributed, and the recognition problem of cognition disorder multiple types is well solved.
Description
Technical Field
The invention relates to the technical field of brain and cognition science, in particular to a multi-modal cognition disorder recognition method based on deep learning.
Background
Alzheimer's disease is a chronic, progressively worsening and irreversible neurodegenerative disease. Patients often suffer from hypomnesis, impaired hearing and vision, impaired speech, impaired coordination of movements, and gradually lose physical function as the condition progresses, ultimately leading to death. Patients with cognitive impairment to different degrees can be grouped through subjective psychology evaluation scores, for example, the simple mental state examination score is between 24 and 30, the dementia index is 0, and the crowd without symptoms such as depression, dementia and the like is divided into groups with normal cognitive ability; for simple mental state examination scores between 24 and 30, the dementia index is 0.5, the WMSLM II score is between 9 and 11 when the education level is greater than 16 years, or the WMSLM II score is between 6 and 9 when the education level is 8 to 15 years, or the WMSLM II score is between 3 and 6 when the education level is 0 to 7 years, and patients are actively informed that the memory is reduced but the daily life is not affected, and the patients without dementia symptoms are classified as early-stage mild cognitive impairment; for patients with simple mental state examination scores between 24 and 40, dementia index of 0.5, WMSLM ii score of less than 8 when education level is greater than 16 years, WMSLM ii score of less than 4 when education level is 8-15 years, or WMSLM ii score of less than 2 when education level is 0-7 years, and the patients are classified as advanced mild cognitive impairment; patients with a simple mental state examination score between 20 and 26 and a dementia index between 0.5 and 1 are classified as Alzheimer's disease. In addition to gradual changes in cognitive dysfunction shown by patient conditions and clinical psychological scores, the existing research results show that similar phenomena exist in classification and identification of cognitive dysfunction, the classification accuracy between Alzheimer disease patients and a healthy control group is highest, the identification difficulty is minimum, the difference is maximum, and the classification results of early-stage mild cognitive dysfunction patients and late-stage mild cognitive dysfunction patients are worst, and the difficulty is maximum. This progressive relationship also exists in the different group pairwise recognition models.
With the continuous development and perfection of deep learning technology, more research and application fields adopt deep learning to realize unprecedented remarkable progress, mainly focus on aspects of image recognition, machine vision, audio and video media data processing, social networks and the like, and the fields have mass data for training a model. In particular, in the last decade, computing and storage technologies have been dramatically improved such that hardware resources are no longer a bottleneck for very large scale data modeling. Unlike traditional machine learning, deep learning uses more complex modeling means, enabling breakthrough progress in nonlinear classification.
Chinese patent document CN107909117B discloses a "classification device for early-late mild cognitive impairment based on brain function network characteristics". Firstly, preprocessing sample data, extracting a plurality of brain region time sequences, adopting pearson correlation to calculate correlation coefficients between the brain region time sequences to construct a brain function network, and calculating brain network parameters. And extracting features by adopting a step-by-step analysis method, training a binary classifier, extracting corresponding feature vectors from resting state functional magnetic resonance data to be classified, and inputting the feature vectors into the trained binary classifier to obtain a medical image classification result. The identification accuracy of the technical scheme aiming at the multi-classification problem is not high.
Disclosure of Invention
The invention mainly solves the technical problem of low recognition accuracy of the prior technical scheme aiming at the multi-classification problem, provides a multi-mode cognitive disorder recognition method based on deep learning, adopts the deep learning technology, designs and realizes a two-layer cascade structure, integrates a GoogLeNet framework, inputs three mode data of structural, functional connectivity and functional topology into the model for training, continuously seeks the optimal weight of various feature combinations in the iterative process through a multi-node multi-layer network, finally establishes a judgment system of unified features of cognitive disorder patients of different degrees, directly distributes the weights of 360 brain areas in HCP MMP brain partitions, well solves the recognition problem of multiple classes of cognitive disorder, and finally realizes the four-classification accuracy of 96.86% through the average thickness of cerebral cortex and the local efficiency feature combination of a binary complex cerebral network.
The technical problems of the invention are mainly solved by the following technical proposal: the invention comprises the following steps:
s1, preprocessing the construction of a brain connectivity matrix;
s2, performing multi-mode brain characterization data calculation and complex brain network measurement;
s3, constructing a multi-mode data processing deep learning frame;
s4, training a multi-class recognition model.
By using the deep learning technology, modeling is performed on multi-modal data classification, and four-classification accuracy of 96.86% is achieved. The core of the deep learning technology is to adopt a 27-layer GoogleNet network, and design a two-layer cascade architecture to integrate multi-mode data as a training sample. Different from the previous research method, the deep learning model trained by the invention does not adopt additional feature selection or priori knowledge driven by hypothesis, but directly distributes weights of 360 brain areas in HCP MMP brain areas, and finally establishes a judgment system of unified features under different degrees of multi-mode of cognition disorder patients.
Preferably, the step S1 preprocessing performs functional region segmentation on the human brain according to the multi-mode brain partitioning method of washington university, and subdivides the whole brain into 360 brain regions, and the specific operations include: s1.1, acquiring structural state and functional state magnetic resonance data and magnetic field distribution information of imaging equipment;
s1.2, registering a human brain into a CIFTI space according to acquired information to form 3.2 ten thousand coordinate points;
s1.3, registering the brain into a multi-mode brain partition by adopting a J-HCPHMP method to form left and right 180 brain sub-regions;
s1.4, performing correlation analysis on functional magnetic resonance data among brain regions by adopting different brain partition methods to form an N multiplied by N complex brain network adjacency matrix;
s1.5, thresholding is carried out on the complex brain network adjacency matrix to remove noise and interference information, and dense brain connection is changed into sparse brain connection.
Preferably, three brain partition methods are adopted in the step S1.4, including HCPHMP, DKT-Atlas and Desikan-Killian brain partition methods are adopted to form 360, 62 and 68 sub brain regions respectively, and functional magnetic resonance data among the sub brain regions are subjected to correlation analysis to form complex brain network adjacent matrixes with the sizes of 360 multiplied by 360, 62 multiplied by 62 and 68 multiplied by 68.
Preferably, the step S2 specifically includes the following steps:
s2.1, extracting the average thickness and the average curvature of the cortex of the brain aiming at the pretreated brain structural and functional magnetic resonance data, wherein the data size is 1 XN as structural data representation;
s2.2, calculating a correlation connection matrix of the functional magnetic resonance data, wherein the correlation connection matrix is used as a functional data representation, and the data size is N multiplied by N;
s2.3, calculating global and regional complex brain network measurement indexes as functional topological data representation, wherein the dimension of the global complex brain network measurement index is 1, and the size of the regional complex brain network measurement index is 1 XN.
Preferably, in the step S2.3, 7 global metrics are calculated in the weighted brain network, including global efficiency, maximization module, optimal module number, homography coefficient, small world characteristic index, characteristic path length and average aggregation coefficient of the complex brain network, and 7 regional metrics including node degree, intensity, aggregation coefficient, local efficiency, medium number centrality, characteristic vector centrality and page ranking centrality of the complex brain network.
Preferably, in the step S2.3, 7 global metrics are calculated in the binary brain network, including global efficiency, maximization module, optimal module number, homography coefficient, small world characteristic index, characteristic path length and average aggregation coefficient of the complex brain network, and 8 regional metrics including node degree, aggregation coefficient, local efficiency, medium number centrality, characteristic vector centrality, page order centrality, K core centrality and flow coefficient of the brain network.
Preferably, the step S3 of constructing the multi-mode data processing deep learning framework specifically includes the following steps:
s3.1, integrating an acceptance structure in a GoogLeNet framework to enable the structure to construct a 27-layer deep learning framework in a stacked mode;
s3.2, integrating structural and functional topological characterization data, and constructing a super-dimensional feature vector in a first-level cascade mode;
and S3.3, integrating the functional connectivity data and the structural functional topological characterization data again in a two-stage cascade mode to serve as a multi-mode data input end.
Preferably, the step S4 of training the multi-class recognition model specifically includes:
s4.1, adopting a SoftMax structure, and taking the integrated multi-mode data as a SoftMax input layer;
and S4.2, adopting a gradient descent algorithm, and iteratively solving parameters until the multi-classification model converges.
The beneficial effects of the invention are as follows: by adopting a deep learning technology, a two-layer cascade structure is designed and realized, a GoogLeNet framework is integrated, three modal data of structural, functional connectivity and functional topology are input into the model for training, and different from a traditional machine learning method, the deep learning model continuously seeks the optimal weights of various feature combinations in an iterative process through a multi-node multi-level network, finally a judgment system of unified features of cognition disorder patients with different degrees is established, and model weights are allocated for 360 sub-areas of a HCP MMP brain partitioning method. By comparing the weight change of the same group of samples in the model solving process with the final model weight distribution of different groups of samples, the areas such as cingulate cortex, forehead leaf cortex and the like can be found to be closely related to the cognitive dysfunction. The deep learning method well solves the recognition problem of cognitive impairment in multiple types, and the model finally realizes 96.86% of test accuracy through the combination of the average thickness of the cerebral cortex and the local efficiency characteristics of the binary complex brain network.
Drawings
Fig. 1 is a flow chart for identifying cognitive impairment based on multi-modal data characterization in accordance with the present invention.
FIG. 2 is a diagram of a deep learning framework for constructing multi-modal data processing in accordance with the present invention.
FIG. 3 is a graph of multi-class performance of models in a combined structural and functional topological modality of the present invention.
Detailed Description
The technical scheme of the invention is further specifically described below through examples and with reference to the accompanying drawings.
Examples: the method for identifying multi-modal cognitive dysfunction based on deep learning in this embodiment, as shown in fig. 1, includes
As shown in fig. 1, the multi-modal cognitive impairment recognition method based on deep learning includes the following steps: and 1, constructing a pretreatment and brain connectivity matrix.
The calculation of the coefficient brain connection matrix by adopting the multi-mode brain partition in the step 1 comprises the following specific operations:
and 1-1, acquiring magnetic resonance data. The specific parameters of the structural magnetic resonance imaging are sagittal plane T1 weight three-dimensional rapid gradient echo imaging (T1W-3D-MPRAGE), an eight-surface coil SENSE parallel imaging algorithm is adopted, the external magnetic field intensity is 3 Tesla, the imaging resolution is 256 multiplied by 256.1.0 mm, the number of slices is 170, and the slice thickness is 1.2 mm. The echo time TR is set to 6.78 milliseconds and the repetition time TE is set to 3.14 milliseconds. The specific parameters of the functional magnetic resonance imaging are that the external magnetic field intensity of 3 tesla is adopted under the resting state of the subject, the imaging resolution is 64 multiplied by 64.3125 mm, the number of slices is 48, the thickness of the slices is 3.313 mm, and 140 time sequences are acquired for 6720 slices in total. The echo time TR is set to 3000 ms and the repetition time TE to 30 ms. Structural and functional magnetic resonance data were processed into the cifi space using the J-hcmmp data pre-processing method.
Three brain partition methods were used, including HCPHMP, DKT-Atlas, and Desikan-Killiank brain partitions, forming 360, 62, and 68 sub-brain regions, respectively. Functional magnetic resonance data between the various sub-brain regions are correlated to form complex brain network adjacency matrices of 360 x 360, 62 x 62 and 68 x 68 sizes. The matrix is thresholded to remove noise and interference information, forming a dense brain connection to a sparse brain connection.
Step 2, multi-mode brain characterization data calculation and complex brain network measurement
Step 2, calculating multi-mode brain characterization data and complex brain network measurement, wherein the specific operation is as follows:
2-1, extracting the average thickness of cerebral cortex and the average curvature of cortex aiming at the pretreated cerebral structural and functional magnetic resonance data, wherein the data size is 1 multiplied by 360, 1 multiplied by 62 and 1 multiplied by 68 as structural data characterization;
2-2, calculating a correlation connection matrix of the functional magnetic resonance data, wherein the data size is 360 multiplied by 360, 62 multiplied by 62 and 68 multiplied by 68 as a functional data representation;
2-3. Calculating global and regional complex brain network metrics as functional topological data representation, wherein the global complex brain network metrics have dimensions of 1 and the regional complex brain network metrics have dimensions of 1×360, 1×62 and 1×68. The method comprises the steps of weighting 7 global measurement indexes in a brain network, wherein the 7 global measurement indexes comprise global efficiency, a maximization module, the number of optimal modules, homography coefficients, small world characteristic indexes, characteristic path lengths and average aggregation coefficients of the complex brain network, and 7 regional measurement indexes comprise node degree, intensity, aggregation coefficients, local efficiency, medium number centrality, characteristic vector centrality and page ordering centrality of the complex brain network. In a binarized brain network, besides 7 global measurement indexes, 8 regional measurement indexes are calculated, including node degree, aggregation coefficient, local efficiency, medium number centrality, feature vector centrality, page ordering centrality, K core centrality and flow coefficient of the brain network.
Step 3, constructing a multi-mode data processing deep learning frame
The construction of the multi-mode data processing deep learning framework in the step 3 is specifically shown in fig. 2, and includes the following steps:
and 3-1, integrating an acceptance structure in the GoogLeNet framework to enable the structure to construct a 27-layer deep learning framework in a stacked mode.
And 3-2, integrating structural and functional topological characterization data, and constructing the super-dimensional feature vector in a first-stage cascade mode.
And 3-3, integrating the functional connectivity data and the structural functional topological characterization data again in a two-stage cascade mode to serve as a multi-mode data input end.
Step 4, multiclass recognition model training
The training of the multi-class recognition model in the step 4 is specifically as follows:
4-1. Adopting a SoftMax structure, and taking the integrated multi-mode data as a SoftMax input layer
And 4-2, adopting a gradient descent algorithm to iteratively solve parameters until the multi-classification model converges, wherein the classification result is shown in figure 3.
The invention adopts a deep learning technology, designs and realizes a two-layer cascade structure, integrates a GoogLeNet framework, and inputs three mode data of structural, functional connectivity and functional topology into the model for training. Different from the traditional machine learning method, the deep learning model continuously seeks the optimal weights of various feature combinations in the iterative process through a multi-node multi-layer network, and finally a judgment system of unified features of cognition disorder patients with different degrees is established, and model weights are distributed to 360 sub-areas of the HCP MMP brain partitioning method. By comparing the weight change of the same group of samples in the model solving process with the final model weight distribution of different groups of samples, the areas of cingulate cortex, prefrontal cortex and the like are found to be closely related to the cognitive dysfunction. The deep learning method well solves the recognition problem of cognitive impairment in multiple types, and the model finally realizes 96.86% of test accuracy through the combination of the average thickness of the cerebral cortex and the local efficiency characteristics of the binary complex brain network.
The specific embodiments described herein are offered by way of example only to illustrate the spirit of the invention. Those skilled in the art may make various modifications or additions to the described embodiments or substitutions thereof without departing from the spirit of the invention or exceeding the scope of the invention as defined in the accompanying claims.
Although the terms multi-modal brain characterization data calculation, complex brain network metrics, etc. are used more herein, the possibility of using other terms is not precluded. These terms are used merely for convenience in describing and explaining the nature of the invention; they are to be interpreted as any additional limitation that is not inconsistent with the spirit of the present invention.
Claims (4)
1. The multi-modal cognitive impairment recognition method based on deep learning is characterized by comprising the following steps of: s1, preprocessing construction of a brain connectivity matrix, wherein the preprocessing is used for carrying out functional region segmentation on human brains according to a multi-mode brain partitioning method of Washington university, and subdividing the whole brains into 360 brain regions, and the specific operation comprises the following steps:
s1.1, acquiring structural state and functional state magnetic resonance data and magnetic field distribution information of imaging equipment;
s1.2, registering a human brain into a CIFTI space according to acquired information to form 3.2 ten thousand coordinate points;
s1.3, registering the brain into a multi-mode brain partition by adopting a J-HCPHMP method to form left and right 180 brain sub-regions;
s1.4, performing correlation analysis on functional magnetic resonance data among brain regions by adopting different brain partition methods to form an N multiplied by N complex brain network adjacency matrix;
s1.5, performing threshold processing on the complex brain network adjacency matrix to remove noise and interference information, and changing dense brain connection into sparse brain connection;
s2, performing multi-mode brain characterization data calculation and complex brain network measurement, and specifically comprising the following steps:
s2.1, extracting the average thickness and the average curvature of the cortex of the brain aiming at the pretreated brain structural and functional magnetic resonance data, wherein the data size is 1 XN as structural data representation;
s2.2, calculating a correlation connection matrix of the functional magnetic resonance data, wherein the correlation connection matrix is used as a functional connectivity data representation, and the data size is N multiplied by N;
s2.3, calculating global and regional complex brain network measurement indexes as functional topological data representation, wherein the dimension of the global complex brain network measurement index is 1, and the size of the regional complex brain network measurement index is 1 XN; s3, constructing a multi-mode data processing deep learning framework, wherein the construction of the multi-mode data processing deep learning framework specifically comprises the following steps:
s3.1, integrating an acceptance structure in a GoogLeNet framework to enable the structure to construct a 27-layer deep learning framework in a stacked mode;
s3.2, integrating structural and functional topological characterization data, and constructing a super-dimensional feature vector in a first-level cascade mode;
s3.3, integrating the functional connectivity data and the structural characteristic data of the functional topology again in a two-stage cascade mode to serve as a multi-mode data input end;
s4, training a multi-class recognition model, which specifically comprises the following steps:
s4.1, adopting a SoftMax structure, and taking the integrated multi-mode data as a SoftMax input layer;
and S4.2, adopting a gradient descent algorithm, and iteratively solving parameters until the multi-classification model converges.
2. The method for recognizing deep learning-based multi-modal cognitive impairment according to claim 1, wherein step S1.4 employs three brain partition methods including hcmmp, DKT-Atlas and Desikan-Killiany brain partitions to form 360, 62 and 68 sub-brain regions, respectively, and performs correlation analysis on functional magnetic resonance data between the sub-brain regions to form complex brain network adjacency matrices of 360 x 360, 62 x 62 and 68 x 68 sizes.
3. The method according to claim 2, wherein the step S2.3 calculates 7 global metrics including global efficiency, maximizing module, optimal module number, homography coefficient, small world characteristic index, characteristic path length and average aggregation coefficient of the complex brain network, and 7 regional metrics including node degree, intensity, aggregation coefficient, local efficiency, medium centrality, characteristic vector centrality and page ranking centrality of the complex brain network in the weighted brain network.
4. The method according to claim 2, wherein the step S2.3 calculates 7 global metrics including global efficiency, maximizing module, number of best modules, homozygosity coefficient, small world characteristic index, characteristic path length, and average aggregation coefficient of the complex brain network, and 8 regional metrics including node degree, aggregation coefficient, local efficiency, medium centrality, characteristic vector centrality, page ranking centrality, K core centrality, and flow coefficient of the brain network in the binary brain network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110654421.6A CN113558603B (en) | 2021-06-11 | 2021-06-11 | Multi-modal cognitive disorder recognition method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110654421.6A CN113558603B (en) | 2021-06-11 | 2021-06-11 | Multi-modal cognitive disorder recognition method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113558603A CN113558603A (en) | 2021-10-29 |
| CN113558603B true CN113558603B (en) | 2023-11-10 |
Family
ID=78161983
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110654421.6A Active CN113558603B (en) | 2021-06-11 | 2021-06-11 | Multi-modal cognitive disorder recognition method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113558603B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114446466A (en) * | 2021-12-22 | 2022-05-06 | 山东师范大学 | Cognitive disorder auxiliary diagnosis system based on surface morphology measurement |
| CN114864051B (en) * | 2022-07-06 | 2022-10-04 | 北京智精灵科技有限公司 | Cognitive improvement method and system based on neural network |
| CN115423054B (en) * | 2022-11-07 | 2023-04-07 | 北京智精灵科技有限公司 | Uncertain training and exciting method and system based on personality characteristics of cognitive disorder patient |
| CN115424108B (en) * | 2022-11-08 | 2023-03-28 | 四川大学 | Cognitive dysfunction evaluation method based on audio-visual fusion perception |
| CN116098587B (en) * | 2023-01-13 | 2023-10-10 | 北京中科睿医信息科技有限公司 | Cognition assessment method, device, equipment and medium based on eye movement |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101317040B1 (en) * | 2012-09-03 | 2013-10-11 | 한양대학교 산학협력단 | Method and system for classifying functional alternation patterns of default mode network in progression of normal aging to mild cognitive impairment and alzheimer's disease |
| CN105726026A (en) * | 2016-01-28 | 2016-07-06 | 电子科技大学 | Mild cognitive impairment disease classifying method based on brain network and brain structure information |
| CN107242873A (en) * | 2017-07-05 | 2017-10-13 | 成都信息工程大学 | A kind of brain network establishing method interacted based on functional MRI psychology physiological |
| CN109087298A (en) * | 2018-08-17 | 2018-12-25 | 电子科技大学 | A kind of Alzheimer's disease MRI image classification method |
| CN110175510A (en) * | 2019-04-10 | 2019-08-27 | 杭州电子科技大学 | Multi-mode Mental imagery recognition methods based on brain function network characterization |
| CN112348833A (en) * | 2020-11-06 | 2021-02-09 | 浙江传媒学院 | Brain function network variation identification method and system based on dynamic connection |
| CN112674720A (en) * | 2020-12-24 | 2021-04-20 | 四川大学 | Alzheimer disease pre-diagnosis method based on 3D convolutional neural network |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8577106B2 (en) * | 2008-11-07 | 2013-11-05 | Department Of Biotechnology | Ready automated screening, diagnosis and classification technique for alzheimer's disease using magnetic resonance imaging signal from ventricular zone contour of brain |
-
2021
- 2021-06-11 CN CN202110654421.6A patent/CN113558603B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101317040B1 (en) * | 2012-09-03 | 2013-10-11 | 한양대학교 산학협력단 | Method and system for classifying functional alternation patterns of default mode network in progression of normal aging to mild cognitive impairment and alzheimer's disease |
| CN105726026A (en) * | 2016-01-28 | 2016-07-06 | 电子科技大学 | Mild cognitive impairment disease classifying method based on brain network and brain structure information |
| CN107242873A (en) * | 2017-07-05 | 2017-10-13 | 成都信息工程大学 | A kind of brain network establishing method interacted based on functional MRI psychology physiological |
| CN109087298A (en) * | 2018-08-17 | 2018-12-25 | 电子科技大学 | A kind of Alzheimer's disease MRI image classification method |
| CN110175510A (en) * | 2019-04-10 | 2019-08-27 | 杭州电子科技大学 | Multi-mode Mental imagery recognition methods based on brain function network characterization |
| CN112348833A (en) * | 2020-11-06 | 2021-02-09 | 浙江传媒学院 | Brain function network variation identification method and system based on dynamic connection |
| CN112674720A (en) * | 2020-12-24 | 2021-04-20 | 四川大学 | Alzheimer disease pre-diagnosis method based on 3D convolutional neural network |
Non-Patent Citations (4)
| Title |
|---|
| A novel joint HCPMMP method for automatically classifying Alzheimer’s and different stage MCI patients;Jinhua Sheng;Behavioural Brain Research;210-221 * |
| Discrimination and conversion prediction of mild cognitive impairment using convolutional neural networks;Congling Wu;Quantitative Imaging in Medicine and Surgery;第8卷(第10期);992-1003 * |
| Identification of the Early Stage of Alzheimer’s Disease Using Structural MRI and Resting-State fMRI;Seyed Hani Hojjati;Frontiers in Neurology;第10卷;1-12 * |
| Robustness of Brain Structural Networks Is Affected in Cognitively Impaired MS Patients;Hamza Farooq;Frontiers in Neurology;第11卷;1-8 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113558603A (en) | 2021-10-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113558603B (en) | Multi-modal cognitive disorder recognition method based on deep learning | |
| Murugan et al. | DEMNET: A deep learning model for early diagnosis of Alzheimer diseases and dementia from MR images | |
| Noreen et al. | A deep learning model based on concatenation approach for the diagnosis of brain tumor | |
| CN113616184B (en) | Brain network modeling and individual prediction method based on multi-mode magnetic resonance image | |
| CN109697718A (en) | A kind of self-closing disease detection method and device based on graph theory | |
| CN107506797A (en) | One kind is based on deep neural network and multi-modal image alzheimer disease sorting technique | |
| CN112418337B (en) | Multi-feature fusion data classification method based on brain function hyper-network model | |
| CN110046550A (en) | Pedestrian's Attribute Recognition system and method based on multilayer feature study | |
| CN105139004A (en) | Face expression identification method based on video sequences | |
| CN111863244B (en) | Functional connection mental disease classification method and system based on sparse pooling graph convolution | |
| CN110084381A (en) | A kind of brain network class method based on weight characteristic attribute fusion and the novel kernel of graph | |
| CN111009324A (en) | Mild cognitive impairment auxiliary diagnosis system and method based on brain network multi-feature analysis | |
| CN112446891A (en) | Medical image segmentation method based on U-Net network brain glioma | |
| Chou et al. | Optimizing parameters of multi-layer convolutional neural network by modeling and optimization method | |
| CN109730678A (en) | A kind of multilayer cerebral function network module division methods | |
| WO2021212715A1 (en) | Schizophrenia classification and identification method, operation control apparatus, and medical equipment | |
| CN114187258A (en) | Method and system for constructing autism classifier based on human brain function magnetic resonance image | |
| CN114469120B (en) | Multi-scale Dtw-BiLstm-Gan electrocardiosignal generation method based on similarity threshold migration | |
| CN113558602B (en) | Hypothesis-driven cognitive impairment brain network analysis method | |
| CN116843667A (en) | Brain image classification and brain cognition scoring prediction method based on multitask learning | |
| CN115474939A (en) | Autism spectrum disorder recognition model based on deep expansion neural network | |
| Dayananda et al. | A squeeze U-SegNet architecture based on residual convolution for brain MRI segmentation | |
| CN111325268A (en) | Image classification method and device based on multi-level feature representation and integrated learning | |
| Wang et al. | Multimodal autism spectrum disorder diagnosis method based on DeepGCN | |
| Tajammal et al. | Deep Learning‐Based Ensembling Technique to Classify Alzheimer’s Disease Stages Using Functional MRI |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |