CN113558603A - Multi-modal cognitive impairment recognition method based on deep learning - Google Patents
Multi-modal cognitive impairment recognition method based on deep learning Download PDFInfo
- Publication number
- CN113558603A CN113558603A CN202110654421.6A CN202110654421A CN113558603A CN 113558603 A CN113558603 A CN 113558603A CN 202110654421 A CN202110654421 A CN 202110654421A CN 113558603 A CN113558603 A CN 113558603A
- Authority
- CN
- China
- Prior art keywords
- brain
- modal
- data
- deep learning
- functional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000013135 deep learning Methods 0.000 title claims abstract description 35
- 208000010877 cognitive disease Diseases 0.000 title claims abstract description 31
- 208000028698 Cognitive impairment Diseases 0.000 title claims abstract description 21
- 210000004556 brain Anatomy 0.000 claims abstract description 97
- 238000005259 measurement Methods 0.000 claims abstract description 18
- 239000011159 matrix material Substances 0.000 claims abstract description 16
- 238000012549 training Methods 0.000 claims abstract description 14
- 238000012545 processing Methods 0.000 claims abstract description 13
- 238000012512 characterization method Methods 0.000 claims abstract description 10
- 238000007781 pre-processing Methods 0.000 claims abstract description 7
- 238000004364 calculation method Methods 0.000 claims abstract description 5
- 238000010276 construction Methods 0.000 claims abstract description 3
- 230000002776 aggregation Effects 0.000 claims description 11
- 238000004220 aggregation Methods 0.000 claims description 11
- 238000005192 partition Methods 0.000 claims description 11
- 239000013598 vector Substances 0.000 claims description 11
- 210000003710 cerebral cortex Anatomy 0.000 claims description 6
- 238000003384 imaging method Methods 0.000 claims description 6
- 238000010219 correlation analysis Methods 0.000 claims description 4
- 230000002490 cerebral effect Effects 0.000 claims description 3
- 238000013145 classification model Methods 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 7
- 230000008569 process Effects 0.000 abstract description 5
- 206010012289 Dementia Diseases 0.000 description 6
- 238000000638 solvent extraction Methods 0.000 description 4
- 208000024827 Alzheimer disease Diseases 0.000 description 3
- 238000013136 deep learning model Methods 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 208000027061 mild cognitive impairment Diseases 0.000 description 3
- 230000000750 progressive effect Effects 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000003925 brain function Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000003920 cognitive function Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 210000004326 gyrus cinguli Anatomy 0.000 description 2
- 210000002442 prefrontal cortex Anatomy 0.000 description 2
- 230000000284 resting effect Effects 0.000 description 2
- 208000024891 symptom Diseases 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 208000000044 Amnesia Diseases 0.000 description 1
- 208000026139 Memory disease Diseases 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001684 chronic effect Effects 0.000 description 1
- 230000003930 cognitive ability Effects 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000000875 corresponding effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000002599 functional magnetic resonance imaging Methods 0.000 description 1
- 230000001771 impaired effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000002427 irreversible effect Effects 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 208000011977 language disease Diseases 0.000 description 1
- 238000002595 magnetic resonance imaging Methods 0.000 description 1
- 230000006984 memory degeneration Effects 0.000 description 1
- 208000023060 memory loss Diseases 0.000 description 1
- 230000006996 mental state Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000004770 neurodegeneration Effects 0.000 description 1
- 208000015122 neurodegenerative disease Diseases 0.000 description 1
- 230000003863 physical function Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves
- A61B5/055—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/40—Detecting, measuring or recording for evaluating the nervous system
- A61B5/4076—Diagnosing or monitoring particular conditions of the nervous system
- A61B5/4088—Diagnosing of monitoring cognitive diseases, e.g. Alzheimer, prion diseases or dementia
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Public Health (AREA)
- Animal Behavior & Ethology (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Veterinary Medicine (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Neurology (AREA)
- General Health & Medical Sciences (AREA)
- Physiology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Artificial Intelligence (AREA)
- Psychiatry (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Fuzzy Systems (AREA)
- Radiology & Medical Imaging (AREA)
- Signal Processing (AREA)
- Child & Adolescent Psychology (AREA)
- Developmental Disabilities (AREA)
- Hospice & Palliative Care (AREA)
- High Energy & Nuclear Physics (AREA)
- Psychology (AREA)
- Neurosurgery (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
Abstract
The invention discloses a multi-modal cognitive impairment recognition method based on deep learning, which comprises the following steps of: preprocessing the construction of a brain connectivity matrix; performing multi-modal brain characterization data calculation and complex brain network measurement; constructing a multi-modal data processing deep learning framework; and carrying out multi-class recognition model training. The technical scheme adopts a deep learning technology, a two-layer cascade structure is designed and realized, a GoogLeNet framework is integrated, three modal data of structural property, functional connectivity and functional topology are input into the model for training, optimal weights of various feature combinations are continuously searched in an iteration process through a multi-node multi-layer network, finally, a judgment system of unified features of patients with different degrees of cognitive impairment is established, weights of 360 brain areas in HCPMMP brain areas are directly distributed, and the problem of multi-class recognition of the cognitive impairment is well solved.
Description
Technical Field
The invention relates to the technical field of brain and cognitive science, in particular to a multi-modal cognitive impairment recognition method based on deep learning.
Background
Alzheimer's disease is a chronic, progressive and irreversible neurodegenerative disease. Patients often suffer from hypomnesis, impaired audiology and vision, language disorder and poor coordination of actions, gradually lose physical functions with the progressive increase of the disease condition, and finally die. The patients with different degrees of cognitive impairment can be grouped by subjective psychological evaluation scores, for example, the simple intellectual state examination score is between 24 and 30, the dementia index is 0, and people without symptoms such as depression and dementia are classified as normal cognitive ability; the simple intelligence state examination score is between 24 and 30, the dementia index is 0.5, when the education degree is more than 16 years, the WMSLM II score is between 9 and 11, or when the education degree is between 8 and 15 years, the WMSLM II score is between 6 and 9, or when the education degree is between 0 and 7 years, the WMSLM II score is between 3 and 6, and the patient actively informs that the memory loss exists but does not influence the daily life, and the patient without the dementia symptom is classified as early mild cognitive impairment; the simple intelligence state examination score is between 24 and 40, the dementia index is 0.5, when the education degree is more than 16 years, the WMSLM II score is less than 8, or when the education degree is between 8 and 15 years, the WMSLM II score is less than 4, or when the education degree is between 0 and 7 years, the patient with the WMSLM II score less than 2 is classified as late mild cognitive impairment; patients with simple mental state examination score of 20-26 and dementia index of 0.5-1 are classified as Alzheimer's disease. Except that the cognitive dysfunction shown by the patient's condition and clinical psychological scoring gradually changes, the existing research results show that similar phenomena also exist in cognitive disorder classification and identification, the highest classification accuracy between patients with Alzheimer's disease and healthy control groups means that the identification difficulty is the smallest and the difference is the largest, and the results of pairwise classification between early-stage mild cognitive disorder patients and late-stage mild cognitive disorder patients are the worst and the difficulty is the largest. This progressive relationship also exists in pairwise recognition models of different groupings.
With the continuous development and improvement of deep learning technology, more research and application fields adopt deep learning to realize unprecedented dramatic progress, mainly focus on the aspects of image recognition, machine vision, audio and video media data processing, social networks and the like, and the fields have mass data to provide for models for training. Especially in the last decade, the computation and storage technology is greatly improved, so that hardware resources are no longer the bottleneck of ultra-large scale data modeling. Different from the traditional machine learning, the deep learning uses a more complex modeling means, and the breakthrough progress of the nonlinear classification is realized.
Chinese patent document CN107909117B discloses a "classification device for early and late mild cognitive impairment based on brain function network characteristics". Firstly, sample data is preprocessed, a plurality of brain region time sequences are extracted, a brain function network is constructed by adopting Pearson correlation to calculate correlation coefficients among the brain region time sequences, and brain network parameters are calculated. And secondly, extracting features by adopting a step-by-step analysis method, training a binary classifier, extracting corresponding feature vectors from the resting state functional magnetic resonance data to be classified, and inputting the feature vectors into the trained binary classifier to obtain a medical image classification result. The technical scheme aims at the problem of low identification accuracy of multi-classification.
Disclosure of Invention
The invention mainly solves the technical problem of low recognition accuracy of the original technical scheme aiming at the multi-classification problem, provides a multi-mode cognitive impairment recognition method based on deep learning, adopts the deep learning technology, designs and realizes a two-layer cascade structure, integrates a GoogLeNet framework, inputs three modal data of structural property, functional connectivity and functional topology into the model for training, passes through a multi-node multi-layer network, continuously seeking the optimal weight of various feature combinations in the iterative process, finally establishing a judging system of unified features of the patients with different degrees of cognitive impairment, directly distributing the weights of 360 brain areas in the HCP MMP brain areas, well solving the problem of recognizing multiple types of cognitive impairment, through the combination of the average thickness of the cerebral cortex and the local efficiency characteristics of the binary complex brain network, the model finally realizes the four-classification accuracy of 96.86%.
The technical problem of the invention is mainly solved by the following technical scheme: the invention comprises the following steps:
s1, preprocessing the construction of the brain connectivity matrix;
s2, performing multi-modal brain characterization data calculation and complex brain network measurement;
s3, constructing a multi-modal data processing deep learning framework;
s4, training the multi-class recognition model.
Using deep learning techniques, a four-classification accuracy of 96.86% was achieved for multi-modal data classification modeling. The core of the deep learning technology is that a 27-layer GoogLeNet network is adopted, and a two-layer cascade structure is designed to integrate multi-modal data as a training sample. Different from the previous research method, the deep learning model trained by the method does not adopt additional feature selection or hypothesis-driven prior knowledge, but directly distributes the weights of 360 brain areas in the HCP MMP brain areas, and finally establishes a unified feature evaluation system under multiple modes for patients with different degrees of cognitive impairment.
Preferably, the step S1 of preprocessing is to perform functional region segmentation on the human brain according to a multi-modal brain partition method of washington university, and subdivide the whole brain into 360 brain regions, and the specific operations include: s1.1, acquiring structural state and functional state magnetic resonance data and magnetic field distribution information of imaging equipment;
s1.2, registering the human brain to a CIFTI space according to the acquired information to form 3.2 ten thousand coordinate points;
s1.3, registering the brain to a multi-modal brain partition by adopting a J-HCPMMP method to form 180 sub-brain areas on the left and right;
s1.4, performing correlation analysis on functional magnetic resonance data among all brain areas by adopting different brain partition methods to form a complex brain network adjacent matrix with the size of N multiplied by N;
s1.5, carrying out threshold processing on the complex brain network adjacency matrix, removing noise and interference information, and changing dense brain connection into sparse brain connection.
Preferably, the step S1.4 adopts three brain partition methods, including HCPMMP, DKT-Atlas and Desikan-Killiany brain partition, to form 360, 62 and 68 sub-brain regions, respectively, and performs correlation analysis on the functional magnetic resonance data between the sub-brain regions to form a complex brain network adjacency matrix with the size of 360 × 360, 62 × 62 and 68 × 68.
Preferably, the step S2 specifically includes the following steps:
s2.1, extracting the average thickness and the average curvature of the cerebral cortex aiming at the preprocessed cerebral structural and functional magnetic resonance data as structural data representation, wherein the data size is 1 multiplied by N;
s2.2, calculating a correlation connection matrix of the functional magnetic resonance data as a functional data representation, wherein the data size is NxN;
and S2.3, calculating global and regional complex brain network measurement indexes as functional topological data representations, wherein the dimension of the global complex brain network measurement index is 1, and the size of the regional complex brain network measurement index is 1 multiplied by N.
Preferably, in the step S2.3, in the weighted brain network, 7 kinds of global metrics including global efficiency, maximum module, optimal module number, homography coefficient, worldlet characteristic index, feature path length, and average aggregation coefficient of the complex brain network, and 7 kinds of regional metrics including node degrees, strength, aggregation coefficient, local efficiency, betweenness centrality, feature vector centrality, and page ranking centrality of the complex brain network are calculated.
Preferably, in the binarization brain network, the step S2.3 calculates 7 kinds of global metrics including global efficiency, maximum module, optimal module number, homozygosity coefficient, small-world characteristic index, feature path length, and average aggregation coefficient of the complex brain network, and 8 kinds of regional metrics including node degrees, aggregation coefficient, local efficiency, betweenness centrality, feature vector centrality, page rank centrality, K-kernel centrality, and flow coefficient of the brain network.
Preferably, the step S3 of constructing the multimodal data processing deep learning framework specifically includes the following steps:
s3.1, integrating an inclusion structure in a GoogleLeNet framework to construct a 27-layer deep learning framework in a stacking form;
s3.2, integrating structural and functional topological characterization data, and constructing a super-dimensional feature vector in a primary cascade mode;
and S3.3, integrating the functional connectivity data and the structural and functional topological representation data again in a two-stage cascade mode to serve as a multi-mode data input end.
Preferably, the step S4 of training the multi-class recognition models specifically includes:
s4.1, adopting a SoftMax structure, and taking the integrated multi-modal data as a SoftMax input layer;
and S4.2, iteratively solving parameters by adopting a gradient descent algorithm until the multi-classification model converges.
The invention has the beneficial effects that: a deep learning technology is adopted, a two-layer cascade structure is designed and realized, a GoogLeNet framework is integrated, three modal data of structural property, functional connectivity and functional topology are input into the model for training, different from a traditional machine learning method, the deep learning model continuously seeks the optimal weight of various feature combinations in an iteration process through a multi-node multi-layer network, finally, a judgment system of unified features of different degrees of cognitive disorder patients is established, and model weights are distributed for 360 sub-regions of a HCP MMP brain partitioning method. By comparing the weight change of the same group of samples in the model solving process with the final model weight distribution of different groups of samples, the areas such as the cingulate gyrus layer, the prefrontal cortex and the like can be found to be closely related to the abnormal cognitive function. The deep learning method well solves the problem of recognizing multiple types of cognitive disorders, and the model finally achieves 96.86% of test accuracy through the combination of the average thickness of the cerebral cortex and the local efficiency characteristics of the binary complex brain network.
Drawings
FIG. 1 is a flow chart for cognitive impairment recognition based on multi-modal data characterization according to the present invention.
FIG. 2 is a diagram of the framework for constructing a multi-modal data processing deep learning framework according to the present invention.
FIG. 3 is a multi-classification performance diagram of models in a structural and functional topological combination modality.
Detailed Description
The technical scheme of the invention is further specifically described by the following embodiments and the accompanying drawings.
Example (b): the method for recognizing multi-modal cognitive impairment based on deep learning of the embodiment, as shown in fig. 1, includes
As shown in fig. 1, the method for recognizing a multi-modal cognitive impairment based on deep learning includes the following steps: step 1, preprocessing and constructing a brain connectivity matrix.
The method for calculating the coefficient brain connection matrix by adopting the multi-modal brain partition in the step 1 specifically comprises the following operations:
1-1, magnetic resonance data is acquired. The specific parameters of the structural magnetic resonance imaging are sagittal plane T1 weighted three-dimensional fast gradient echo imaging (T1W-3D-MPRAGE), an eight-surface coil SENSE parallel imaging algorithm is adopted, the external magnetic field strength is 3 Tesla, the imaging resolution is 256 multiplied by 2561.0 mm, the number of slices is 170, and the slice thickness is 1.2 mm. The echo time TR is set to 6.78 msec and the repetition time TE is set to 3.14 msec. The specific parameters of functional magnetic resonance imaging are that 3 Tesla external magnetic field strength is adopted in a resting state of a subject, imaging resolution is 64 multiplied by 643.3125 mm, the number of slices is 48, slice thickness is 3.313 mm, and a total of 140 time sequences of 6720 slices are acquired. The echo time TR is set to 3000 milliseconds, and the repetition time TE is set to 30 milliseconds. And processing the structural and functional magnetic resonance data to a CIFTI space by adopting a J-HCPMMP data preprocessing method.
1-2 three brain partitioning methods were used, including HCPMMP, DKT-Atlas and Desikan-Killiany brain partitioning, forming 360, 62 and 68 sub-brain regions, respectively. The functional magnetic resonance data between the respective sub-brain regions are correlated to form a complex brain network adjacency matrix of 360 × 360, 62 × 62, and 68 × 68 sizes. And carrying out threshold processing on the matrix, removing noise and interference information, and forming a brain connection changed from dense brain connection to sparse brain connection.
Step 2, multi-modal brain characterization data calculation and complex brain network measurement
Step 2, calculating multi-modal brain characterization data and measuring a complex brain network, and specifically operating as follows:
2-1, extracting the average thickness and the average curvature of the cerebral cortex aiming at the preprocessed cerebral structural and functional magnetic resonance data as structural data representation, wherein the data size is 1 × 360, 1 × 62 and 1 × 68;
2-2, calculating a correlation connection matrix of the functional magnetic resonance data as a functional data representation, wherein the data size is 360 × 360, 62 × 62 and 68 × 68;
and 2-3, calculating global and regional complex brain network measurement indexes as functional topological data representation, wherein the dimension of the global complex brain network measurement index is 1, and the sizes of the regional complex brain network measurement indexes are 1 × 360, 1 × 62 and 1 × 68. The method comprises the following steps of weighting 7 global measurement indexes in the brain network, wherein the 7 global measurement indexes comprise the global efficiency, the maximum module, the optimal module number, the homological coefficient, the small world characteristic index, the characteristic path length and the average aggregation coefficient of the complex brain network, and the 7 regional measurement indexes comprise the node degree, the strength, the aggregation coefficient, the local efficiency, the betweenness centrality, the feature vector centrality and the page ordering centrality of the complex brain network. In the binary brain network, besides 7 kinds of global measurement indexes, 8 kinds of regional measurement indexes are calculated, including each node degree, aggregation coefficient, local efficiency, betweenness centrality, feature vector centrality, page ordering centrality, K core centrality and flow coefficient of the brain network.
Step 3, constructing a multi-modal data processing deep learning framework
Specifically, as shown in fig. 2, the constructing of the multi-modal data processing deep learning framework in step 3 includes the following steps:
3-1, integrating the inclusion structure in the GooglLeNet framework, so that the integration structure is stacked to construct a 27-layer deep learning framework.
And 3-2, integrating structural and functional topological characterization data, and constructing the super-dimensional feature vector in a one-level cascade mode.
And 3, integrating the functional connectivity data and the structural and functional topological representation data again in a two-stage cascade mode to serve as a multi-mode data input end.
Step 4, training of multi-class recognition models
The training of the multi-class recognition model in the step 4 is specifically as follows:
4-1, adopting a SoftMax structure, and taking the integrated multi-modal data as a SoftMax input layer
And 4-2, iteratively solving the parameters by adopting a gradient descent algorithm until the multi-classification model converges, wherein the classification result is shown in figure 3.
The invention adopts deep learning technology, designs and realizes a two-layer cascade structure, integrates a GoogLeNet framework, and inputs three modal data of structural, functional connectivity and functional topology into the model for training. Different from the traditional machine learning method, the deep learning model continuously seeks the optimal weight of various feature combinations in the iteration process through a multi-node multi-level network, finally establishes a judgment system of unified features of different degrees of cognitive impairment patients, and allocates model weights for 360 sub-areas of the HCP MMP brain partitioning method. By comparing the weight change of the same group of samples in the model solving process with the final model weight distribution of different groups of samples, the areas such as the cingulate gyrus layer and the prefrontal cortex are closely related to the abnormal cognitive function. The deep learning method well solves the problem of recognizing multiple types of cognitive disorders, and the model finally achieves 96.86% of test accuracy through the combination of the average thickness of the cerebral cortex and the local efficiency characteristics of the binary complex brain network.
The specific embodiments described herein are merely illustrative of the spirit of the invention. Various modifications or additions may be made to the described embodiments or alternatives may be employed by those skilled in the art without departing from the spirit or ambit of the invention as defined in the appended claims.
Although the terms multimodal brain characterization data calculation, complex brain network metrics, etc. are used more herein, the possibility of using other terms is not excluded. These terms are used merely to more conveniently describe and explain the nature of the present invention; they are to be construed as being without limitation to any additional limitations that may be imposed by the spirit of the present invention.
Claims (8)
1. A multi-modal cognitive impairment recognition method based on deep learning is characterized by comprising the following steps:
s1, preprocessing the construction of the brain connectivity matrix;
s2, performing multi-modal brain characterization data calculation and complex brain network measurement;
s3, constructing a multi-modal data processing deep learning framework;
s4, training the multi-class recognition model.
2. The method for recognizing the multi-modal cognitive impairment based on the deep learning of claim 1, wherein the step S1 is implemented by preprocessing, according to a multi-modal brain partition method of washington university, functional region segmentation is performed on the human brain, so as to subdivide the whole brain into 360 brain regions, and the specific operations include:
s1.1, acquiring structural state and functional state magnetic resonance data and magnetic field distribution information of imaging equipment;
s1.2, registering the human brain to a CIFTI space according to the acquired information to form 3.2 ten thousand coordinate points;
s1.3, registering the brain to a multi-modal brain partition by adopting a J-HCPMMP method to form 180 sub-brain areas on the left and right;
s1.4, performing correlation analysis on functional magnetic resonance data among all brain areas by adopting different brain partition methods to form a complex brain network adjacent matrix with the size of N multiplied by N;
s1.5, carrying out threshold processing on the complex brain network adjacency matrix, removing noise and interference information, and changing dense brain connection into sparse brain connection.
3. The method for identifying multi-modal cognitive impairment based on deep learning of claim 2, wherein the step S1.4 employs three brain partition methods including HCPMMP, DKT-Atlas and Desikan-Killiany brain partitions to form 360, 62 and 68 sub-brain regions respectively, and performs correlation analysis on the functional magnetic resonance data between the sub-brain regions to form a complex brain network adjacency matrix with the size of 360 x 360, 62 x 62 and 68 x 68.
4. The method for recognizing multi-modal cognitive impairment based on deep learning according to claim 1, wherein the step S2 specifically comprises the following steps:
s2.1, extracting the average thickness and the average curvature of the cerebral cortex aiming at the preprocessed cerebral structural and functional magnetic resonance data as structural data representation, wherein the data size is 1 multiplied by N;
s2.2, calculating a correlation connection matrix of the functional magnetic resonance data as a functional data representation, wherein the data size is NxN;
and S2.3, calculating global and regional complex brain network measurement indexes as functional topological data representations, wherein the dimension of the global complex brain network measurement index is 1, and the size of the regional complex brain network measurement index is 1 multiplied by N.
5. The method according to claim 3, wherein the step S2.3 is to calculate 7 global metrics including global efficiency, maximum module, optimal module number, homography coefficient, small-world characteristic index, feature path length and average aggregation coefficient of the complex brain network, and 7 regional metrics including node degree, strength, aggregation coefficient, local efficiency, betweenness centrality, feature vector centrality and page ranking centrality of the complex brain network in the weighted brain network.
6. The method according to claim 3, wherein the step S2.3 is to calculate 7 kinds of global metrics including global efficiency, maximum module, optimal module number, homography coefficient, small-world characteristic index, feature path length and average aggregation coefficient of the complex brain network and 8 kinds of regional metrics including node degrees, aggregation coefficient, local efficiency, betweenness centrality, feature vector centrality, page ranking centrality, K-kernel centrality and flow coefficient of the brain network in the binary brain network.
7. The method for recognizing the multi-modal cognitive impairment based on the deep learning of claim 1, wherein the step S3 of constructing the multi-modal data processing deep learning framework specifically comprises the following steps:
s3.1, integrating an inclusion structure in a GoogleLeNet framework to construct a 27-layer deep learning framework in a stacking form;
s3.2, integrating structural and functional topological characterization data, and constructing a super-dimensional feature vector in a primary cascade mode;
and S3.3, integrating the functional connectivity data and the structural and functional topological representation data again in a two-stage cascade mode to serve as a multi-mode data input end.
8. The method for recognizing multi-modal cognitive impairment based on deep learning of claim 1, wherein the training of the multi-class recognition model in the step S4 specifically comprises:
s4.1, adopting a SoftMax structure, and taking the integrated multi-modal data as a SoftMax input layer;
and S4.2, iteratively solving parameters by adopting a gradient descent algorithm until the multi-classification model converges.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110654421.6A CN113558603B (en) | 2021-06-11 | 2021-06-11 | Multi-modal cognitive disorder recognition method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110654421.6A CN113558603B (en) | 2021-06-11 | 2021-06-11 | Multi-modal cognitive disorder recognition method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113558603A true CN113558603A (en) | 2021-10-29 |
| CN113558603B CN113558603B (en) | 2023-11-10 |
Family
ID=78161983
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110654421.6A Active CN113558603B (en) | 2021-06-11 | 2021-06-11 | Multi-modal cognitive disorder recognition method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113558603B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114446466A (en) * | 2021-12-22 | 2022-05-06 | 山东师范大学 | Cognitive disorder auxiliary diagnosis system based on surface morphology measurement |
| CN114864051A (en) * | 2022-07-06 | 2022-08-05 | 北京智精灵科技有限公司 | Cognitive improvement method and system based on neural network |
| CN115424108A (en) * | 2022-11-08 | 2022-12-02 | 四川大学 | Cognitive dysfunction evaluation method based on audio-visual fusion perception |
| CN115423054A (en) * | 2022-11-07 | 2022-12-02 | 北京智精灵科技有限公司 | Indefinite training and exciting method and system based on personality characteristics of cognitive disorder patient |
| CN116098587A (en) * | 2023-01-13 | 2023-05-12 | 北京中科睿医信息科技有限公司 | Cognition assessment method, device, equipment and medium based on eye movement |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110286650A1 (en) * | 2008-11-07 | 2011-11-24 | National Brain Research Centre | Ready automated screening, diagnosis & classification technique for alzheimer's disease using magnetic resonance imaging signal from ventricular zone contour of brain |
| KR101317040B1 (en) * | 2012-09-03 | 2013-10-11 | 한양대학교 산학협력단 | Method and system for classifying functional alternation patterns of default mode network in progression of normal aging to mild cognitive impairment and alzheimer's disease |
| CN105726026A (en) * | 2016-01-28 | 2016-07-06 | 电子科技大学 | Mild cognitive impairment disease classifying method based on brain network and brain structure information |
| CN107242873A (en) * | 2017-07-05 | 2017-10-13 | 成都信息工程大学 | A kind of brain network establishing method interacted based on functional MRI psychology physiological |
| CN109087298A (en) * | 2018-08-17 | 2018-12-25 | 电子科技大学 | A kind of Alzheimer's disease MRI image classification method |
| CN110175510A (en) * | 2019-04-10 | 2019-08-27 | 杭州电子科技大学 | Multi-mode Mental imagery recognition methods based on brain function network characterization |
| CN112348833A (en) * | 2020-11-06 | 2021-02-09 | 浙江传媒学院 | Brain function network variation identification method and system based on dynamic connection |
| CN112674720A (en) * | 2020-12-24 | 2021-04-20 | 四川大学 | Alzheimer disease pre-diagnosis method based on 3D convolutional neural network |
-
2021
- 2021-06-11 CN CN202110654421.6A patent/CN113558603B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110286650A1 (en) * | 2008-11-07 | 2011-11-24 | National Brain Research Centre | Ready automated screening, diagnosis & classification technique for alzheimer's disease using magnetic resonance imaging signal from ventricular zone contour of brain |
| KR101317040B1 (en) * | 2012-09-03 | 2013-10-11 | 한양대학교 산학협력단 | Method and system for classifying functional alternation patterns of default mode network in progression of normal aging to mild cognitive impairment and alzheimer's disease |
| CN105726026A (en) * | 2016-01-28 | 2016-07-06 | 电子科技大学 | Mild cognitive impairment disease classifying method based on brain network and brain structure information |
| CN107242873A (en) * | 2017-07-05 | 2017-10-13 | 成都信息工程大学 | A kind of brain network establishing method interacted based on functional MRI psychology physiological |
| CN109087298A (en) * | 2018-08-17 | 2018-12-25 | 电子科技大学 | A kind of Alzheimer's disease MRI image classification method |
| CN110175510A (en) * | 2019-04-10 | 2019-08-27 | 杭州电子科技大学 | Multi-mode Mental imagery recognition methods based on brain function network characterization |
| CN112348833A (en) * | 2020-11-06 | 2021-02-09 | 浙江传媒学院 | Brain function network variation identification method and system based on dynamic connection |
| CN112674720A (en) * | 2020-12-24 | 2021-04-20 | 四川大学 | Alzheimer disease pre-diagnosis method based on 3D convolutional neural network |
Non-Patent Citations (4)
| Title |
|---|
| CONGLING WU: "Discrimination and conversion prediction of mild cognitive impairment using convolutional neural networks", QUANTITATIVE IMAGING IN MEDICINE AND SURGERY, vol. 8, no. 10, pages 992 - 1003 * |
| HAMZA FAROOQ: "Robustness of Brain Structural Networks Is Affected in Cognitively Impaired MS Patients", FRONTIERS IN NEUROLOGY, vol. 11, pages 1 - 8 * |
| JINHUA SHENG: "A novel joint HCPMMP method for automatically classifying Alzheimer’s and different stage MCI patients", BEHAVIOURAL BRAIN RESEARCH, pages 210 - 221 * |
| SEYED HANI HOJJATI: "Identification of the Early Stage of Alzheimer’s Disease Using Structural MRI and Resting-State fMRI", FRONTIERS IN NEUROLOGY, vol. 10, pages 1 - 12 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114446466A (en) * | 2021-12-22 | 2022-05-06 | 山东师范大学 | Cognitive disorder auxiliary diagnosis system based on surface morphology measurement |
| CN114864051A (en) * | 2022-07-06 | 2022-08-05 | 北京智精灵科技有限公司 | Cognitive improvement method and system based on neural network |
| CN114864051B (en) * | 2022-07-06 | 2022-10-04 | 北京智精灵科技有限公司 | Cognitive improvement method and system based on neural network |
| CN115423054A (en) * | 2022-11-07 | 2022-12-02 | 北京智精灵科技有限公司 | Indefinite training and exciting method and system based on personality characteristics of cognitive disorder patient |
| CN115424108A (en) * | 2022-11-08 | 2022-12-02 | 四川大学 | Cognitive dysfunction evaluation method based on audio-visual fusion perception |
| CN116098587A (en) * | 2023-01-13 | 2023-05-12 | 北京中科睿医信息科技有限公司 | Cognition assessment method, device, equipment and medium based on eye movement |
| CN116098587B (en) * | 2023-01-13 | 2023-10-10 | 北京中科睿医信息科技有限公司 | Cognition assessment method, device, equipment and medium based on eye movement |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113558603B (en) | 2023-11-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113558603B (en) | Multi-modal cognitive disorder recognition method based on deep learning | |
| Noreen et al. | A deep learning model based on concatenation approach for the diagnosis of brain tumor | |
| CN113616184B (en) | Brain network modeling and individual prediction method based on multi-mode magnetic resonance image | |
| CN105469376B (en) | The method and apparatus for determining picture similarity | |
| CN110236543A (en) | The more classification diagnosis systems of Alzheimer disease based on deep learning | |
| Shmulev et al. | Predicting conversion of mild cognitive impairments to Alzheimer’s disease and exploring impact of neuroimaging | |
| CN113693563B (en) | Brain function network classification method based on hypergraph attention network | |
| Cheng et al. | Classification of long sequential data using circular dilated convolutional neural networks | |
| CN114469120B (en) | Multi-scale Dtw-BiLstm-Gan electrocardiosignal generation method based on similarity threshold migration | |
| Chou et al. | Optimizing parameters of multi-layer convolutional neural network by modeling and optimization method | |
| CN112418337A (en) | Multi-feature fusion data classification method based on brain function hyper-network model | |
| Lewis et al. | Cooperative learning: Decentralized data neural network | |
| CN115272295A (en) | Dynamic brain function network analysis method and system based on time domain-space domain combined state | |
| Morris et al. | Autism spectrum disorder diagnosis using sparse graph embedding of morphological brain networks | |
| CN107256408B (en) | Method for searching key path of brain function network | |
| CN113558602B (en) | Hypothesis-driven cognitive impairment brain network analysis method | |
| Krishna Priya et al. | Improved particle swarm optimized deep convolutional neural network with super‐pixel clustering for multiple sclerosis lesion segmentation in brain MRI imaging | |
| Semerjian et al. | White blood cells classification using built-in customizable trained convolutional neural network | |
| Wang et al. | Multimodal autism spectrum disorder diagnosis method based on DeepGCN | |
| Yan et al. | Resnet-like CNN architecture and saliency map for human activity recognition | |
| CN108256569A (en) | A kind of object identifying method under complex background and the computer technology used | |
| CN112561935B (en) | Intelligent classification method, device and equipment for brain images | |
| Tajammal et al. | Deep Learning‐Based Ensembling Technique to Classify Alzheimer’s Disease Stages Using Functional MRI | |
| Rajasree et al. | Ensemble-of-classifiers-based approach for early Alzheimer’s Disease detection | |
| Nallusamy et al. | Classifying schizophrenic and controls from fMRI data using graph theoretic framework and community detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |