WO2024083057A1 - 基于多模态磁共振成像的图卷积神经网络疾病预测系统 - Google Patents

基于多模态磁共振成像的图卷积神经网络疾病预测系统 Download PDF

Info

Publication number
WO2024083057A1
WO2024083057A1 PCT/CN2023/124639 CN2023124639W WO2024083057A1 WO 2024083057 A1 WO2024083057 A1 WO 2024083057A1 CN 2023124639 W CN2023124639 W CN 2023124639W WO 2024083057 A1 WO2024083057 A1 WO 2024083057A1
Authority
WO
WIPO (PCT)
Prior art keywords
brain
magnetic resonance
data
neural network
convolutional neural
Prior art date
Application number
PCT/CN2023/124639
Other languages
English (en)
French (fr)
Inventor
张瑜
孙超良
王志超
张欢
钱浩天
蒋田仔
Original Assignee
之江实验室
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 之江实验室 filed Critical 之江实验室
Publication of WO2024083057A1 publication Critical patent/WO2024083057A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0033—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
    • A61B5/004—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room adapted for image acquisition of a particular organ or body part
    • A61B5/0042—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room adapted for image acquisition of a particular organ or body part for the brain
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves 
    • A61B5/055—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves  involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7271—Specific aspects of physiological measurement analysis
    • A61B5/7275—Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10088—Magnetic resonance imaging [MRI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30016—Brain

Definitions

  • the present invention relates to the field of neuroimaging data analysis, and in particular to a graph convolutional neural network disease prediction system based on multimodal magnetic resonance imaging.
  • Multimodal magnetic resonance imaging has imaging information of multiple modalities, providing a variety of physiological indicators for studying various diseases.
  • T1-weighted structural images can segment gray matter and white matter, and can extract cortical information (cortical volume, thickness, surface area, etc.) through cortical reconstruction.
  • Resting-state functional magnetic resonance imaging reflects the neural activity of the brain at rest, and is an important method for studying brain function and brain network connectivity in recent years. Resting-state magnetic resonance data can be used to calculate low-frequency amplitude (ALFF), regional homogeneity (ReHo), and functional connectivity (FC).
  • ALFF low-frequency amplitude
  • ReHo regional homogeneity
  • FC functional connectivity
  • ALFF is used to measure the activity of neurons in different brain regions
  • ReHo describes the synchronization of the time series of a voxel with the surrounding adjacent voxels
  • FC can be used to evaluate the degree of functional association between brain regions.
  • Diffusion magnetic resonance imaging detects the microstructural characteristics of brain white matter and the direction of fiber bundles by measuring the difference in the diffusion of water molecules.
  • Diffusion MRI data can be used for diffusion tensor imaging (DTI) to calculate the anisotropy fraction (FA), mean diffusivity (MD) and other indicators. It can also be used for neurite orientation dispersion and density imaging (NODDI) to calculate the intra-neurite volume fraction (ICVF) and orientation dispersion index (ODI).
  • DTI diffusion tensor imaging
  • NODDI neurite orientation dispersion and density imaging
  • the fiber connection matrix between brain regions can be obtained through the fiber tracking results at the whole brain level, which is used to evaluate the structural connection between brain regions.
  • FA reflects the ratio of the anisotropic part of the diffusion to the total value of the diffusion tensor
  • MD reflects the range of diffusion movement of water molecules per unit time
  • ICVF can reflect the nerve density
  • ODI can quantify the dispersion of neurite direction.
  • Graph convolutional neural network is a method that can extract features from graph data, and then use these features to perform node classification and graph classification on graph data. By processing multimodal magnetic resonance data, the above-mentioned indicators with physiological significance can be obtained.
  • the methods for disease prediction based on magnetic resonance imaging data generally only process single-modality data separately.
  • the method based on support vector machines uses brain surface area and cortical thickness to automatically classify schizophrenia patients (Yuan X, Yan Z, Zhao Y, et al. Support vector machine-based classification of first episode Schizophrenia patients and healthy controls using structural MRI[J].Schizophrenia Research,2017,214), or based on sparse group lasso (Sparse group lasso, SGL) method uses FA value to classify patients with ALS (Richie-Halford A, Yeatman JD, Simon N, et al. Multidimensional analysis and detection of informative features in human brain white matter[J].
  • Parkinsonism&Related Disorders 2018: 55-60
  • Alzheimer's disease Liu X, Guo Z, Ding Y, et al. Abnormal baseline brain activity in Alzheimer's disease patients with depression: a resting-state functional magnetic resonance imaging study[J].
  • Neuroradiology, 2017, 59(7): 709-714 and depression (Vythilingam M, Vermetten E, Anderson G M, et al. Hippocampal volume, memory, and cortisol status in major depressive disorder: effects of treatment.[J].
  • large-scale brain imaging data is a type of unstructured data, which is difficult to perform automated and intelligent data mining and analysis.
  • Medical image data and text data in electronic medical records are common unstructured data in medical data types.
  • the convolutional neural network which has been rapidly developed, has been widely used in the field of image processing, it still has a lot of limitations in the face of complex medical images due to the diversity of imaging modalities, resolutions, and imaging angles. Converting unstructured image data into structured data through reasonable methods is conducive to subsequent data mining and analysis.
  • the purpose of the present invention is to address the deficiencies of the prior art and propose a graph convolutional neural network disease prediction system based on multimodal magnetic resonance imaging, which can integrate information from multimodal data and improve the prediction ability of the model and the generalization ability of the model under different diseases.
  • a graph convolutional neural network disease prediction system based on multimodal magnetic resonance imaging the system comprising the following modules:
  • a multimodal MRI data acquisition module is used to extract information from multimodal MRI data based on brain maps, including structural images, resting state MRI data, and diffusion MRI data;
  • a data preprocessing module used for preprocessing structural images, resting state magnetic resonance data and diffusion magnetic resonance data
  • the brain imaging genomics information extraction module is used to calculate the cortical volume, thickness and surface area information of different brain regions based on the structural images processed by the data preprocessing module; and calculate the cortical volume, thickness and surface area information of different brain regions based on the resting state magnetic resonance data processed by the data preprocessing module.
  • the low-frequency amplitude and local consistency information of the brain were obtained; the anisotropy fraction, mean diffusion coefficient, intracellular volume fraction and directional diffusion fraction information of different brain regions were calculated based on the diffusion magnetic resonance data processed by the data preprocessing module;
  • a brain connectomics information extraction module is used to calculate the functional connection matrix of each subject based on the resting-state magnetic resonance data processed by the data preprocessing module; and to calculate the structural connection matrix of each subject based on the diffusion magnetic resonance data processed by the data preprocessing module;
  • a brain map structure construction module is used to multiply the functional connection matrix and the structural connection matrix obtained by the brain connectomics information extraction module to obtain an adjacency matrix as the edge set of the brain map structure, and each brain region of the brain map is taken as a node set, which contains the brain imaging omics information of the corresponding brain region, and the brain map structure is constructed by the edge set and the node set;
  • the graph convolutional neural network model construction module is used to construct a graph convolutional neural network model. It uses brain map structure data as model input and the group of the subject as the label as model output to train the graph convolutional neural network model, and predicts brain diseases through the trained graph convolutional neural network model.
  • the data preprocessing module preprocesses the structure image by removing the skull and retaining only the brain tissue structure; performs head motion correction and time correction on the resting magnetic resonance data; and performs denoising, head motion correction and eddy current correction on the diffusion magnetic resonance data.
  • the brain imaging genomics information extraction module performs gray-white matter segmentation on the structural image of the brain tissue structure; spatially standardizes the structural image after gray-white matter segmentation, maps it to a unified brain surface template fsaverage, and divides it into different brain regions according to a given brain map; finally, the cortex is reconstructed through the FreeSurfer software to obtain the cortical volume, thickness and surface area information of different brain regions.
  • the brain imaging genomics information extraction module spatially normalizes the corrected resting-state MRI data, linearly aligns it to the structural image and nonlinearly aligns it to the unified brain volume template MNI152NLinin2009cAsym; denoises, bandpass filters, regresses covariates and spatially smoothes the aligned resting-state MRI data, and calculates the low-frequency amplitude ALFF and local consistency ReHo value information of different brain regions; wherein, the frequency of the low-frequency amplitude ALFF is between 0.01Hz-0.1Hz, and the local consistency ReHo value must be calculated before spatial smoothing.
  • the brain imaging genomics information extraction module performs reverse spatial normalization on the denoised and corrected diffusion magnetic resonance data, nonlinearly registers the standard brain template MNI152NLinin2009cAsym to the structural image of each subject, and then linearly registers it to the diffusion image individual space of each subject, and also registers the brain atlas to the diffusion image space of each subject according to the same mapping;
  • the diffusion magnetic resonance data after reverse spatial normalization are fitted with a diffusion tensor imaging DTI model, and the anisotropy fraction FA and the mean diffusion coefficient MD value of each brain region are calculated;
  • the diffusion magnetic resonance data are fitted with a neurite directional diffusion and density imaging model NODDI, and the intracellular volume fraction ICVF and the directional diffusion fraction ODI value of each brain region are calculated.
  • the brain connectomics information extraction module is used to extract basic information from the preprocessed resting state magnetic resonance data.
  • the average time series of each brain region in the standard brain atlas was calculated; the Pearson correlation coefficient of the time series between brain regions was calculated to obtain the Pearson correlation coefficient matrix between brain regions; the Pearson correlation coefficient matrix was subjected to Fisher Z transformation to obtain the functional connection matrix of each subject.
  • the brain connectomics information extraction module is used to estimate the response function of the preprocessed diffusion magnetic resonance data and reconstruct the fiber direction distribution model through spherical constrained deconvolution, and perform whole-brain fiber tracking based on the reconstructed model; the fiber tracking results are screened and normalized based on the brain atlas registered to the diffusion image space to obtain the structural connection matrix of each subject.
  • the brain map structure construction module is used to use each brain region of the brain map as a node, and construct a feature vector for the multimodal information of each brain region, including brain structure indicators of cortical volume, thickness and surface area of different brain regions extracted from structural images, brain function indicators of low-frequency amplitude ALFF and local consistency ReHo value of different brain regions calculated from resting magnetic resonance data, and brain diffusion indicators of anisotropy fraction FA, mean diffusion coefficient MD, intracellular volume fraction ICVF and directional diffusion fraction ODI value of each brain region calculated from diffusion magnetic resonance data; multiply the normalized functional connection matrix and the structural connection matrix as an adjacency matrix; construct brain map structure data G(V,E) based on the adjacency matrix, wherein the node set V is composed of brain regions extracted from the brain map, and the edge set E is composed of the adjacency matrix obtained by multiplication.
  • the graph convolutional neural network model construction module is used to construct a graph convolutional neural network model;
  • the graph convolutional neural network model includes two layers of graph convolutional neural network GCN layers, in which the filter uses a Chebyshev convolution kernel, the order uses 3rd order, and the loss function adopts a cross entropy function;
  • the multimodal magnetic resonance data of the constructed brain map structure data is used as the input of the graph convolutional neural network model, and the group of the subject is used as the label as the output of the graph convolutional neural network model, and the graph convolutional neural network model is trained using back propagation technology.
  • the present invention uses a deep learning method through multimodal data preprocessing, image index extraction and structured data integration, which can better integrate multiple cross-modal physiological indicators of brain regions and the correlation between brain regions, realize the fusion of multimodal information, and the fusion of brain region nodes and brain region connections, thereby improving the predictive ability of the model and the generalization ability of the model under different diseases.
  • FIG1 is a schematic diagram of the structure of a graph convolutional neural network disease prediction system based on multimodal magnetic resonance imaging proposed in the present invention.
  • FIG. 2 is a schematic diagram of constructing brain map structure data proposed by the present invention.
  • FIG3 is a schematic diagram of the graph convolutional neural network model structure proposed in the present invention.
  • the present invention proposes a graph convolutional neural network disease prediction system based on multimodal magnetic resonance imaging, which integrates multimodal magnetic resonance imaging data into graph structured data and predicts diseases using a graph convolutional neural network method.
  • Using the multimodal features calculated by the system to predict diseases can fuse the information of multimodal data and improve the prediction ability of the model and the generalization ability of the model under different diseases.
  • the overall system structure is shown in Figure 1, including a multimodal magnetic resonance data acquisition module, a data preprocessing module, a brain imaging genomics information extraction module, a brain connection genomics information extraction module, a brain map structure construction module and a graph convolutional neural network model construction module;
  • the multimodal magnetic resonance data acquisition module is used to acquire multimodal magnetic resonance brain imaging data, including structural images, resting state magnetic resonance data and diffusion magnetic resonance data;
  • the data preprocessing module is used to preprocess and align each mode of multimodal brain imaging data.
  • Structural image data preprocessing includes removing skull and non-brain tissue, spatial normalization, segmenting gray matter, white matter, and cerebrospinal fluid, cortical reconstruction using the FreeSurfer software package, quantifying the function, connection, and structural properties of the human brain, three-dimensional reconstruction of structural images, generating flattened or expanded images, and obtaining anatomical parameters such as cortical thickness, area, and gray matter volume.
  • Resting state functional magnetic resonance imaging data preprocessing includes: time layer correction, head motion correction, alignment of structural and functional images, spatial normalization, ICA-AROMA denoising, and other steps.
  • the standard spatial resting state functional magnetic resonance imaging data after ICA-AROMA denoising is used for further regression denoising, including: head motion parameters, global signal, white matter signal, cerebrospinal fluid signal, etc.
  • the data is then bandpass filtered and spatially smoothed.
  • the preprocessing of diffusion magnetic resonance imaging data includes: image denoising, distortion correction, extraction of b0 image for signal normalization, head motion and eddy current correction.
  • the diffusion model of the image is reconstructed based on the data b vector distribution, the response function and the fiber direction distribution function are calculated, and the fiber tracking of the whole brain is performed;
  • the brain imaging genomics information extraction module is based on the standard brain atlas, and the data of different modalities and different resolutions are aligned with the standard brain atlas through the previous preprocessing method, and the brain imaging genomics information of each modality is calculated in each brain region, including brain structural indicators such as cortical volume, thickness and surface area of different brain regions extracted from structural images, brain function indicators such as ALFF and ReHo values of different brain regions are calculated in the resting state, and diffusion magnetic resonance imaging is used to calculate the anisotropy fraction FA, mean diffusion coefficient MD, intracellular volume fraction ICVF and directional diffusion fraction ODI value of each brain region.
  • the brain connectomics information extraction module is used to calculate the connectomics information of each modality between each brain region, including the functional connection matrix calculated by resting magnetic resonance imaging data and the structural connection matrix calculated by diffusion magnetic resonance imaging.
  • the normalized brain functional connection matrix and the structural connection matrix are multiplied together to form the adjacency matrix of the graph model.
  • the brain graph structure construction module is used to use the imaging genomics information in the brain region as the node features and integrate the brain connectomics information as the edge weights to construct the graph structure data.
  • the graph convolutional neural network model construction module is used to construct Build a graph convolutional neural network model and input the brain map structure data into the graph convolutional neural network model for disease prediction.
  • the preprocessing process includes: structural image preprocessing, resting state MRI data preprocessing and diffusion MRI data preprocessing.
  • the structural image preprocessing uses the Skull-stripping method of ANTs software to remove the skull of the structural image and only retain the brain tissue structure;
  • the resting state functional MRI data preprocessing first uses the MCFLIRT method of FSL software to correct the resting state data for head motion, and uses the 3dTshift method of AFNI software for time correction.
  • the resting state data is spatially standardized and aligned to the structural image and the standard template using ANTs software.
  • the resting state data is denoised using the ICA-AROMA method, and a frequency domain filter of 0.01Hz-0.1Hz is performed to regress out the signals of white matter and cerebrospinal fluid. Finally, a Gaussian kernel with a half-width of 6mm is used for spatial smoothing.
  • the diffusion MRI data preprocessing first uses the DWIdenoise method of MRtrix3 software to denoise the diffusion MRI data. Then, the head motion correction method of FSL software is used for head motion correction. Finally, eddy current correction is performed using FSL's Eddy method.
  • the process of brain imaging genomics information extraction includes: structural image node information extraction, resting state MRI data node information extraction, and diffusion MRI data node information extraction.
  • the structural image node information extraction uses the FAST method of FSL software to perform gray and white matter segmentation on the structural image.
  • the nonlinear registration method of ANTs software is used to spatially standardize the structural image, map it to the unified brain volume template MNI152NLin2009cAsym, and divide it into different brain regions according to the given brain atlas.
  • the cortical information is obtained by the cortical reconstruction method of FreeSurfer software, and the structural image is mapped to the unified brain surface template fsaverage and registered to each brain region.
  • the node information extraction of resting state MRI data uses the 3dRSFC and 3dReHo functions of AFNI to calculate the ALFF and ReHo values of different brain regions.
  • the ReHo value should be calculated before spatial smoothing;
  • the node information extraction of diffusion MRI data includes reverse spatial normalization of the diffusion MRI data, nonlinear registration of the standard brain template MNI152NLinin2009cAsym to the structural image of each subject, and then linear registration to the diffusion image individual space of each subject, and the brain atlas is also registered to the diffusion image space of each subject according to the same mapping.
  • the DTI model is fitted to the diffusion MRI data, and the anisotropy fraction FA and mean diffusion coefficient MD values of each brain region are calculated.
  • the NODDI Matlab toolbox is used to fit the diffusion MRI data with the neurite directional diffusion and density imaging model NODDI, and the intracellular volume fraction ICVF and directional diffusion fraction ODI values of each brain region are calculated.
  • the brain connectomics information extraction module is used to extract the connection information of resting state magnetic resonance data in the following process: extract the average time series of each brain region based on the standard brain atlas from the preprocessed resting state magnetic resonance data; secondly, calculate the Pearson correlation coefficient of the average time series between brain regions to obtain the Pearson correlation coefficient matrix between each brain region; finally, perform Fisher Z transformation on the Pearson correlation coefficient matrix to obtain the functional connection matrix of each subject.
  • the brain connectomics information extraction module is used to extract the connection information of diffusion magnetic resonance data in the following process: using MRtrix3 software to estimate the response function of the preprocessed diffusion magnetic resonance data and reconstructing the fiber direction distribution model through spherical constrained deconvolution; secondly, whole-brain fiber tracking is performed based on the reconstructed model, with the number of tracking being 10 million; finally, the fiber tracking results are screened based on the SIFT method, the fiber bundles with physiological significance are retained, and the number of fiber bundles is normalized, and the number of fiber connections between every two brain regions is counted to obtain the structural connection matrix of each subject.
  • the brain map structure construction module is used to integrate the structured brain image model data. As shown in Figure 2, each brain region of the selected brain map is used as a node.
  • the multimodal information of each brain region is calculated to construct a feature vector, including brain structural indicators such as cortical volume, thickness and surface area of different brain regions extracted from the structural image, brain functional indicators such as ALFF and ReHo values of different brain regions calculated in the resting state, and brain diffusion indicators such as anisotropy fraction FA, mean diffusion coefficient MD, intracellular volume fraction ICVF and directional diffusion fraction ODI value of each brain region calculated by diffusion magnetic resonance.
  • brain structural indicators such as cortical volume, thickness and surface area of different brain regions extracted from the structural image
  • brain functional indicators such as ALFF and ReHo values of different brain regions calculated in the resting state
  • brain diffusion indicators such as anisotropy fraction FA, mean diffusion coefficient MD, intracellular volume fraction ICVF and directional diffusion fraction ODI value of each brain region calculated by diffusion magnetic resonance.
  • the normalized brain functional connection matrix and structural connection matrix are multiplied as the adjacency matrix of the graph model;
  • the graph structure data G(V,E) is constructed, wherein the node set V is composed of brain regions extracted from the brain map, and the edge set E is composed of the adjacency matrix obtained by multiplication.
  • the graph convolutional neural network model building module is used to build a graph convolutional neural network model.
  • the model includes two GCN layers, a readout layer and a fully connected layer.
  • the readout layer folds the node representation of each graph into a graph representation, and the fully connected layer is used to perform a weighted sum of the previously designed features.
  • the filter uses a Chebyshev convolution kernel with an order of 3, where the Chebyshev convolution kernel is:
  • K represents the order of the Chebyshev convolution kernel
  • x represents the input feature
  • g ⁇ is the convolution kernel
  • ⁇ k is the parameter of the filter
  • Tk is the k-order Chebyshev polynomial
  • the loss function uses the cross entropy function:
  • yik represents the kth cognitive function state label corresponding to the ith sample
  • pik is the probability of belonging to the kth cognitive function state predicted by the graph convolutional neural network model.
  • the data set is randomly divided into two groups according to the ratio of 7:1:2.
  • the training set, validation set and test set are formed.
  • the multimodal magnetic resonance data with the constructed graph structure is used as the input of the graph convolutional neural network model.
  • the group of the subject is used as the label as the output of the graph convolutional neural network model.
  • the graph convolutional neural network model is trained using back-propagation technology.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Public Health (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Pathology (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Veterinary Medicine (AREA)
  • Theoretical Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Neurology (AREA)
  • Signal Processing (AREA)
  • Quality & Reliability (AREA)
  • Psychiatry (AREA)
  • Physiology (AREA)
  • High Energy & Nuclear Physics (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Magnetic Resonance Imaging Apparatus (AREA)

Abstract

本发明公开了一种基于多模态磁共振成像的图卷积神经网络疾病预测系统,从多模态的磁共振数据中提取多个脑区跨模态下的影像组学信息作为节点的特征，提取脑区间的连接组学信息构成邻接矩阵。T1加权结构像通过皮层重建来进行皮层信息提取，静息态磁共振数据用于计算低频振幅，局部一致性以及功能连接。通过多模态数据预处理、影像指标提取和结构化数据整合，将多模态的非结构化磁共振影像数据整合成统一的图结构数据，用图卷积神经网络的方法对疾病进行预测。以本发明计算得到的多模态特征进行疾病预测，可以更好地融合多个脑区跨模态的生理学指标以及脑区之间的相关性并提高模型的预测能力和模型在不同疾病下的泛化能力。

Description

基于多模态磁共振成像的图卷积神经网络疾病预测系统 技术领域
本发明涉及神经影像数据分析领域，尤其涉及一种基于多模态磁共振成像的图卷积神经网络疾病预测系统。
背景技术
多模态磁共振成像(Magnetic resonance imaging，MRI)具有多个模态的影像学信息，为研究各类疾病提供了多种生理学指标。T1加权结构像能够进行灰质和白质的分割，并可通过皮层重建来进行皮层信息提取(皮层体积、厚度和表面积等)。静息态功能磁共振成像反映了大脑在静息状态下的神经活动情况，是近年来研究脑功能和脑网络连接的一种重要方法。静息态磁共振数据可以用于计算低频振幅(Amplitude of low frequency fluctuations,ALFF)，局部一致性(Regional homogeneity,ReHo)以及功能连接(Functional connectivity,FC)。其中，ALFF用于衡量不同脑区神经元的活动强弱，ReHo描述的是某个体素与周围相邻体素时间序列的同步性，FC可用于评估脑区之间的功能关联程度。扩散磁共振成像通过测量水分子扩散差异来检测脑白质的微结构特性以及纤维束走向。扩散磁共振数据可用于扩散张量成像(Diffusion tensor imaging,DTI)，计算各向异性分数(Fractional anisotropy,FA)、平均扩散率(Mean diffusivity,MD)等指标，也可用于神经突方向分散度和密度成像(Neurite orientation dispersion and density imaging,NODDI)，计算神经突内容积比(Intra-neurite volume fraction,ICVF)以及方向分散度(Orientation dispersion index,ODI)等。并且可以通过全脑层面的纤维追踪结果，得到脑区间的纤维连接矩阵，用于评估脑区之间的结构连接。其中，FA反映了扩散的各项异性部分与扩散张量总值的比值，MD反映了水分子单位时间内扩散运动的范围，ICVF可以反映神经密度，ODI可以量化神经突方向的离散度。
图卷积神经网络(Graph convolutional neural network,GCN)是一种可以从图数据中提取特征的方法，从而使用这些特征去对图数据进行节点分类和图分类。通过对多模态磁共振数据的处理，可以得到上述具有生理学意义的指标。
目前基于磁共振数据进行疾病预测的方法，一般只是分别进行单一模态的数据处理，比如基于支持向量机支持向量机(Support vector machines,SVM)的方法用大脑表面积和皮层厚度对精神分裂症患者做自动分类(Yuan X,Yan Z,Zhao Y,et al.Support vector machine-based classification of first episodeschizophrenia patients and healthy controls using structural MRI[J].Schizophrenia Research,2017,214)，或是基于稀疏组套索(Sparse group  lasso,SGL)的方法用FA值来进行渐冻症患者的分类(Richie-Halford A,Yeatman J D,Simon N,et al.Multidimensional analysis and detection of informative features in human brain white matter[J].PLoS Computational Biology,2021,17(6):e1009136)，或是基于SVM用ALFF值对创伤后应激障碍的病人进行分类(Yuan M,Qiu C,Meng Y,et al.Pre-treatment Resting-State Functional MR Imaging Predicts the Long-Term Clinical Outcome After Short-Term Paroxtine Treatment in Post-traumatic Stress Disorder[J].Frontiers in Psychiatry,2018,9)，对于多模态数据的融合还比较欠缺。多模态脑影像指标在以往研究中被证实与帕金森病(Andica C,Kamagata K,Hatano T,et al.Neurite orientation dispersion and density imaging of the nigrostriatal pathway in Parkinson's disease:Retrograde degeneration observed by tract-profile analysis[J].Parkinsonism&Related Disorders,2018:55-60)，阿尔兹海默症(Liu X,Guo Z,Ding Y,et al.Abnormal baseline brain activity in Alzheimer's disease patients with depression:a resting-state functional magnetic resonance imaging study[J].Neuroradiology,2017,59(7):709-714)，抑郁症(Vythilingam M,Vermetten E,Anderson G M,et al.Hippocampal volume,memory,and cortisol status in major depressive disorder:effects of treatment.[J].Biological Psychiatry,2004,56(2):101-112)等各类疾病密切相关。并且大规模脑影像数据是一类非结构化数据，难以进行自动化、智能化程度高的数据挖掘和分析。医学图像数据与电子病历中的文本数据都是医疗数据类型中常见的非结构化数据。尽管目前得到长足发展的卷积神经网络在图像处理领域取得广泛的应用，然而面对结构复杂的医学图像，由于其成像模态、分辨率、成像角度的多样性，仍然存在大量的局限性。通过合理的方法将非结构化的图像数据转化为结构化数据有利于后续的数据挖掘和分析。
发明内容
本发明的目的在于针对现有技术的不足，提出一种基于多模态磁共振成像的图卷积神经网络疾病预测系统，该系统可以融合多模态数据的信息并提高模型的预测能力和模型在不同疾病下的泛化能力。
本发明是通过以下技术方案来实现的：一种基于多模态磁共振成像的图卷积神经网络疾病预测系统，该系统包括如下模块：
多模态磁共振数据获取模块，用于根据脑图谱提取多模态磁共振数据中的信息，包括结构像、静息态磁共振数据和扩散磁共振数据；
数据预处理模块，用于对结构像、静息态磁共振数据和扩散磁共振数据进行预处理；
脑影像组学信息提取模块，用于根据数据预处理模块处理后的结构像计算不同脑区的皮层体积、厚度和表面积信息；根据数据预处理模块处理后的静息态磁共振数据计算不同脑区 的低频振幅和局部一致性信息；根据数据预处理模块处理后的扩散磁共振数据计算不同脑区的各向异性分数、平均弥散系数、细胞内体积分数和方向扩散分数信息；
脑连接组学信息提取模块，用于根据数据预处理模块处理后的静息态磁共振数据计算得到每个被试的功能连接矩阵；根据数据预处理模块处理后的扩散磁共振数据计算得到每个被试的结构连接矩阵；
脑图结构构建模块，用于将脑连接组学信息提取模块得到的功能连接矩阵和结构连接矩阵相乘得到邻接矩阵，作为脑图结构的边集，将脑图谱的各个脑区作为节点集，其中包含对应脑区的脑影像组学信息，由边集和节点集构建得到脑图结构；
图卷积神经网络模型构建模块，用于构建图卷积神经网络模型，以脑图结构数据作为模型输入，以被试所在的组别为标签作为模型输出，对图卷积神经网络模型进行训练，通过训练后的图卷积神经网络模型预测脑疾病。
进一步地，所述数据预处理模块预处理过程具体为对结构像去脑壳，只保留脑组织结构；对静息态磁共振数据进行头动矫正与时间矫正；对扩散磁共振数据进行去噪、头动矫正和涡流矫正。
进一步地，所述脑影像组学信息提取模块对脑组织结构的结构像进行灰白质分割；对灰白质分割后的结构像进行空间标准化，将其映射到统一的脑表面模板fsaverage上，并根据给定的脑图谱分为不同脑区；最后通过freesurfer软件进行皮层重构得到不同脑区的皮层体积、厚度和表面积信息。
进一步地，所述脑影像组学信息提取模块对矫正后的静息态磁共振数据进行空间标准化，将其线性配准到结构像以及非线性配准到统一的脑体积模板MNI152NLinin2009cAsym上；对配准后的静息态磁共振数据进行去噪、带通滤波、回归协变量和空间平滑操作，并计算得到不同脑区的低频振幅ALFF和局部一致性ReHo值信息；其中，低频振幅ALFF的频率0.01Hz-0.1Hz之间，局部一致性ReHo值要在空间平滑前计算。
进一步地，所述脑影像组学信息提取模块对对去噪和校正后的扩散磁共振数据进行反向的空间标准化，将标准脑模板MNI152NLinin2009cAsym非线性配准到每个被试的结构像上，进而线性配准到每个被试的扩散像个体空间，并将脑图谱也根据相同的映射配准到每个被试的扩散像空间；对进行反向的空间标准化后的扩散磁共振数据进行弥散张量成像DTI模型拟合，计算各个脑区的各向异性分数FA和平均弥散系数MD值；最后对扩散磁共振数据进行神经突定向弥散和密度成像模型NODDI拟合，计算各个脑区的细胞内体积分数ICVF和方向扩散分数ODI值。
进一步地，所述脑连接组学信息提取模块用于从预处理后的静息态磁共振数据中提取基 于标准脑图谱的每个脑区的平均时间序列；计算脑区间时间序列的皮尔逊相关系数，得到各个脑区之间的皮尔逊相关系数矩阵；对皮尔逊相关系数矩阵进行费希尔Z变换，得到每个被试的功能连接矩阵。
进一步地，所述脑连接组学信息提取模块用于对预处理后的扩散磁共振数据进行响应函数的估计以及通过球面约束反卷积重建纤维方向分布模型，并基于重建的模型进行全脑纤维追踪；基于配准到扩散像空间的脑图谱对纤维追踪的结果做筛选和归一化，得到每个被试的结构连接矩阵。
进一步地，所述脑图结构构建模块用于以脑图谱的各个脑区作为节点，对得到各个脑区的多模态信息构建特征向量，包括结构像上提取的不同脑区的皮层体积、厚度和表面积的大脑结构指标，静息态磁共振数据上计算得到不同脑区的低频振幅ALFF和局部一致性ReHo值的大脑功能指标，扩散磁共振数据计算得到各个脑区的各向异性分数FA、平均弥散系数MD、细胞内体积分数ICVF和方向扩散分数ODI值的大脑扩散指标；将归一化后的功能连接矩阵和结构连接矩阵相乘，作为邻接矩阵；基于邻接矩阵构建脑图结构数据G(V,E)，其中节点集V由脑图谱中提取的脑区构成，边集E由相乘得到的邻接矩阵构成。
进一步地，所述图卷积神经网络模型构建模块用于构建图卷积神经网络模型；图卷积神经网络模型包括两层图卷积神经网络GCN层，其中滤波器使用切比雪夫Chebyshev卷积核，阶数使用3阶，损失函数采用交叉熵函数；以构建的脑图结构数据的多模态磁共振数据作为图卷积神经网络模型的输入，以被试所在的组别为标签，作为图卷积神经网络模型的输出，利用反向传播技术进行图卷积神经网络模型训练。
本发明的有益效果：本发明通过多模态数据预处理、影像指标提取和结构化数据整合，使用深度学习的方法，可以更好的融合多个脑区跨模态的生理学指标以及脑区之间的相关性，实现了多模态信息的融合，以及脑区节点与脑区连接的融合，提高了模型的预测能力和模型在不同疾病下的泛化能力。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图做简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动前提下，还可以根据这些附图获得其他附图。
图1是本发明提出的基于多模态磁共振成像的图卷积神经网络疾病预测系统结构示意图。
图2是本发明提出的脑图结构数据构建示意图。
图3是本发明提出的图卷积神经网络模型结构示意图。
具体实施方式
下面将结合附图对本发明作进一步的说明。为了使本领域的人员更好地理解本申请中的技术方案，下面将结合附图对本发明作进一步的说明。但这仅仅是本申请的一部分实施例，而不是全部的实施例。基于本申请所述的具体实施例，本领域的其他人员在没有做出创造性劳动的前提下所获得的其他实施例，都应当落在本发明的构思范围之内。
总体而言，本发明提出一种基于多模态磁共振成像的图卷积神经网络疾病预测系统，将多模态的磁共振影像数据整合成图结构的数据，用图卷积神经网络的方法对疾病进行预测。以该系统计算得到的多模态特征进行疾病预测，可以融合多模态数据的信息并提高模型的预测能力和模型在不同疾病下的泛化能力。整体系统结构如图1所示，包括多模态磁共振数据获取模块、数据预处理模块、脑影像组学信息提取模块、脑连接组学信息提取模块、脑图结构构建模块和图卷积神经网络模型构建模块；所述多模态磁共振数据获取模块用于获取多模态磁共振的脑影像数据，包括结构像、静息态磁共振数据和扩散磁共振数据；所述数据预处理模块用于对多模态的脑影像数据的各个模态进行预处理和配准。结构像数据预处理包括去除颅骨和非脑组织、空间标准化、对图像进行灰质、白质、脑脊液的分割、通过FreeSurfer软件包进行皮层重建，量化人脑的功能、连接以及结构属性，对结构像进行三维重建，生成展平或胀平图像，并得到皮质厚度、面积、灰质容积等解剖参数。静息态功能磁共振数据预处理包括：时间层校正，头动校正，结构像和功能像对齐，空间标准化，ICA-AROMA去噪等步骤。完成上述步骤后，使用ICA-AROMA去噪后的标准空间静息态功能磁共振数据进行进一步的回归去噪，包括：头动参数，全局信号，白质信号，脑脊液信号等。随后对数据进行带通滤波及空间平滑。扩散磁共振影像数据的预处理包括：图像去噪、畸变矫正、提取b0图像做信号归一化、头动和涡流矫正。完成上述步骤后，基于数据b向量分布情况对图像进行弥散模型重建，计算响应函数和纤维方向分布函数，进行全脑的纤维追踪；所述脑影像组学信息提取模块基于标准的脑图谱，将不同模态不同分辨率的数据通过前面的预处理方法与标准脑图谱配准，在各个脑区内计算各模态的脑影像组学信息，包括结构像上提取的不同脑区的皮层体积、厚度和表面积等大脑结构指标，静息态上计算得到不同脑区的ALFF和ReHo值等大脑功能指标，扩散磁共振计算得到各个脑区的各向异性分数FA，平均弥散系数MD，细胞内体积分数ICVF和方向扩散分数ODI值等。所述脑连接组学信息提取模块用于在各个脑区间计算各模态的连接组学信息，包括静息态磁共振数据计算得到的功能连接矩阵，扩散磁共振计算得到的结构连接矩阵，将归一化后的大脑功能连接矩阵和结构连接矩阵相乘，作为图模型的邻接矩阵；所述脑图结构构建模块用于将脑区内的影像组学信息作为节点特征，整合脑连接组学信息作为边的权重构建图结构的数据；所述图卷积神经网络模型构建模块用于构 建图卷积神经网络模型，将脑图结构数据输入到图卷积神经网络模型进行疾病预测。
本发明系统的具体实施过程如下：
预处理过程包括：结构像预处理，静息态磁共振数据预处理以及扩散磁共振数据预处理。其中，结构像预处理调用ANTs软件的Skull-stripping方法对结构像去脑壳，只保留脑组织结构；静息态功能磁共振数据预处理首先使用FSL软件的MCFLIRT方法对静息态数据进行头动矫正，用AFNI软件的3dTshift方法进行时间矫正。其次，对静息态数据进行空间标准化，用ANTs软件将其配准到结构像以及标准模板上。然后，对静息态数据用ICA-AROMA的方法进行去噪，并进行0.01Hz-0.1Hz的频域滤波，回归掉白质和脑脊液的信号。最后，用半峰全宽为6mm的高斯核进行空间平滑。扩散磁共振数据预处理首先用MRtrix3软件的DWIdenoise方法对扩散磁共振数据进行去噪。然后，用FSL软件的head motion correction的方法进行头动矫正。最后，用FSL的Eddy方法进行涡流矫正。
脑影像组学信息提取过程包括：结构像影像节点信息提取，静息态磁共振数据节点信息提取，扩散磁共振数据节点信息提取。其中，结构像影像节点信息提取调用FSL软件的FAST方法对结构像进行灰白质分割。其次，使用ANTs软件的非线性配准方法对结构像进行空间标准化，将其映射到统一的脑体积模板MNI152NLin2009cAsym上，并根据给定的脑图谱分为不同脑区。然后，通过FreeSurfer软件的皮层重构方法得到皮层信息，将结构像映射到统一的脑表面模板fsaverage上并将其配准到各个脑区。最后，得到不同脑区的皮层体积、厚度和表面积等信息；静息态磁共振数据节点信息提取调用AFNI的3dRSFC和3dReHo函数计算得到不同脑区的ALFF和ReHo值等信息。特别地，ReHo值要在空间平滑前计算；扩散磁共振数据节点信息提取包括对扩散磁共振数据进行反向的空间标准化，将标准脑模板MNI152NLinin2009cAsym非线性配准到每个被试的结构像上，进而线性配准到每个被试的扩散像个体空间，并将脑图谱也根据相同的映射配准到每个被试的扩散像空间。然后，对扩散磁共振数据进行DTI模型拟合，计算各个脑区的各向异性分数FA和平均弥散系数MD值。最后，用NODDI Matlab toolbox对扩散磁共振数据进行神经突定向弥散和密度成像模型NODDI拟合，计算各个脑区的细胞内体积分数ICVF和方向扩散分数ODI值。
所述脑连接组学信息提取模块用于静息态磁共振数据连接信息提取的过程具体为：从预处理后的静息态磁共振数据中提取基于标准脑图谱的每个脑区的平均时间序列；其次，计算脑区间平均时间序列的皮尔逊相关系数，得到各个脑区之间的皮尔逊相关系数矩阵；最后，对皮尔逊相关系数矩阵进行费希尔Z变换，得到每个被试的功能连接矩阵。其中，费希尔Z变换公式为：
f＝ar tanh(r)
其中，r是时间序列的皮尔逊相关系数，范围为[-1，1]，f是经过Fisher’s Z变换的结果。
所述脑连接组学信息提取模块用于扩散磁共振数据连接信息提取的过程具体为：使用MRtrix3软件对预处理后的扩散磁共振数据进行响应函数的估计以及通过球面约束反卷积重建纤维方向分布模型；其次，基于重建的模型进行全脑纤维追踪，追踪数目为1000万条；最后，基于SIFT方法对纤维追踪的结果做筛选，保留有生理学意义的纤维束，并对纤维束数量做归一化，统计每两个脑区之间的纤维连接数目，得到每个被试的结构连接矩阵。
所述脑图结构构建模块用于对结构化脑影像图模型数据整合，如图2所示，以选定脑图谱的各个脑区作为节点，上述计算得到各个脑区的多模态信息构建特征向量，包括结构像上提取的不同脑区的皮层体积、厚度和表面积等大脑结构指标，静息态上计算得到不同脑区的ALFF和ReHo值等大脑功能指标，扩散磁共振计算得到各个脑区的各向异性分数FA，平均弥散系数MD，细胞内体积分数ICVF和方向扩散分数ODI值等大脑扩散指标；其次，将归一化后的大脑功能连接矩阵和结构连接矩阵相乘，作为图模型的邻接矩阵；再次，构建图结构数据G(V,E)，其中节点集V由脑图谱中提取的脑区构成，边集E由相乘得到的邻接矩阵构成。
所述图卷积神经网络模型构建模块用于构建图卷积神经网络模型，如图3所示，模型包括两层GCN层，读出层以及全连接层。读出层将每个图的节点表示折叠为图表示，全连接层用于对前面设计的特征做加权和。其中滤波器使用Chebyshev卷积核，阶数使用3阶，其中Chebyshev卷积核是：
其中，K表示Chebyshev卷积核阶数，x表示输入特征，gθ为卷积核，θk是滤波器的参数，Tk为k阶的Chebyshev多项式，是正则化后的Laplacian矩阵，由邻接矩阵计算得到。损失函数采用交叉熵函数：
其中yik表示第i个样本所对应的第k个认知功能状态标签，pik为图卷积神经网络模型预测的属于第k个认知功能状态的概率。训练过程将数据集以被试为单位按7:1:2的比例随机分 成训练集、验证集和测试集，以构建的图结构的多模态磁共振数据作为图卷积神经网络模型的输入，以被试所在的组别为标签，作为图卷积神经网络模型的输出，利用反向传播技术进行图卷积神经网络模型训练。
上述实施例用来解释说明本发明，而不是对本发明进行限制，在本发明的精神和权利要求的保护范围内，对本发明作出的任何修改和改变，都落入本发明的保护范围。

Claims (8)

  1. 一种基于多模态磁共振成像的图卷积神经网络疾病预测系统，其特征在于，该系统包括如下模块：
    多模态磁共振数据获取模块，用于根据脑图谱提取多模态磁共振数据中的信息，包括结构像、静息态磁共振数据和扩散磁共振数据；
    数据预处理模块，用于对结构像、静息态磁共振数据和扩散磁共振数据进行预处理；
    脑影像组学信息提取模块，用于根据数据预处理模块处理后的结构像计算不同脑区的皮层体积、厚度和表面积信息；根据数据预处理模块处理后的静息态磁共振数据计算不同脑区的低频振幅和局部一致性信息；根据数据预处理模块处理后的扩散磁共振数据计算不同脑区的各向异性分数、平均弥散系数、细胞内体积分数和方向扩散分数信息；
    脑连接组学信息提取模块，用于根据数据预处理模块处理后的静息态磁共振数据计算得到每个被试的功能连接矩阵；根据数据预处理模块处理后的扩散磁共振数据计算得到每个被试的结构连接矩阵；
    所述脑图结构构建模块用于以脑图谱的各个脑区作为节点，对得到各个脑区的多模态信息构建特征向量，包括结构像上提取的不同脑区的皮层体积、厚度和表面积的大脑结构指标，静息态磁共振数据上计算得到不同脑区的低频振幅ALFF和局部一致性ReHo值的大脑功能指标，扩散磁共振数据计算得到各个脑区的各向异性分数FA、平均弥散系数MD、细胞内体积分数ICVF和方向扩散分数ODI值的大脑扩散指标；将归一化后的功能连接矩阵和结构连接矩阵相乘，作为邻接矩阵；基于邻接矩阵构建脑图结构数据G(V,E)，其中节点集V由脑图谱中提取的脑区构成，边集E由相乘得到的邻接矩阵构成；
    图卷积神经网络模型构建模块，用于构建图卷积神经网络模型，以脑图结构数据作为模型输入，以被试所在的组别为标签作为模型输出，对图卷积神经网络模型进行训练，通过训练后的图卷积神经网络模型预测脑疾病。
  2. 根据权利要求1所述的基于多模态磁共振成像的图卷积神经网络疾病预测系统，其特征在于，所述数据预处理模块预处理过程具体为对结构像去脑壳，只保留脑组织结构；对静息态磁共振数据进行头动矫正与时间矫正；对扩散磁共振数据进行去噪、头动矫正和涡流矫正。
  3. 根据权利要求2所述的基于多模态磁共振成像的图卷积神经网络疾病预测系统，其特征在于，所述脑影像组学信息提取模块对脑组织结构的结构像进行灰白质分割；对灰白质分割后的结构像进行空间标准化，将其映射到统一的脑表面模板fsaverage上，并根据给定的脑 图谱分为不同脑区；最后通过freesurfer软件进行皮层重构得到不同脑区的皮层体积、厚度和表面积信息。
  4. 根据权利要求2所述的基于多模态磁共振成像的图卷积神经网络疾病预测系统，其特征在于，所述脑影像组学信息提取模块对矫正后的静息态磁共振数据进行空间标准化，将其线性配准到结构像以及非线性配准到统一的脑体积模板MNI152NLinin2009cAsym上；对配准后的静息态磁共振数据进行去噪、带通滤波、回归协变量和空间平滑操作，并计算得到不同脑区的低频振幅ALFF和局部一致性ReHo值信息；其中，低频振幅ALFF的频率0.01Hz-0.1Hz之间，局部一致性ReHo值要在空间平滑前计算。
  5. 根据权利要求2所述的基于多模态磁共振成像的图卷积神经网络疾病预测系统，其特征在于，所述脑影像组学信息提取模块对对去噪和校正后的扩散磁共振数据进行反向的空间标准化，将标准脑模板MNI152NLinin2009cAsym非线性配准到每个被试的结构像上，进而线性配准到每个被试的扩散像个体空间，并将脑图谱也根据相同的映射配准到每个被试的扩散像空间；对进行反向的空间标准化后的扩散磁共振数据进行弥散张量成像DTI模型拟合，计算各个脑区的各向异性分数FA和平均弥散系数MD值；最后对扩散磁共振数据进行神经突定向弥散和密度成像模型NODDI拟合，计算各个脑区的细胞内体积分数ICVF和方向扩散分数ODI值。
  6. 根据权利要求1所述的基于多模态磁共振成像的图卷积神经网络疾病预测系统，其特征在于，所述脑连接组学信息提取模块用于从预处理后的静息态磁共振数据中提取基于标准脑图谱的每个脑区的平均时间序列；计算脑区间时间序列的皮尔逊相关系数，得到各个脑区之间的皮尔逊相关系数矩阵；对皮尔逊相关系数矩阵进行费希尔Z变换，得到每个被试的功能连接矩阵。
  7. 根据权利要求5所述的基于多模态磁共振成像的图卷积神经网络疾病预测系统，其特征在于，所述脑连接组学信息提取模块用于对预处理后的扩散磁共振数据进行响应函数的估计以及通过球面约束反卷积重建纤维方向分布模型，并基于重建的模型进行全脑纤维追踪；基于配准到扩散像空间的脑图谱对纤维追踪的结果做筛选和归一化，得到每个被试的结构连接矩阵。
  8. 根据权利要求1所述的基于多模态磁共振成像的图卷积神经网络疾病预测系统，其特征在于，所述图卷积神经网络模型构建模块用于构建图卷积神经网络模型；图卷积神经网络模型包括两层图卷积神经网络GCN层，其中滤波器使用切比雪夫Chebyshev卷积核，阶数使用3阶，损失函数采用交叉熵函数；以构建的脑图结构数据的多模态磁共振数据作为图卷积神经网络模型的输入，以被试所在的组别为标签，作为图卷积神经网络模型的输出，利用反 向传播技术进行图卷积神经网络模型训练。
PCT/CN2023/124639 2022-10-19 2023-10-16 基于多模态磁共振成像的图卷积神经网络疾病预测系统 WO2024083057A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211276172.2A CN115359045B (zh) 2022-10-19 2022-10-19 基于多模态磁共振成像的图卷积神经网络疾病预测系统
CN202211276172.2 2022-10-19

Publications (1)

Publication Number Publication Date
WO2024083057A1 true WO2024083057A1 (zh) 2024-04-25

Family

ID=84007779

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/124639 WO2024083057A1 (zh) 2022-10-19 2023-10-16 基于多模态磁共振成像的图卷积神经网络疾病预测系统

Country Status (2)

Country Link
CN (1) CN115359045B (zh)
WO (1) WO2024083057A1 (zh)

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115359045B (zh) * 2022-10-19 2023-03-17 之江实验室 基于多模态磁共振成像的图卷积神经网络疾病预测系统
WO2024119338A1 (zh) * 2022-12-05 2024-06-13 中国科学院深圳先进技术研究院 知识和数据驱动的脑网络计算方法、装置、电子设备及存储介质
CN116309336B (zh) * 2023-01-20 2024-06-11 首都医科大学宣武医院 一种血管认知障碍的关键影像标记物的提取方法
CN116503680B (zh) * 2023-06-30 2023-08-29 之江实验室 基于脑图谱的脑影像结构化分析和脑疾病分类系统
CN116664605B (zh) * 2023-08-01 2023-10-10 昆明理工大学 基于扩散模型和多模态融合的医学图像肿瘤分割方法
CN116740221B (zh) * 2023-08-16 2023-10-20 之江实验室 实时脑功能激活图生成方法、装置、计算机设备和介质
CN116883396B (zh) * 2023-09-06 2023-11-28 天津医科大学 一种基于人工智能的静息态磁共振图像分析方法与系统
CN117058514B (zh) * 2023-10-12 2024-04-02 之江实验室 基于图神经网络的多模态脑影像数据融合解码方法和装置
CN117079093B (zh) * 2023-10-18 2023-12-12 澄影科技(北京)有限公司 一种基于多模态影像数据的异动症预测方法、装置及设备
CN117095824B (zh) * 2023-10-19 2024-04-16 之江实验室 基于孪生脑仿真模型的多巴胺动态耦合方法、装置和设备
CN117095823B (zh) * 2023-10-19 2024-04-16 之江实验室 基于孪生脑仿真模型的药物成瘾关联脑区确定系统
CN117789988B (zh) * 2024-02-27 2024-06-11 首都医科大学附属北京友谊医院 训练预测帕金森步态障碍的预测模型的方法及相关产品
CN117942080A (zh) * 2024-03-27 2024-04-30 北京大学第三医院（北京大学第三临床医学院） 基于多模态影像组学的肌萎缩侧索硬化抑郁风险评估系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107658018A (zh) * 2017-10-12 2018-02-02 太原理工大学 一种基于结构连接和功能连接的融合脑网络构建方法
CN113539435A (zh) * 2021-09-17 2021-10-22 之江实验室 一种基于图模型的脑功能配准方法
WO2022147871A1 (zh) * 2021-01-09 2022-07-14 深圳先进技术研究院 影像驱动的脑图谱构建方法、装置、设备及存储介质
CN115116607A (zh) * 2022-08-30 2022-09-27 之江实验室 一种基于静息态磁共振迁移学习的脑疾病预测系统
CN115359045A (zh) * 2022-10-19 2022-11-18 之江实验室 基于多模态磁共振成像的图卷积神经网络疾病预测系统

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3965117A1 (en) * 2020-09-03 2022-03-09 GE Precision Healthcare LLC Multi-modal computer-aided diagnosis systems and methods for prostate cancer
CN114334140B (zh) * 2022-03-08 2022-07-19 之江实验室 基于多关系功能连接矩阵的疾病预测系统及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107658018A (zh) * 2017-10-12 2018-02-02 太原理工大学 一种基于结构连接和功能连接的融合脑网络构建方法
WO2022147871A1 (zh) * 2021-01-09 2022-07-14 深圳先进技术研究院 影像驱动的脑图谱构建方法、装置、设备及存储介质
CN113539435A (zh) * 2021-09-17 2021-10-22 之江实验室 一种基于图模型的脑功能配准方法
CN115116607A (zh) * 2022-08-30 2022-09-27 之江实验室 一种基于静息态磁共振迁移学习的脑疾病预测系统
CN115359045A (zh) * 2022-10-19 2022-11-18 之江实验室 基于多模态磁共振成像的图卷积神经网络疾病预测系统

Also Published As

Publication number Publication date
CN115359045B (zh) 2023-03-17
CN115359045A (zh) 2022-11-18

Similar Documents

Publication Publication Date Title
WO2024083057A1 (zh) 基于多模态磁共振成像的图卷积神经网络疾病预测系统
CN113571195B (zh) 基于小脑功能连接特征的阿尔茨海默病早期的预测模型
Liu et al. Classification of Alzheimer's disease using whole brain hierarchical network
Song et al. An effective multimodal image fusion method using MRI and PET for Alzheimer's disease diagnosis
Amico et al. Mapping hybrid functional-structural connectivity traits in the human connectome
Stanley et al. Defining nodes in complex brain networks
Zhang et al. Characterization of U-shape streamline fibers: methods and applications
CN107944490B (zh) 一种基于半多模态融合特征约简框架的图像分类方法
Liu et al. An enhanced multi-modal brain graph network for classifying neuropsychiatric disorders
Hu et al. Localizing sources of brain disease progression with network diffusion model
Ortiz et al. Learning longitudinal MRI patterns by SICE and deep learning: Assessing the Alzheimer’s disease progression
Yang et al. Large-scale brain functional network integration for discrimination of autism using a 3-D deep learning model
Zhu et al. Discovering dense and consistent landmarks in the brain
Sharma et al. Deep-learning-based diagnosis and prognosis of Alzheimer’s disease: A comprehensive review
Chen et al. Early identification of Alzheimer’s disease using an ensemble of 3D convolutional neural networks and magnetic resonance imaging
Brown et al. Predictive connectome subnetwork extraction with anatomical and connectivity priors
CN115170540A (zh) 一种基于多模态影像特征融合的轻度创伤性脑损伤分类方法
Xu et al. Deep learning-based automated detection of arterial vessel wall and plaque on magnetic resonance vessel wall images
Yang et al. Diagnosis of Parkinson’s disease based on 3D ResNet: The frontal lobe is crucial
CN114373095A (zh) 基于影像信息的阿尔茨海默病分类系统及方法
Jiao et al. Constructing dynamic functional networks via weighted regularization and tensor low-rank approximation for early mild cognitive impairment classification
Li et al. Identification of Mild cognitive impairment based on quadruple GCN model constructed with multiple features from higher-order brain connectivity
CN116051545B (zh) 一种双模态影像的脑龄预测方法
Chen et al. A deep learning-based model for classification of different subtypes of subcortical vascular cognitive impairment with FLAIR
Yeung et al. Pipeline comparisons of convolutional neural networks for structural connectomes: predicting sex across 3,152 participants

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23879046

Country of ref document: EP

Kind code of ref document: A1