WO2022047613A1

WO2022047613A1 - System and method for analyzing ad or mci or cn disease development trend by means of multi-modal modeling

Info

Publication number: WO2022047613A1
Application number: PCT/CN2020/112799
Authority: WO
Inventors: 朱帆; 尚明生; 陈琳; 高高
Original assignee: 中国科学院重庆绿色智能技术研究院
Priority date: 2020-09-01
Filing date: 2020-09-01
Publication date: 2022-03-10
Also published as: CN114449948A

Abstract

The present invention belongs to the field of computer application, and particularly relates to a system and method for analyzing an AD or MCI or CN disease development trend by means of multi-modal modeling. The system comprises: a model A composed of clinical medicine data and an apolipoprotein E data input module and a non-linear classifier; a model B composed of a genomic data input module and a non-linear classifier; a model C composed of MRI data, a convolutional neural network and a recurrent neural network; an ADNI data set; and a model D composed of prediction value receiving modules of the model A, the model B and the model C, and a logistic regression model, wherein the ADNI data set provides the clinical medicine data, apolipoprotein E data, genomic data and the MRI data. On the basis of a traditional method and a deep learning method, by means of the present invention, the two methods are combined by means of multi-modal modeling, and more a priori information and external features are added, such that the generalization ability of a model can be enhanced to a certain extent.

Description

System and analysis method for multimodal analysis of AD or MCI or CN disease development trend

technical field

The invention belongs to the field of computer application, and in particular relates to a system and an analysis method for multimodal modeling and analyzing the development trend of AD, MCI or CN.

Background technique

Most of the existing research methods are mainly divided into traditional methods, deep learning based and multimodal methods. The traditional method is to extract medical features from MRI images of subjects, including volume calculation of hippocampus, three-dimensional texture analysis and other methods. The extracted characteristic clinical medical data and genetic data are combined to construct a classifier to realize the classification of AD, MCI and NC and the prediction of the changing trend of patients. This method of manually extracting features tends to take a lot of time and is susceptible to the influence of subjective individuals. With the continuous development of deep learning technology, methods using features extracted by neural networks perform better on many tasks. MRI images are modeled by convolutional neural networks, avoiding the need for manual feature extraction. Convolutional neural network can analyze the spatial features of MRI images, and the deep structure of the network can explicitly use the spatial structure of brain images to gradually extract spatial features from low, medium and high levels for classification. Recurrent neural networks enable longitudinal analysis of sequential MRI, modeling and measuring disease progression with images at multiple time points. Finally, the features extracted by the deep neural network are also combined with clinical medical data and genetic data, or external features are directly added to the neural network for end-to-end learning. In addition, since the feature extraction of the entire MRI image is too complicated and accompanied by a large amount of noise, the segmentation and registration methods based on deep learning are also gradually applied to the ROI extraction of MRI images. Both traditional methods and deep learning-based methods are of great significance for the early identification and prediction of Alzheimer's disease. On the basis of traditional methods and deep learning methods, multimodal modeling combines the two, adding more prior information and external features, which can strengthen the generalization ability of the model to a certain extent.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a multimodal analysis system for AD or MCI or CN disease development trend.

Specifically, CN: that is, cognitively normal, and it can be considered that the population in the CN stage is the normal elderly (healthy elderly); AD is Alzheimer's disease.

Said includes:

Model A composed of clinical medical data and apolipoprotein E data input module and nonlinear classifier;

Model B composed of genomic data input module and nonlinear classifier;

Model C composed of MRI data and convolutional neural network and recurrent neural network;

ADNI dataset;

Model D composed of the predicted value receiving module of the model A, the model B and the model C and the logistic regression model;

The ADNI dataset provides the clinical medical data and the Apolipoprotein E data, the genomic data, the MRI data.

Further, the nonlinear classifier is a support vector machine.

Further, the model C further includes a pre-trained segmentation model and a logistic regression model for processing the MRI data.

Further, the system is loaded on the terminal.

Further, the terminals include fixed terminals and mobile terminals.

Specifically, model A will predict a score_A based on the individual's clinical and APOE information; model B will predict a score_B based on the individual's genomic data; model C will predict a score_C based on MRI data; model D uses logistic regression , using score_A, score_B and score_C as features for training. The input of model D has only three features, and the output is the confidence that the individual is a positive example.

ADNI dataset and even most datasets do not have all the information of individuals. For example, sample A has only clinical data, sample B has only clinical and genetic data, and sample C has only image data. The samples with three complete data at the same time are only a small part. If only the samples with all complete data are used for modeling, If the number of samples is too small, the generalization ability of the model is poor and unconvincing. Therefore, we build three independent models, each of which has enough data support to ensure good generalization performance, and then use the scores of the three independent models as features and train a model with only 3 features as In the final model, although there are few samples with complete data, the number of features is only 3, and it is not easy to overfit.

For example: Suppose the data of 2000 individuals are collected in the ADNI dataset, but there are only 400 individuals with clinical, genetic and image information at the same time. If only one model is used for modeling, then our dataset has only 400 samples , the amount of data is too small. But among these 2000 individuals, 1800 have complete clinical and APOE data, 1200 have complete genetic data, and 800 have complete image data, so we can use three independent models to model 1800, 1200, 800 individuals, In this way, the data volume of each model can be greatly increased, and then the scores of these three independent models are sent to model D as features for training. Since there are only three features, there is almost no obvious overfitting.

In the ADNI dataset, the MMSE score of the individual at the current moment and the MMSE score of the individual after 24 months are provided. A more generally accepted criterion is that the MMSE score of the individual drops by more than 3 after 24 months, and the individual is considered to be very likely. Converted to AD. The model actually predicts the probability of an individual's MMSE falling by more than 3 within 24 months.

The purpose of the present invention is to also provide the aforementioned method for systematically analyzing the development trend of individual CN or MCI disease.

The method includes the following steps:

S1: ADNI dataset sample training

Use the clinical medical data and apolipoprotein E data of the samples in the ADNI data set to input the model A to predict the confidence a of the conversion of the sample CN or MCI into AD;

Using the genomic data data of the samples in the ADNI dataset to input the model B to predict the confidence level b that the CN or MCI of the sample is converted into AD;

Utilize the MRI data of the samples in the ADNI data set to input the model C to predict the confidence c that the sample CN or MCI is converted into AD;

Using the confidence level a, the confidence level b and the confidence level c as input features, and using the model D for training to obtain the confidence level that the sample CN or MCI is converted into AD;

S2: Individual CN or MCI training

Inputting the clinical medical data and apolipoprotein E data of the individual CN or MCI into the model A to predict the confidence a1 that the individual CN or MCI is converted into AD;

The genomic data of the individual CN or MCI is input into the model B to predict the confidence level b1 that the sample CN or MCI is converted into AD;

The MRI data of the individual CN or MCI is input into the model C to predict the confidence level c1 that the sample CN or MCI is converted into AD;

The confidence level a1, the confidence level b1, and the confidence level c1 are used as input features, and the model D is used for training to obtain the confidence level g that the individual CN or MCI is converted into AD.

Further, comparing the confidence level g with the confidence level, if the confidence level g falls within the confidence level, the individual CN or MCI may be converted into AD; if the confidence level g does not fall within the confidence level confidence, the individual CN or MCI is unlikely to convert to AD.

Specifically, the MMSE score of the individual at the current moment and the MMSE score of the individual after 24 months are provided in the ADNI data set. A more generally accepted criterion is that the MMSE score of the individual decreases by more than 3 after 24 months, and the individual is considered to be the individual. Most likely to convert to AD. The model actually predicts the probability of an individual's MMSE falling by more than 3 within 24 months.

Further, in S1 or S2, standardization processing and histogram equalization processing are performed on the MRI image data to enhance the image contrast, and then a non-parametric non-uniform intensity normalization (N3) algorithm is used to correct the intensity inhomogeneity, and the skull is peeled off. Extract ROIs using a pretrained segmentation model.

Further, in S2, described adopting the convolutional neural network and the cyclic neural network, using the MRI image data to predict that the individual CN or MCI is converted into the confidence level c of AD is specifically:

S1: Extract ROI by using the pre-trained segmentation model;

S2: using the convolutional neural network to extract spatial features;

S3: using the cyclic neural network to extract time series features;

S4: Use the logistic regression model to make predictions.

The purpose of the present invention is to also provide an application of the aforementioned system in predicting the development trend of AD or MCI or CN disease of an individual.

Further, the model A predicts the confidence A of the conversion of the MCI or CN into the AD according to clinical medical data and apolipoprotein E data; the model B predicts the conversion of the MCI or CN into the AD according to the genomic data. the confidence level B of the model C; the model C predicts the confidence level C of the conversion of the MCI or CN into the AD according to the MRI data; the confidence level A, the confidence level B and the confidence level C are used as input features using the The model D described above is trained.

The beneficial effects of the present invention are:

The present invention models the MRI image through the convolutional neural network, and avoids manual feature extraction.

The present invention performs longitudinal analysis of sequential MRI through a recurrent neural network, and uses images to model and measure disease progression at multiple time points.

Based on the traditional method and the deep learning method, the present invention combines the two with multimodal modeling, adds more prior information and external features, and can strengthen the generalization ability of the model to a certain extent.

The genetic data and the image data in the present invention contain a lot of useful information, and the model accuracy and model generalization performance are effectively improved after adding the two kinds of data.

Description of drawings

Figure 1 is a simplified flowchart of the multimodal analysis of the disease development trend of AD/MCI/CN patients.

FIG. 2 is a flowchart of processing an MRI image using a Convolutional Neural Network (CNN) and a Recurrent Neural Network (RNN).

detailed description

The examples are given to better illustrate the present invention, but the content of the present invention is not limited to the examples. Therefore, those skilled in the art make non-essential improvements and adjustments to the embodiments according to the above-mentioned contents of the invention, which still belong to the protection scope of the present invention.

Example 1

The invention provides a multimodal analysis system for AD or MCI or CN disease development trend, comprising: a model A composed of a clinical medical data and apolipoprotein E (APOE) data input module and a nonlinear classifier; a genome data input module Model B composed of nonlinear classifiers; Model C composed of MRI data and Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNNs); ADNI datasets; ...consists of Model D.

Among them, the nonlinear classifier is a support vector machine (SVM).

Among them, the system also includes a pre-training model and a logistic regression model (softmax).

Example 2

Referring to Figure 1,

(1) Sample prediction in ADNI dataset

Use the clinical medical data and apolipoprotein E data of the samples in the ADNI dataset to input the model A to predict the confidence a (Score1) that the sample CN or MCI is converted into AD;

Use the genomic data data of the samples in the ADNI dataset to input model B to predict the confidence level b (Score2) of converting the sample CN or MCI into AD;

Use the MRI data of the samples in the ADNI data set to input the model C to predict the confidence c (Score3) of the conversion of the sample CN or MCI to AD;

Taking confidence a, confidence b and confidence c as input features, and using model D to train the sample CN or MCI to convert the confidence (Score) of AD into AD;

(2) Individual CN or MCI prediction

The clinical medical data and apolipoprotein E data of individual CN or MCI are input into model A to predict the confidence a1 that individual CN or MCI is converted into AD;

The genomic data of individual CN or MCI is input into model B to predict the confidence level b1 of the conversion of sample CN or MCI to AD;

The MRI data of individual CN or MCI is input into model C to predict the confidence level c1 that the sample CN or MCI is converted into AD;

The confidence level a1, confidence level b1 and confidence level c1 are used as input features, and model D is used to train the individual CN or MCI to convert the confidence level g of AD into AD.

(3) Comparison

Comparing the confidence level g with the confidence level, if the confidence level g falls within the confidence level, the individual CN or MCI may be converted into AD;

If the confidence level g does not fall within the confidence level, it is impossible for the individual CN or MCI to convert to AD.

Among them, the MRI image data is standardized and histogram equalized to enhance the image contrast, and then the non-parametric non-uniform intensity normalization (N3) algorithm is used to correct the intensity inhomogeneity, the skull is stripped, and the pre-trained segmentation model is used to extract ROI.

Wherein, with reference to Fig. 2, wherein a convolutional neural network (CNN) and a recurrent neural network (RNN) are used, and the confidence level c of using MRI image data to predict that an individual is transformed into AD is specifically: (1) using a pre-training model to extract ROI; ( 2) Convolutional Neural Network (CNN) is used to extract spatial features; (3) Recurrent Neural Network (RNN) is used to extract temporal features; (4) Logistic regression model (softmax) is used for prediction.

Example 3

ADNI dataset has a total of 980 samples

There are 850 samples with complete clinical data, so Model A uses these 850 samples as the training set, and the acc of 5-fold cross-validation is 0.79;

The total number of samples with complete genome data is 880, so model B uses these 880 samples as the training set, and the acc of 5-fold cross-validation is 0.71;

The total number of samples with complete MRI image data is 980, so Model C uses these 980 samples as the training set, and the acc = 0.77 of the 5-fold cross-validation.

A total of 422 samples exist for clinical + genetic + image data. Model ABC is used to calculate 3 scores for these 422 samples as input features, logistic regression is used as classifier training, and the result of 5-fold cross-validation is acc=0.82. If we directly use the clinical, genetic and image features of these 422 samples to build a model, the 5-fold cross-validation result is acc=0.68.

The acc is only so low because the number of samples is too small. Compared with the previous scheme of merging a D model with the ABC model, the number of samples is almost reduced by half.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be Modifications or equivalent substitutions without departing from the spirit and scope of the technical solutions of the present invention should be included in the scope of the claims of the present invention.

Claims

A system for multimodal analysis of AD or MCI or CN disease development trend, characterized in that it includes:

Model A composed of clinical medical data and apolipoprotein E data input module and nonlinear classifier;

Model B composed of genomic data input module and nonlinear classifier;

Model C composed of MRI data and convolutional neural network and recurrent neural network;

ADNI dataset;

Model D composed of the predicted value receiving module of the model A, the model B and the model C and the logistic regression model;

The ADNI dataset provides the clinical medical data and the Apolipoprotein E data, the genomic data, the MRI data.
The system of claim 1, wherein the nonlinear classifier is a support vector machine.
The system of claim 1, wherein the model C further comprises a pre-trained segmentation model and a logistic regression model for processing the MRI data.
The system according to any one of claims 1-3, wherein the system is loaded on a terminal.
Utilize the method for the systematic analysis of individual CN or MCI disease development trend according to any one of claims 1-4, it is characterized in that, described method comprises the following steps:

S1: ADNI dataset sample training

Use the clinical medical data and apolipoprotein E data of the samples in the ADNI data set to input the model A to predict the confidence a of the conversion of the sample CN or MCI into AD;

Using the genomic data data of the samples in the ADNI dataset to input the model B to predict the confidence level b that the CN or MCI of the sample is converted into AD;

Utilize the MRI data of the sample in the ADNI data set to input the model C to predict the confidence c that the sample CN or MCI is converted into AD;

Using the confidence level a, the confidence level b and the confidence level c as input features, and using the model D for training to obtain the confidence level that the sample CN or MCI is converted into AD;

S2: Individual CN or MCI training

Inputting the clinical medical data and apolipoprotein E data of the individual CN or MCI into the model A to predict the confidence a1 that the individual CN or MCI is converted into AD;

The genomic data of the individual CN or MCI is input into the model B to predict the confidence level b1 that the sample CN or MCI is converted into AD;

The MRI data of the individual CN or MCI is input into the model C to predict the confidence level c1 that the sample CN or MCI is converted into AD;

The confidence level a1, the confidence level b1, and the confidence level c1 are used as input features, and the model D is used for training to obtain the confidence level g that the individual CN or MCI is converted into AD.
The method according to claim 4, characterized in that, by comparing the confidence level g with the confidence level, if the confidence level g falls within the confidence level, the individual CN or MCI may be converted into AD;

If the confidence level g does not fall within the confidence level, it is impossible for the individual CN or MCI to convert to AD.
The method according to claim 5 or 6, characterized in that, in S1 or S2, normalization processing and histogram equalization processing are performed on the MRI image data to enhance image contrast, and then non-parametric non-uniform intensity normalization is used. The (N3) algorithm corrects for intensity inhomogeneity, skull dissection, and extracts ROIs using a pretrained segmentation model.
method according to claim 5 or 6, is characterized in that, in S2, described adopting convolutional neural network and cyclic neural network, utilizes MRI image data to predict that individual CN or MCI is converted into the confidence degree c of AD specifically:

S1: Extract ROI by using the pre-trained segmentation model;

S2: using the convolutional neural network to extract spatial features;

S3: using the cyclic neural network to extract time series features;

S4: Use the logistic regression model to make predictions.
Application of the system according to any one of claims 1-4 in predicting the development trend of individual AD or MCI or CN disease.
The application according to claim 9, wherein the model A predicts the confidence A of the conversion of the MCI or CN into the AD according to clinical medical data and apolipoprotein E data; the model B is based on the genomic data Predicting the confidence level B that the MCI or CN is converted into the AD; the model C predicts the confidence level C that the MCI or CN is converted into the AD according to the MRI data; the confidence level A, the confidence level B And the confidence C is used as an input feature for training with the model D.