CN113706434A - Post-processing method for chest enhanced CT image based on deep learning - Google Patents
Post-processing method for chest enhanced CT image based on deep learning Download PDFInfo
- Publication number
- CN113706434A CN113706434A CN202010387797.0A CN202010387797A CN113706434A CN 113706434 A CN113706434 A CN 113706434A CN 202010387797 A CN202010387797 A CN 202010387797A CN 113706434 A CN113706434 A CN 113706434A
- Authority
- CN
- China
- Prior art keywords
- deep learning
- image
- value
- features
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000013135 deep learning Methods 0.000 title claims abstract description 49
- 238000012805 post-processing Methods 0.000 title claims abstract description 16
- 208000008732 thymoma Diseases 0.000 claims abstract description 40
- 238000012549 training Methods 0.000 claims abstract description 21
- 238000012216 screening Methods 0.000 claims abstract description 18
- 238000003384 imaging method Methods 0.000 claims abstract description 8
- 238000012502 risk assessment Methods 0.000 claims abstract description 6
- 238000003759 clinical diagnosis Methods 0.000 claims abstract description 4
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 17
- 238000003745 diagnosis Methods 0.000 claims description 15
- 238000012360 testing method Methods 0.000 claims description 14
- 238000002790 cross-validation Methods 0.000 claims description 13
- 230000006870 function Effects 0.000 claims description 12
- 230000011218 segmentation Effects 0.000 claims description 12
- 238000011156 evaluation Methods 0.000 claims description 9
- 210000000481 breast Anatomy 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000011176 pooling Methods 0.000 claims description 8
- 210000000038 chest Anatomy 0.000 claims description 6
- 210000002569 neuron Anatomy 0.000 claims description 6
- 238000012795 verification Methods 0.000 claims description 6
- 210000004072 lung Anatomy 0.000 claims description 5
- 210000001370 mediastinum Anatomy 0.000 claims description 5
- 238000010276 construction Methods 0.000 claims description 4
- 238000010835 comparative analysis Methods 0.000 claims description 3
- 230000000877 morphologic effect Effects 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 3
- 230000035945 sensitivity Effects 0.000 claims description 3
- 238000001207 Hosmer–Lemeshow test Methods 0.000 claims description 2
- 241000270295 Serpentes Species 0.000 claims description 2
- 230000001133 acceleration Effects 0.000 claims description 2
- 238000013136 deep learning model Methods 0.000 claims description 2
- 238000013210 evaluation model Methods 0.000 claims description 2
- 238000010832 independent-sample T-test Methods 0.000 claims description 2
- 238000007477 logistic regression Methods 0.000 claims description 2
- 238000013507 mapping Methods 0.000 claims description 2
- 238000010202 multivariate logistic regression analysis Methods 0.000 claims description 2
- 238000007473 univariate analysis Methods 0.000 claims description 2
- 230000001419 dependent effect Effects 0.000 claims 1
- 230000008901 benefit Effects 0.000 abstract description 10
- 230000008569 process Effects 0.000 abstract description 6
- 230000007547 defect Effects 0.000 abstract description 5
- 238000013480 data collection Methods 0.000 abstract description 4
- 238000002372 labelling Methods 0.000 abstract description 4
- 238000011160 research Methods 0.000 description 11
- 238000000605 extraction Methods 0.000 description 10
- 230000003902 lesion Effects 0.000 description 7
- 230000004044 response Effects 0.000 description 7
- 206010028980 Neoplasm Diseases 0.000 description 6
- 238000013527 convolutional neural network Methods 0.000 description 6
- 101150039208 KCNK3 gene Proteins 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 3
- 201000011510 cancer Diseases 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 230000036210 malignancy Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000004393 prognosis Methods 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 101000960235 Dictyostelium discoideum Isocitrate dehydrogenase [NADP] cytoplasmic Proteins 0.000 description 1
- 238000000729 Fisher's exact test Methods 0.000 description 1
- 208000032612 Glial tumor Diseases 0.000 description 1
- 206010018338 Glioma Diseases 0.000 description 1
- 206010054107 Nodule Diseases 0.000 description 1
- 206010056342 Pulmonary mass Diseases 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000013170 computed tomography imaging Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000013211 curve analysis Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000003748 differential diagnosis Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012854 evaluation process Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000003211 malignant effect Effects 0.000 description 1
- 201000000349 mediastinal cancer Diseases 0.000 description 1
- 230000035772 mutation Effects 0.000 description 1
- 208000018280 neoplasm of mediastinum Diseases 0.000 description 1
- 238000004451 qualitative analysis Methods 0.000 description 1
- 238000004445 quantitative analysis Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/30—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for calculating health indices; for individual health risk assessment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Public Health (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Primary Health Care (AREA)
- Epidemiology (AREA)
- Pathology (AREA)
- Computational Linguistics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biophysics (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The method for post-processing the chest enhanced CT image based on deep learning overcomes the defects of the traditional image omics method, balances the data collection and labeling cost, can improve the intelligent degree of the image omics process and the performance of a prediction model on the premise of limited sample size, and has great advantages and application value. The post-processing method for the chest enhanced CT image based on the deep learning comprises the following steps: (1) automatically dividing image data and inputting the image data into a ResNet network for deep learning; (2) acquiring comprehensive and complete focus information through a training model, and extracting deep learning vector features from the output of a first full-connection layer; (3) screening characteristics and constructing a prediction model; (4) and comparing and analyzing the result obtained based on deep learning with the result of the traditional imaging group, and evaluating the value of the result on clinical diagnosis and risk assessment of the thymoma.
Description
Technical Field
The invention relates to the technical field of medical image processing, in particular to a post-processing method for a chest enhanced CT image based on deep learning.
Background
The purpose of the imaging examination before the thymoma operation is to primarily evaluate the malignancy degree of the tumor, further presume the histopathological change and carry out risk evaluation, thereby assisting the selection of a preoperative treatment scheme and the judgment of clinical prognosis. However, because of the relatively low incidence of thymoma, there are still major challenges in preoperative accurate diagnosis and risk assessment of malignancy. CT has high sensitivity to diagnosis of the anterior mediastinal tumor, and the enhancement of CT can better display the outline, the boundary and the local infiltrates of the thymoma and evaluate the malignancy degree of the thymoma. Is an important examination means for diagnosis and identification.
However, the evaluation of CT imaging signs is based on the thymoma focus itself and the relationship between the thymoma focus itself and the adjacent surrounding tissue structure, and often adopts empirical, observability indexes rather than quantitative indexes, and the images of other tumors or tumor-like lesions in the anterior mediastinum are similar in appearance and difficult to identify in clinical work, so that how to perform more accurate diagnosis on thymoma based on CT image data by using an objective quantitative method needs further research.
The traditional imaging omics adopts high-throughput feature extraction on medical image images, performs qualitative analysis by focusing on a clinical problem construction model, and can be better applied to the aspects of accurate diagnosis of diseases, histopathological risk assessment, clinical prognosis assessment and the like. However, in clinical practice, the traditional imaging omics have inherent problems and difficulties, firstly, the focus segmentation step of the medical image depends on manual delineation, and the process is time-consuming and is easily influenced by human factors; secondly, the current image omics feature extraction method is difficult to perform algorithm presetting and complete extraction on all image features in the segmented region. Therefore, the automation and standardization degree of the traditional imaging omics is low, and the accuracy and robustness of the prediction result are still limited, so that the method has a larger promotion space.
Deep learning is a specific research direction in the field of machine learning, and in recent years, deep learning technology is rapidly developed and is widely applied to various fields of image analysis. The CNN and the derivative network structure thereof are representative deep learning methods and have been successfully applied to the fields of image recognition and feature extraction. In recent years, segmentation of medical images using CNN has been studied and achieved better than conventional methods. According to the characteristics and the structure of the original data, the parameters of the CNN are adjusted to the optimal structure, so that the optimal output can be obtained and the purpose of ideal classification can be achieved. A large amount of global space information is stored by performing convolution kernel operation on the whole image, so that the complete target image region characteristics can be extracted from the last layer of convolution layer. Therefore, the CNN is adopted to stabilize the lesion segmentation result of the medical image, and the extracted image features are comprehensive and complete, so that the accuracy and robustness of the prediction result can be improved finally, and an important methodology approach is provided for solving the dilemma and problems of the conventional image omics.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention aims to provide a method for post-processing a chest enhanced CT image based on deep learning, which not only overcomes the defects of an image omics method, but also balances the data collection and labeling cost, can improve the intelligent degree of the image omics process and the performance of a prediction model on the premise of limited sample size, and has great advantages and clinical application value.
The technical scheme of the invention is as follows: the deep learning method based on the segmented chest enhanced CT image comprises the following steps:
(1) inputting the segmented lesion data into a ResNet network for deep learning;
(2) acquiring comprehensive and complete focus information through a training model, and extracting deep learning vector features from the output of a first full-connection layer;
(3) screening characteristics and constructing a prediction model;
(4) and comparing and analyzing the result obtained by the deep learning-based image group with the result of the traditional image group, and evaluating the value of the deep learning-based image group to clinical diagnosis and risk assessment of the thymoma.
The segmented images are input into a ResNet network, comprehensive and complete lesion information is obtained through a training model, function response characteristics in a first full connecting layer in the ResNet network are extracted, the segmented images are mapped into deep learning characteristics after convolution operation, characteristics are finally screened, a prediction model is constructed to diagnose thymoma and distinguish histopathological risk types of the thymoma, a Delong test and DCA method is adopted in a training set to carry out comparison and analysis with a traditional image model, and the value of the thymoma to diagnosis and risk omic evaluation is evaluated, so that the defects of an image omics method are overcome, the data collection and labeling cost is balanced, the intelligentization degree of an image omics process and the performance of the prediction model can be improved on the premise of limited sample size, and the thymoma diagnosis and risk evaluation method has great advantages and application values.
Drawings
Fig. 1 shows the residual block structure of ResNet-34. ResNet-34 adopts two residual block structures Resblock1 and Resblock2, Resblock1 does not need to change the number of channels of the network, and Resblock2 needs to increase the number of channels of the network.
Fig. 2 shows a schematic diagram of a deep learning feature extraction method. The function response characteristics in the first fully-connected layer in the ResNet network are mapped and coded into vector characteristics of 4096 x 1 after convolution operation.
Figure 3 shows the use of Kendall correlation coefficients to screen for features of high discrimination ability. When the threshold is set to 0.8, the number of features retained in Task1 is 542, and the number of features retained in Task2 is 468.
Figure 4 shows screening for stable features using 10 fold cross validation. (A) After cross validation, the number of reserved features in Task1 is 53, and the reserved features sequentially correspond to the response sequence of the full connection layer in the ResNet network; (B) the number of features reserved after cross validation in Task2 was 41 which in turn corresponded to the order of response of the fully connected layers in the ResNet network.
Fig. 5 shows LASSO regression final screening features. (A) LASSO regression in Task1 finally screens to obtain 3 characteristics; (B) LASSO regression in Task2 final screening yielded 3 features.
FIG. 6 shows the performance of the DLBR prediction model in the training set and the validation set. (A) The classification diagnosis of thymoma and non-thymoma in the training set is carried out, and the AUC is 0.8344; (B) verifying the classification diagnosis of the concentrated thymoma and the non-thymoma, wherein the AUC is 0.7415; (C) the histopathological risk category AUC of the thymoma differentiated in the training set is 0.8255; (D) the AUC of the histopathological risk category of the thymoma in the validation set was 0.7752.
Fig. 7 shows DLBR versus CR model decision curve analysis. (A) For the differential diagnosis of Task1 thymoma versus non-thymoma, if the threshold probability of a clinical decision is greater than 46%, then there is more clinical benefit to using the DLBR model than the CR model for prediction. Within this range, the net benefits of the two models are comparable, and the DLBR model performs better; (B) for Task2 to differentiate thymoma risk categories, if the threshold probability of a clinical decision is greater than 60%, there is more clinical benefit to using the DLBR model than the CR model for prediction. Within this range, the net benefits of the two models are comparable, and the DLBR model performs better.
FIG. 8 illustrates constructing a nomogram based on the DLBR prediction model. (A) DLBR nomogram was used to identify thymomas versus non-thymomas (Task 1); (B) the DLBR nomogram is used to distinguish the thymoma risk category (Task 2).
Fig. 9 shows a flow chart of a method for post-processing of a breast enhanced CT image based on deep learning according to the present invention.
Detailed Description
As shown in fig. 9, the method for post-processing a chest enhanced CT image based on deep learning includes the following steps:
(1) inputting the segmented lesion data into a ResNet network for deep learning;
(2) acquiring comprehensive and complete focus information through a training model, and extracting deep learning vector features from the output of a first full-connection layer;
(3) screening characteristics and constructing a prediction model;
(4) and comparing and analyzing the result obtained by the deep learning-based image group with the result of the traditional image group, and evaluating the value of the deep learning-based image group to clinical diagnosis and risk assessment of the thymoma.
The segmented images are input into a ResNet network, comprehensive and complete lesion information is obtained through a training model, function response characteristics in a first full connecting layer in the ResNet network are extracted, the segmented images are mapped into deep learning characteristics after convolution operation, characteristics are finally screened, a prediction model is constructed to diagnose thymoma and distinguish histopathological risk types of the thymoma, a Delong test and DCA method is adopted in a training set to carry out comparison and analysis with a traditional image model, and the value of the thymoma to diagnosis and risk omic evaluation is evaluated, so that the defects of an image omics method are overcome, the data collection and labeling cost is balanced, the intelligentization degree of an image omics process and the performance of the prediction model can be improved on the premise of limited sample size, and the thymoma diagnosis and risk evaluation method has great advantages and application values.
Preferably, the step (1) comprises the following substeps:
(1.1) CT image acquisition: all patients are subjected to CT contrast enhanced scanning to obtain a thin layer image reconstructed at a chest mediastinum window;
(1.2) designing a double-lung mask file on an original CT image by utilizing the natural density difference between the lung and surrounding tissues and adopting a method for calculating a lung tissue pixel value, removing the region outside the mediastinum and reserving a focus;
(1.3) carrying out primary segmentation on the focus by adopting a V _ Net network;
(1.4) using Morphologic Snakes algorithm to finely divide the focus.
Preferably, in the step (2), the training is performed by using ResNet-34 containing two residual blocks, each of which is expressed as formula (1)
y＝F(x)+x (1)
Wherein x and y represent the input and output of the residual block, respectively;
the function F (x) represents the residual mapping, expressed as equation (2)
F(x)＝W2σ(W1x) (2)
Wherein, σ represents ReLU, W1Represents the weight, W, learned from top to bottom through the first convolutional layer in the residual block2Representing the weights learned by the second convolutional layer in the residual block.
Preferably, in the step (2), the ResNet-34 network consists of 1 convolutional layer, 2 pooling layers, 16 residual blocks and 2 fully-connected layers, wherein the 16 residual blocks comprise 13 Resblock1 and 3 Resblock 2; the input image data firstly passes through 1 convolution layer with the kernel size of 7 multiplied by 7 and the step length of 2, and then passes through a maximum pooling layer with the window size of 3 multiplied by 3 and the step length of 2 after being activated by a ReLU function; then continuously passing through 16 residual blocks, passing through an average pooling layer with a window size of 3 x 3 and a step length of 2, and finally inputting into a full-connection layer to extract features; the first fully-connected layer in this ResNet network is designed to be 4096 neurons, each neuron outputs a vector, and the vector encoded as 4096 × 1 by the fully-connected layer is the extracted feature.
Preferably, in the step (3), the characteristic screening is carried out in three steps:
(3.1) based on Kendall correlation coefficient of the calculated features, setting the threshold value of the Kendall correlation coefficient to be 0.15 screening features, firstly, calculating whether each feature of each patient in the classified sample 1 and the sample 2 is a coordinated feature value or not, and adopting a symbolic function as follows:
sgn(Xij-Xik)＝sgn(Yi-Yk) (3)
sgn(Xij-Xik)＝-sgn(Yi-Yk) (4)
in the formula XijRepresents the ith feature of the jth sample in the class 1 samples, j is the sample ordinal, i is the feature ordinal, YjA class label representing the sample; xikRepresents the ith feature of the kth sample in sample 2, k is sample number, i is feature number, YkA class label representing the sample; if equation (3) is true, the characteristic value is considered to be the harmonized characteristic value, and if equation (4) is true, the characteristic value is considered to be the harmonized characteristic valueRespectively counting the number of the coordinated and uncoordinated features for the uncoordinated feature values; the Kendall correlation coefficient tau of the characteristic i is calculated by adopting the formula (5):
τi＝(Nc-Nd)/m×n (5)
in the formula, m is the number of samples in the category 1, n is the number of samples in the category 2, Nc is the number of the coordination features, and Nd is the number of the non-coordination features;
(3.2) screening stable features by a 10-fold cross validation method for the preliminarily screened features, searching a feature set with the highest classification correctness as an optimal feature set in each cross validation, iterating the sequence of each feature in the feature set in each cross validation to be slightly different, respectively finding out the features which are all appeared in each cross validation iteration, and respectively counting and screening the features again to form alternative stable features;
and (3.3) adopting LASSO regression, and carrying out weighted calculation on the selected characteristics according to respective coefficients in a regression equation to construct the label of the DLBR.
Preferably, in the step (3), in the training set, a prediction model is constructed for the DLBR labels obtained by the above method and combined with clinical information of patients and CT image signs as final features, univariate analysis is used to evaluate the relationship between each feature and the classification variables, multivariate Logistic regression analysis is finally carried out to construct the prediction model to perform classification diagnosis on thymoma and non-thymoma and to distinguish the histopathological risk categories of the thymoma, and the verification evaluation of the prediction model is carried out on the prediction model in the verification set.
Preferably, in the step (4), a Delong test is adopted to perform comparative analysis on prediction models of CR and DLBR methods, so as to evaluate the application value of DLBR; the Delong test firstly calculates the difference value of AUC of two models, then calculates the variance and covariance of AUC respectively, then calculates z value, and makes the z value distribution as the significance test of normal distribution, and takes p < 0.05 as the significance difference between AUC; the z value is obtained by the formula (6)
Wherein M is1AUC values, M, for a model constructed for manual segmentation of lesions2AUC value, var (M), for a model constructed for automated segmentation of lesions1) Is M1Variance of (c), var (M)2) Is M2Variance of cov (M)1,M2) Is the covariance of the two.
Preferably, the method further comprises the steps of (5) inter-group comparison of patient general clinical data, CT image signs, independent sample T test for continuous variables, χ 2 test or Fisher's exact test for categorical variables; constructing a deep learning model frame by adopting PyToch and carrying out image acceleration processing in a GPU; screening the characteristics of the image group by adopting Matlab software; performing LASSO regression construction equation calculation on the basis of the R software environment by using a generalized linear model kit; using SPSS software to sort general clinical data and CT image signs of all patients, using STATA software to analyze the relation between the general data, the CT image signs and patient classification variables, performing Logistic regression analysis to construct a prediction model and an ROC curve, and evaluating a prediction result by adopting sensitivity, specificity and accuracy; respectively carrying out goodness-of-fit test on the prediction results in the training set and the verification set by using a Hosmer-Lemeshow test, and evaluating the coincidence degree of the model fitting value and the observation value; and analyzing the value of the evaluation model influencing clinical decision by adopting a decision curve and respectively constructing a nomogram based on the prediction model.
On the basis that two centers automatically divide focuses based on deep learning, divided images are input into a ResNet network, comprehensive and complete focus information is obtained through a training model, function response characteristics in a first full connection layer in the ResNet network are extracted, the function response characteristics are mapped into deep learning characteristics after convolution operation, characteristics are finally screened, a prediction model is constructed to diagnose thymoma and distinguish histopathological risk types of the thymoma, a Delong test and DCA method is adopted in a training set to carry out comparative analysis with a traditional image omics model, and the values of the thymoma diagnosis and risk evaluation are evaluated.
The automatic segmentation of the focus is an important step in the process of the image omics based on deep learning, and the traditional image omics research generally adopts a method for manually delineating the focus for segmentation, so that the focus is easily influenced by personal experience of a delineation person and a uniform and fine delineation standard, and therefore, the subsequent extraction of the characteristics of the image omics and the label calculation are influenced. According to the research, the focus is automatically segmented based on deep learning, the high-quality segmented image is obtained, and the standardization of deep learning feature extraction and the accuracy of omics label calculation are guaranteed.
In the research, the segmented image is input into a ResNet network to complete model training, and the commonly used ResNet has five structures: ResNet-18, ResNet-34, ResNet-50, ResNet-101, ResNet-152 (the numbers after ResNet indicate the number of network parameter layers). According to the data size and training time in the research, a ResNet-34 network structure is used, if two residual block structures Resblock1 and Resblock2 are included, Resblock1 does not need to change the number of channels of the network, and Resblock2 needs to increase the number of channels of the network. In order to ensure that the depth features can map the image data features in depth, the input image data passes through a convolution layer and a pooling layer, then passes through 16 residual blocks and an average pooling layer continuously, and finally is input into a full-link layer to extract features. In order to ensure that the extracted depth features can fully map the original image data features, a first full connection layer in a ResNet network is designed into 4096 neurons, and each neuron outputs a vector code to be finally extracted features.
Features that are typically extracted by traditional imaging based on the theory include: FOS, GLCM, GLRLM, GLSZM, NGTDM, GLDM, 3D morphological characteristics, etc. These features are mostly first-order statistic features based on the lesion gray level histogram, or second-order background texture features such as direction and distribution characteristics based on the spatial combination of pixel or voxel discrete gray levels, or morphology based on the original image extraction only. Compared with the traditional image omics, the deep learning is powerful in that the multilayer structure of the neural network can automatically learn the characteristics with rich layers, for example, a shallow convolutional layer can learn the characteristics of a local region of a focus, and a deep convolutional layer can learn more abstract characteristics, so that the performance of classification and identification is improved. The existing image omics research does not extract the characteristics of the traditional image omics any more, the output of a deep network in a CNN network structure is directly adopted as the characteristics and then input into a prediction model (classifier) for classification and prediction, and the result shows that the deep learning method can better predict the mutation state of IDH-1 in low-level glioma compared with the traditional image omics method. Therefore, the features extracted by the traditional image omics can be obtained by adopting a deep learning method to perform coverage calculation, the convolution layer in the deep neural network structure can be used as a feature extractor to obtain more comprehensive and complete features, and the traditional image omics feature extraction step can be better replaced in the feature extraction stage.
4096 deep learning features are extracted in the research, a Kendall correlation coefficient based on the calculated features is firstly preliminarily screened, different sign functions are respectively counted according to whether each feature of each patient in a classified sample 1 and a sample 2 is a coordination feature value, finally a Kendall correlation coefficient set is calculated, a threshold value of the Kendall correlation coefficient set is set to be 0.15 screening features, and then the features appearing in each cross-validation iteration are searched through cross-validation and are respectively counted and screened again to be optional stable features. And finally, adopting LASSO regression, and calculating the label value of the DLBR according to the respective coefficient weight in a regression equation. Compared with the conventional research, the research adopts various feature calculation methods, the types and the number of the extracted image omics features are relatively large, and in order to ensure the robustness of a prediction model and the generalization capability of the model, the features with high stability are screened from the deep learning features by adopting various methods.
Comparing the prediction models obtained by CR and DLBR, wherein the difference of the two models in Task1 has statistical significance, and the difference of the two models in Task2 has no statistical significance; the DCA is adopted to compare the two models respectively, and the result shows that the prediction model of the DLBR has larger clinical benefit than the CR model. In general, the DLBR model has great advantages. In the existing research, a deep learning method is adopted to identify the lung nodule properties, three different CNNs are trained in advance to extract deep features of a neural network, the pre-trained deep learning and traditional image omics features are screened and then input into a classification model, and finally good and malignant nodules of the lung are well identified. Therefore, compared with the traditional imaging omics, the DLBR can improve the intelligent degree of a diagnosis and evaluation process and the accuracy of a prediction result.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention in any way, and all simple modifications, equivalent variations and modifications made to the above embodiment according to the technical spirit of the present invention still belong to the protection scope of the technical solution of the present invention.
Claims (8)
1. The post-processing method for the chest enhanced CT image based on the deep learning is characterized by comprising the following steps: which comprises the following steps:
(1) inputting the image data into a ResNet network for deep learning after automatic segmentation;
(2) acquiring comprehensive and complete focus information through a training model, and extracting deep learning vector features from the output of a first full-connection layer;
(3) screening characteristics and constructing a prediction model;
(4) and comparing and analyzing the result obtained based on deep learning with the result of the traditional imaging group, and evaluating the value of the result on clinical diagnosis and risk assessment of the thymoma.
2. The method for post-processing of breast enhanced CT images based on deep learning of claim 1, wherein: the step (1) comprises the following sub-steps:
(1.1) CT image acquisition: all patients are subjected to CT contrast enhanced scanning to obtain a thin layer image reconstructed at a chest mediastinum window;
(1.2) designing a double-lung mask file on an original CT image by utilizing the natural density difference between the lung and surrounding tissues and adopting a method for calculating a lung tissue pixel value, removing the region outside the mediastinum and reserving a focus;
(1.3) carrying out primary segmentation on the focus by adopting a V _ Net network;
(1.4) using Morphologic Snakes algorithm to finely divide the focus.
3. The method for post-processing of breast enhanced CT images based on deep learning of claim 2, wherein: in the step (2), the ResNet-34 containing two residual blocks is used for training, and each residual block is expressed as formula (1)
y＝F(x)+x (1)
Wherein x and y represent the input and output of the residual block, respectively;
the function F (x) represents the residual mapping, expressed as equation (2)
F(x)＝W2σ(W1x) (2)
Wherein, σ represents ReLU, W1Represents the weight, W, learned from top to bottom through the first convolutional layer in the residual block2Representing the weights learned by the second convolutional layer in the residual block.
4. The method for post-processing of breast enhanced CT images based on deep learning of claim 3, wherein: in the step (2), the ResNet-34 network consists of 1 convolutional layer, 2 pooling layers, 16 residual blocks and 2 full-connection layers, wherein the 16 residual blocks comprise 13 Resblock1 and 3 Resblock 2; the input image data firstly passes through 1 convolution layer with the kernel size of 7 multiplied by 7 and the step length of 2, and then passes through a maximum pooling layer with the window size of 3 multiplied by 3 and the step length of 2 after being activated by a ReLU function; then continuously passing through 16 residual blocks, passing through an average pooling layer with a window size of 3 x 3 and a step length of 2, and finally inputting into a full-connection layer to extract features; the first fully-connected layer in this ResNet network is designed to be 4096 neurons, each neuron outputs a vector, and the vector encoded as 4096 × 1 by the fully-connected layer is the extracted feature.
5. The method for post-processing of breast enhanced CT images based on deep learning of claim 4, wherein: in the step (3), the screening characteristics are divided into three steps:
(3.1) based on Kendall correlation coefficient of the calculated features, setting the threshold value of the Kendall correlation coefficient to be 0.15 screening features, firstly, calculating whether each feature of each patient in the classified sample 1 and the sample 2 is a coordinated feature value or not, and adopting a symbolic function as follows:
sgn(Xij-Xik)＝sgn(Yi-Yk) (3)
sgn(Xij-Xik)＝-sgn(Yi-Yk) (4)
in the formula XijRepresents the ith feature of the jth sample in the class 1 samples, j is the sample ordinal, i is the feature ordinal, YjA class label representing the sample; xikRepresents the ith feature of the kth sample in sample 2, k is sample number, i is feature number, YkA class label representing the sample; if the formula (3) is established, the characteristic value is a coordination characteristic value, otherwise, if the formula (4) is established, the characteristic value is a non-coordination characteristic value, and the number of coordination and non-coordination characteristics is respectively counted; the Kendall correlation coefficient tau of the characteristic i is calculated by adopting the formula (5):
τi＝(Nc-Nd)/m×n (5)
in the formula, m is the number of samples in the category 1, n is the number of samples in the category 2, Nc is the number of the coordination features, and Nd is the number of the non-coordination features;
(3.2) screening stable features by a 10-fold cross validation method for the preliminarily screened features, searching a feature set with the highest classification correctness as an optimal feature set in each cross validation, iterating the sequence of each feature in the feature set in each cross validation to be slightly different, respectively finding out the features which are all appeared in each cross validation iteration, and respectively counting and screening the features again to form alternative stable features;
and (3.3) adopting LASSO regression, and carrying out weighted calculation on the selected characteristics according to respective coefficients in a regression equation to construct the label of the DLBR.
6. The method for post-processing of breast enhanced CT images based on deep learning of claim 5, wherein: in the step (3), in a training set, a DLBR label obtained by the method is combined with patient clinical information and CT image signs to serve as a final feature construction prediction model, univariate analysis is used for evaluating the relation between each feature and a classification dependent variable, multivariate Logistic regression analysis is finally carried out to construct the prediction model, classification diagnosis is carried out on thymoma and non-thymoma, histopathological risk categories of the thymoma are distinguished, and verification evaluation of the prediction model is carried out in a verification set.
7. The method for post-processing of breast enhanced CT images based on deep learning of claim 6, wherein: in the step (4), a Delong test is adopted to carry out comparative analysis on prediction models of a CR method and a DLBR method, and the application value of the DLBR is evaluated; the Delong test firstly calculates the difference value of AUC of two models, then calculates the variance and covariance of AUC respectively, then calculates z value, and makes the z value distribution as the significance test of normal distribution, and takes p < 0.05 as the significance difference between AUC; the z value is obtained by the formula (6)
Wherein M is1AUC values, M, for a model constructed for manual segmentation of lesions2AUC value, var (M), for a model constructed for automated segmentation of lesions1) Is M1Variance of (c), var (M)2) Is M2Variance of cov (M)1,M2) Is the covariance of the two.
8. The method for post-processing of breast enhanced CT images based on deep learning of claim 7, wherein: the method further comprises the steps of (5) inter-group comparison of patient general clinical data and CT image signs, independent sample T test for continuous variables, and χ 2 test or Fisher's accurate test for categorical variables; constructing a deep learning model frame by adopting PyToch and carrying out image acceleration processing in a GPU; screening the characteristics of the image group by adopting Matlab software; performing LASSO regression construction equation calculation on the basis of the R software environment by using a generalized linear model kit; using SPSS software to sort general clinical data and CT image signs of all patients, using STATA software to analyze the relation between the general data, the CT image signs and patient classification variables, performing Logistic regression analysis to construct a prediction model and an ROC curve, and evaluating a prediction result by adopting sensitivity, specificity and accuracy; respectively carrying out goodness-of-fit test on the prediction results in the training set and the verification set by using a Hosmer-Lemeshow test, and evaluating the coincidence degree of the model fitting value and the observation value; and analyzing the value of the evaluation model influencing clinical decision by adopting a decision curve and respectively constructing a nomogram based on the prediction model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010387797.0A CN113706434B (en) | 2020-05-09 | 2020-05-09 | Post-processing method for chest enhancement CT image based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010387797.0A CN113706434B (en) | 2020-05-09 | 2020-05-09 | Post-processing method for chest enhancement CT image based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113706434A true CN113706434A (en) | 2021-11-26 |
| CN113706434B CN113706434B (en) | 2023-11-07 |
Family
ID=78645323
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010387797.0A Active CN113706434B (en) | 2020-05-09 | 2020-05-09 | Post-processing method for chest enhancement CT image based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113706434B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116030987A (en) * | 2023-02-20 | 2023-04-28 | 北京大学第三医院（北京大学第三临床医学院） | Method for constructing evaluation model of systemic sclerosis esophagus power |
| CN116250837A (en) * | 2023-02-14 | 2023-06-13 | 天津大学 | Depression detection device based on dynamic factor brain network |
| CN116958128A (en) * | 2023-09-18 | 2023-10-27 | 中南大学 | Medical image automatic positioning method based on deep learning |
| CN117173167A (en) * | 2023-11-02 | 2023-12-05 | 中日友好医院(中日友好临床医学研究所) | Method and device for predicting tumor prognosis by image histology machine learning survival model |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106372390A (en) * | 2016-08-25 | 2017-02-01 | 姹ゅ钩 | Deep convolutional neural network-based lung cancer preventing self-service health cloud service system |
| CN108257135A (en) * | 2018-02-01 | 2018-07-06 | 浙江德尚韵兴图像科技有限公司 | The assistant diagnosis system of medical image features is understood based on deep learning method |
| CN110458249A (en) * | 2019-10-10 | 2019-11-15 | 点内（上海）生物科技有限公司 | A kind of lesion categorizing system based on deep learning Yu probability image group |
-
2020
- 2020-05-09 CN CN202010387797.0A patent/CN113706434B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106372390A (en) * | 2016-08-25 | 2017-02-01 | 姹ゅ钩 | Deep convolutional neural network-based lung cancer preventing self-service health cloud service system |
| CN108257135A (en) * | 2018-02-01 | 2018-07-06 | 浙江德尚韵兴图像科技有限公司 | The assistant diagnosis system of medical image features is understood based on deep learning method |
| CN110458249A (en) * | 2019-10-10 | 2019-11-15 | 点内（上海）生物科技有限公司 | A kind of lesion categorizing system based on deep learning Yu probability image group |
Non-Patent Citations (2)
| Title |
|---|
| 宫进昌;赵尚义;王远军;: "基于深度学习的医学图像分割研究进展", 中国医学物理学杂志, no. 04 * |
| 陈书晶;帅仁俊;: "一种快速提取连续CT脑肿瘤病灶区的算法", 计算机测量与控制, no. 02 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116250837A (en) * | 2023-02-14 | 2023-06-13 | 天津大学 | Depression detection device based on dynamic factor brain network |
| CN116250837B (en) * | 2023-02-14 | 2024-02-13 | 天津大学 | Depression detection device based on dynamic factor brain network |
| CN116030987A (en) * | 2023-02-20 | 2023-04-28 | 北京大学第三医院（北京大学第三临床医学院） | Method for constructing evaluation model of systemic sclerosis esophagus power |
| CN116030987B (en) * | 2023-02-20 | 2023-08-04 | 北京大学第三医院（北京大学第三临床医学院） | Method for constructing evaluation model of systemic sclerosis esophagus power |
| CN116958128A (en) * | 2023-09-18 | 2023-10-27 | 中南大学 | Medical image automatic positioning method based on deep learning |
| CN116958128B (en) * | 2023-09-18 | 2023-12-26 | 中南大学 | Medical image automatic positioning method based on deep learning |
| CN117173167A (en) * | 2023-11-02 | 2023-12-05 | 中日友好医院(中日友好临床医学研究所) | Method and device for predicting tumor prognosis by image histology machine learning survival model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113706434B (en) | 2023-11-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113706434B (en) | Post-processing method for chest enhancement CT image based on deep learning | |
| CN106295124B (en) | The method of a variety of image detecting technique comprehensive analysis gene subgraph likelihood probability amounts | |
| CN109544518B (en) | Method and system applied to bone maturity assessment | |
| CN107977671A (en) | A kind of tongue picture sorting technique based on multitask convolutional neural networks | |
| CN111985536A (en) | Gastroscope pathological image classification method based on weak supervised learning | |
| Ben-Ari et al. | Domain specific convolutional neural nets for detection of architectural distortion in mammograms | |
| CN111767952B (en) | Interpretable lung nodule benign and malignant classification method | |
| CN112784879A (en) | Medical image segmentation or classification method based on small sample domain self-adaption | |
| Taher et al. | Bayesian classification and artificial neural network methods for lung cancer early diagnosis | |
| CN111340770B (en) | Method for constructing cancer prognosis model by combining global weighted LBP (local binary pattern) and texture analysis | |
| Naresh et al. | Early detection of lung cancer using neural network techniques | |
| Lee et al. | An adaptive resource-allocating network for automated detection, segmentation, and classification of breast cancer nuclei topic area: image processing and recognition | |
| CN112365464A (en) | GAN-based medical image lesion area weak supervision positioning method | |
| CN116884623B (en) | Medical rehabilitation prediction system based on laser scanning imaging | |
| CN110021019A (en) | A kind of thickness distributional analysis method of the AI auxiliary hair of AGA clinical image | |
| Liu et al. | Pathological lung segmentation based on random forest combined with deep model and multi-scale superpixels | |
| Anand et al. | An intelligent approach to segment the liver cancer using Machine Learning Method | |
| CN112036298A (en) | Cell detection method based on double-segment block convolutional neural network | |
| CN114782948A (en) | Global interpretation method and system for cervical liquid-based cytology smear | |
| Tian et al. | Radiomics and its clinical application: artificial intelligence and medical big data | |
| CN117315380A (en) | Deep learning-based pneumonia CT image classification method and system | |
| CN116230237B (en) | Lung cancer influence evaluation method and system based on ROI focus features | |
| CN117036288A (en) | Tumor subtype diagnosis method for full-slice pathological image | |
| Thapa et al. | Deep learning for breast cancer classification: Enhanced tangent function | |
| CN116664932A (en) | Colorectal cancer pathological tissue image classification method based on active learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |