CN110929789A - Liver tumor automatic classification method and device based on multi-stage CT image analysis - Google Patents
Liver tumor automatic classification method and device based on multi-stage CT image analysis Download PDFInfo
- Publication number
- CN110929789A CN110929789A CN201911159730.5A CN201911159730A CN110929789A CN 110929789 A CN110929789 A CN 110929789A CN 201911159730 A CN201911159730 A CN 201911159730A CN 110929789 A CN110929789 A CN 110929789A
- Authority
- CN
- China
- Prior art keywords
- model
- stage
- training
- liver
- inputting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 208000014018 liver neoplasm Diseases 0.000 title claims abstract description 60
- 238000000034 method Methods 0.000 title claims abstract description 50
- 206010019695 Hepatic neoplasm Diseases 0.000 title claims abstract description 17
- 238000010191 image analysis Methods 0.000 title claims abstract description 17
- 208000006990 cholangiocarcinoma Diseases 0.000 claims abstract description 48
- 238000012549 training Methods 0.000 claims abstract description 46
- 206010073071 hepatocellular carcinoma Diseases 0.000 claims abstract description 43
- 231100000844 hepatocellular carcinoma Toxicity 0.000 claims abstract description 43
- 201000007270 liver cancer Diseases 0.000 claims abstract description 42
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 21
- 238000013145 classification model Methods 0.000 claims abstract description 17
- 210000005228 liver tissue Anatomy 0.000 claims abstract description 16
- 238000013528 artificial neural network Methods 0.000 claims abstract description 15
- 230000011218 segmentation Effects 0.000 claims abstract description 15
- 230000003187 abdominal effect Effects 0.000 claims abstract description 14
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 claims abstract description 14
- 206010028980 Neoplasm Diseases 0.000 claims abstract description 11
- 238000002591 computed tomography Methods 0.000 claims abstract description 10
- 239000000284 extract Substances 0.000 claims description 28
- 238000010586 diagram Methods 0.000 claims description 20
- 210000004185 liver Anatomy 0.000 claims description 7
- 201000011510 cancer Diseases 0.000 claims description 6
- 238000010276 construction Methods 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 6
- 239000013598 vector Substances 0.000 claims description 6
- 238000003745 diagnosis Methods 0.000 abstract description 7
- 210000003240 portal vein Anatomy 0.000 abstract description 3
- 238000000605 extraction Methods 0.000 description 16
- 238000013461 design Methods 0.000 description 7
- 238000003384 imaging method Methods 0.000 description 6
- 238000010801 machine learning Methods 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 210000001367 artery Anatomy 0.000 description 2
- 238000003759 clinical diagnosis Methods 0.000 description 2
- 238000013170 computed tomography imaging Methods 0.000 description 2
- 238000003066 decision tree Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 210000005036 nerve Anatomy 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000013188 needle biopsy Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013610 patient sample Substances 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Apparatus For Radiation Diagnosis (AREA)
Abstract
The liver tumor automatic classification method and device based on multi-stage CT image analysis can identify full-automatic cholangiocellular carcinoma and hepatocellular carcinoma, and obtain high-precision cholangiocellular carcinoma and hepatocellular carcinoma identification models. The method comprises the following steps: (1) acquiring contrast-enhanced abdominal CT scanning images, storing the images as an arterial phase, a portal vein phase and a delay phase, and performing definite diagnosis on liver cancer categories to which all data belong to serve as a model training gold standard; (2) constructing a three-dimensional full-convolution neural network segmentation model, training and learning the internal characteristics of the liver tissue in each stage through the model, and segmenting the internal characteristics from the abdominal CT image; (3) and constructing a three-dimensional convolutional neural network classification model, inputting the image data obtained by segmentation into the classification model for training, and enabling the model to carry out combined learning and training on the characteristics of the cancers in multiple stages, so as to predict the categories of the cancers, comparing the prediction result with a gold standard, and supervising the training process of the model in a way of feeding back a loss value.
Description
Technical Field
The invention relates to the technical field of medical image processing, in particular to a liver tumor automatic classification method based on multi-stage CT image analysis and a liver tumor automatic classification device based on multi-stage CT image analysis, which are mainly applied to the field of liver cancer identification research based on CT images.
Background
In recent years, the incidence rate of cholangiocellular carcinoma is on the rise, the cholangiocellular carcinoma is located in the liver, no obvious clinical symptoms exist in the early stage, and the cholangiocellular carcinoma is easy to be misdiagnosed as hepatocellular carcinoma, but the cholangiocellular carcinoma treatment method is quite different from a hepatocellular carcinoma treatment method, and most patients lose the best operation time when the cholangiocellular carcinoma is found, so that the cholangiocellular carcinoma early-stage diagnosis has important clinical significance. The traditional cholangiocellular carcinoma diagnosis is mainly judged by observing a CT image of a patient by a doctor, and because cholangiocellular carcinoma is similar to appearance of hepatocellular carcinoma, diagnosis by reading a film has certain difficulty, the patient can be finally diagnosed only by performing a needle biopsy operation, and extra pain and risk are brought to the patient. In addition, because of the large number of slices of the contrast enhanced CT scan, the diagnosis is time-consuming and labor-consuming, and is prone to misdiagnosis. The liver cancer auxiliary identification based on the artificial intelligence method is helpful for solving the above clinical problems, so that the method also becomes a research hotspot in the field of medical image processing in recent years.
At present, research teams propose a task of realizing liver cancer identification by using traditional machine learning methods such as decision trees, support vector machines and the like. The recognition method based on traditional machine learning is to design some liver cancer feature extraction algorithms manually, for example, different features such as gray scale, shape or outline presented by different liver cancers on the imaging are utilized, and then classifiers such as decision trees, support vector machines and the like are adopted to train the manually extracted features, so as to obtain a feature recognition model. Because the classifier is designed based on the manually extracted liver cancer features, the accuracy of the feature extraction algorithm design often directly affects the final recognition performance, and the feature difference presented by different types of liver cancer is large, so the manually designed feature extraction algorithm often hardly achieves a satisfactory recognition effect. The traditional machine learning-based identification method can well identify a part of liver cancer cases, but has the problems of poor noise resistance, low identification precision and the like. With the rise of deep learning technology, the convolutional neural network has the best effect in various fields such as target recognition, detection and classification, is a classical algorithm model in deep learning, has excellent performance in the fields of image recognition and classification, is improved by combining the basic structure with a specific application field, and can be suitable for various tasks. Therefore, the invention is applied to the task of automatically identifying cholangiocarcinoma and hepatocellular carcinoma guided by multi-phase contrast enhanced CT images, and obtains good automatic diagnosis results. In addition, due to the popularization of the contrast enhanced CT imaging technology with high signal-to-noise ratio and high resolution, more and more doctors favor the application of the contrast enhanced CT imaging technology to the field of liver cancer detection. Generally, doctors use image data of three phases, namely a flat scanning phase, an arterial phase and a portal venous phase to diagnose liver cancer, the corresponding liver cancer in each phase has a unique imaging representation form, and the doctors can remarkably improve the diagnosis accuracy rate by looking up and comparing image data between different phases.
Therefore, although the prior identification method has received extensive attention and obtained certain research results, the following disadvantages still exist:
1. the sizes, shapes and positions of tumors of different patients are different, and the lesion parts are not clearly displayed, so that the traditional identification method is challenged.
2. The traditional machine learning-based identification method needs to manually design feature extraction methods of different types of liver cancer, and the final identification performance is directly influenced by the quality of the method design.
3. Most of the existing methods are based on flat scan CT image training, the tumor characteristic information which can be reflected by the data is very limited, and the identification rate is not improved by comprehensively utilizing multi-phase contrast enhancement data.
Therefore, the automatic cholangiocarcinoma and hepatocellular carcinoma identification method based on multi-stage CT image guidance has to meet the following requirements: (1) good robustness is needed when different patients are identified; (2) the method does not need to manually design feature extraction methods of different types of liver cancer, and realizes full-automatic feature learning and identification; (3) the method can carry out combined mining and learning on the liver cancer characteristics under the multi-stage CT images, so that the specific liver cancer characteristic information of each stage is fully utilized, and the identification accuracy of the model is improved.
Disclosure of Invention
In order to overcome the defects of the prior art, the technical problem to be solved by the invention is to provide an automatic classification method of liver tumor based on multi-stage CT image analysis, which can perform full-automatic cholangiocellular carcinoma and hepatocellular carcinoma identification and can obtain cholangiocellular carcinoma and hepatocellular carcinoma identification models with higher accuracy.
The technical scheme of the invention is as follows: the liver tumor automatic classification method based on the multi-stage CT image analysis comprises the following steps:
(1) collecting contrast-enhanced abdominal CT scan images of cholangiocellular carcinoma and hepatocellular carcinoma patients, storing the images into three stages of an arterial stage, a portal venous stage and a delay stage, and diagnosing liver cancer types to which all data belong as a gold standard for model training;
(2) constructing a three-dimensional full-convolution neural network segmentation model, learning by taking the cholangiocellular carcinoma and hepatocellular carcinoma image data acquired in the step (1) as the input of the model, and performing full-automatic training learning on the internal characteristics of liver tissues in each stage through the model so as to segment the liver tissues from abdominal CT images and serve as the interested region of a subsequent liver cancer identification model;
(3) and (3) constructing a three-dimensional convolutional neural network classification model, inputting the liver region-of-interest image data of each stage obtained by segmentation in the step (2) into the three-dimensional convolutional neural network classification model for training, performing combined learning and training on the characteristics of cholangiocellular carcinoma and hepatocellular carcinoma in multiple stages by the model, predicting the category of the cancer, comparing the prediction result with a gold standard, and supervising the training process of the model in a way of feeding back a loss value.
The invention uses convolution nerve network technique, integrates clinical diagnosis prior knowledge, inputs CT scanning images of two types of liver cancer of cholangiocellular carcinoma and hepatocellular carcinoma diagnosed by doctors in artery stage, portal vein stage and delay stage into the model for training and learning, so that the model can automatically learn the imaging characteristics of cholangiocellular carcinoma and hepatocellular carcinoma in each stage, and performs combined learning and training on the liver cancer characteristics in each stage, thereby being capable of carrying out full-automatic cholangiocellular carcinoma and hepatocellular carcinoma recognition and obtaining a cholangiocellular carcinoma and hepatocellular carcinoma recognition model with higher precision.
Also provided is a liver tumor automatic classification device based on multi-stage CT image analysis, which comprises:
the acquisition and classification determination module acquires abdominal CT scanning images of cholangiocellular carcinoma and hepatocellular carcinoma patients with enhanced contrast, stores the images as three phases, namely an arterial phase, a portal venous phase and a delay phase, and determines liver cancer classifications to which all data belong to serve as a gold standard for model training;
the three-dimensional full convolution neural network segmentation model construction module is used for constructing a three-dimensional full convolution neural network segmentation model, learning is carried out by taking collected cholangiocellular carcinoma and hepatocellular carcinoma image data as input of the model, and full-automatic training learning is carried out on the internal characteristics of liver tissues in each stage through the model, so that the liver tissues are segmented from abdominal CT images and serve as interested regions of a subsequent liver cancer identification model;
the three-dimensional convolutional neural network classification model building module is used for building a three-dimensional convolutional neural network classification model, inputting the segmented liver region-of-interest image data of each stage into the three-dimensional convolutional neural network classification model for training, enabling the model to carry out combined learning and training on the characteristics of cholangiocellular carcinoma and hepatocellular carcinoma in multiple stages, then predicting the category of the cancer, comparing the prediction result with a gold standard, and supervising the training process of the model in a loss value feedback mode.
Drawings
Fig. 1 is a flowchart of an automatic classification method of liver tumor based on multi-phase CT image analysis according to the present invention.
FIG. 2 is a flowchart of the establishment of a convolutive neural network liver cancer identification model guided by multi-phase contrast-enhanced CT images according to the present invention.
Detailed Description
As shown in fig. 1, the method for automatically classifying liver tumor based on multi-stage CT image analysis includes the following steps:
(1) collecting contrast-enhanced abdominal CT scan images of cholangiocellular carcinoma and hepatocellular carcinoma patients, storing the images into three stages of an arterial stage, a portal venous stage and a delay stage, and diagnosing liver cancer types to which all data belong as a gold standard for model training;
(2) constructing a three-dimensional full-convolution neural network segmentation model, learning by taking the cholangiocellular carcinoma and hepatocellular carcinoma image data acquired in the step (1) as the input of the model, and performing full-automatic training learning on the internal characteristics of liver tissues in each stage through the model so as to segment the liver tissues from abdominal CT images and serve as the interested region of a subsequent liver cancer identification model;
(3) and (3) constructing a three-dimensional convolutional neural network classification model, inputting the liver region-of-interest image data of each stage obtained by segmentation in the step (2) into the three-dimensional convolutional neural network classification model for training, performing combined learning and training on the characteristics of cholangiocellular carcinoma and hepatocellular carcinoma in multiple stages by the model, predicting the category of the cancer, comparing the prediction result with a gold standard, and supervising the training process of the model in a way of feeding back a loss value.
The invention uses convolution nerve network technique, integrates clinical diagnosis prior knowledge, inputs CT scanning images of two types of liver cancer of cholangiocellular carcinoma and hepatocellular carcinoma diagnosed by doctors in artery stage, portal vein stage and delay stage into the model for training and learning, so that the model can automatically learn the imaging characteristics of cholangiocellular carcinoma and hepatocellular carcinoma in each stage, and performs combined learning and training on the liver cancer characteristics in each stage, thereby being capable of carrying out full-automatic cholangiocellular carcinoma and hepatocellular carcinoma recognition and obtaining a cholangiocellular carcinoma and hepatocellular carcinoma recognition model with higher precision.
Preferably, the step (3) comprises the following substeps:
(3.1) cascading the multi-stage liver tissue images obtained by dividing the full convolution neural network, and inputting the images into 1 convolution layer 1 of 7 × 7 to extract the characteristics of cholangiocellular carcinoma and hepatocellular carcinoma;
(3.2) inputting the output characteristic diagram in the step (3.1) into a pooling layer 1 for scaling, reducing the size of the characteristic diagram under the condition of ensuring that the resolution of the characteristic diagram extracted in the step (3.1) is not reduced, and reducing the parameter quantity of network training;
(3.3) inputting the output characteristic diagram of the step into a full connection layer 1, and flattening the characteristic diagram into a one-dimensional vector;
and (3.4) inputting the output characteristic vectors in the step into a softmax layer 1 for classification to obtain classification results of cholangiocellular carcinoma and hepatocellular carcinoma, and completing construction of a convolutional neural network liver cancer identification model guided by a multi-phase contrast enhanced CT image.
Preferably, the following steps are further included between the steps (3.2) and (3.3):
(a) inputting the output feature map in the step (3.2) into a convolution layer 2 with 3 x 3 to further extract features;
(b) and inputting the output feature diagram in the step into a pooling layer 2 for scaling, reducing the size of the feature diagram under the condition of ensuring that the resolution of the feature diagram is not reduced, and reducing the parameters of network training.
Preferably, the step (a) is followed by the step of: inputting the output feature map of the step (a) into a plurality of convolution layers of 3 x 3 to further extract features.
Preferably, said step (a) is followed by 31 convolution layers 3-33 of 3 x 3.
FIG. 2 is a flowchart of the establishment of a convolutive neural network liver cancer identification model guided by multi-phase contrast-enhanced CT images according to the present invention.
The model establishment comprises the following steps:
the method comprises the following steps: after cascade connection, the multi-stage liver tissue images obtained by the segmentation of the full convolution neural network are input into 1 convolution layer 1 of 7 × 7 to extract the characteristics of cholangiocellular carcinoma and hepatocellular carcinoma.
Step two: and inputting the output feature diagram in the step I into a pooling layer 1 for scaling, and reducing the size and parameters of network training under the condition of ensuring that the resolution of the feature diagram extracted in the step I is not reduced.
Step three: and inputting the output feature map in the step two into a convolution layer 2 with 3 x 3 to further extract features.
Step four: and inputting the output feature map of the step three into a convolution layer 3 of 3 x 3 to further extract features.
Step five: inputting the output feature map of the step four into a convolution layer 4 of 3 x 3 to further extract features.
Step six: inputting the output feature map of the step five into a convolution layer 5 of 3 x 3 to further extract features.
Step seven: and inputting the output feature map of the step six into a convolution layer 6 of 3 x 3 to further extract features.
Step eight: and inputting the output feature map of the step seven into a convolution layer 7 of 3 x 3 to further extract features.
Step nine: and inputting the output feature map of the step eight into a convolution layer 8 of 3 x 3 to further extract features.
Step ten: and inputting the output feature map of the step nine into a convolution layer 9 of 3 x 3 to further extract features.
Step eleven: inputting the output feature map of the step ten into a convolution layer 10 of 3 x 3 to further extract features.
Step twelve: inputting the output feature map of the step eleven into a convolution layer 11 of 3 × 3 to further extract features.
Step thirteen: and inputting the output feature map of the step twelve into a convolution layer 12 of 3 x 3 to further extract features.
Fourteen steps: inputting the output feature map of the step thirteen into a convolution layer 13 of 3 x 3 to further extract features.
Step fifteen: and inputting the output feature map of the step fourteen into a convolution layer 14 of 3 x 3 to further extract features.
Sixthly, the steps are as follows: and inputting the output feature map of the step fifteen into a convolution layer 15 of 3 x 3 to further extract features.
Seventeen steps: the output signature of step sixteen is input into a 3 x 3 convolutional layer 16 for further feature extraction.
Eighteen steps: inputting the output feature map of the seventeenth step into a convolution layer 17 of 3 × 3 to further extract features.
Nineteen steps: and inputting the output feature map of the step eighteen into a convolution layer 18 of 3 x 3 to further extract features.
Twenty steps: the output feature map from the nineteenth step is input into a 3 × 3 convolutional layer 19 for further feature extraction.
Twenty one: the output signature of step twenty is input into a 3 x 3 convolutional layer 20 for further feature extraction.
Step twenty-two: and inputting the output feature map of the twenty-first step into a convolution layer 21 of 3 x 3 to further extract features.
Twenty-three steps: the output signature of step twenty-two is input into a 3 x 3 convolutional layer 22 for further feature extraction.
Twenty-four steps: and inputting the output feature map of the twenty-third step into a convolution layer 23 of 3 x 3 to further extract features.
Twenty-five steps: and inputting the output feature map of the twenty-four step into a convolution layer 24 of 3 x 3 to further extract features.
Twenty-six steps: and inputting the output feature map of the twenty-five step into a convolution layer 25 of 3 x 3 to further extract features.
Twenty-seven steps: the output signature of step twenty-six is input into a 3 x 3 convolutional layer 26 for further feature extraction.
Twenty-eight steps: the output signature of step twenty-seventh is input into a 3 x 3 convolutional layer 27 for further feature extraction.
Thirty steps are as follows: the output signature of step twenty-nine is input into a 3 x 3 convolutional layer 28 for further feature extraction.
Thirty-one steps: and inputting the output feature map of the step thirty into a convolution layer 29 of 3 x 3 to further extract features.
Step thirty-two: the output signature from step thirty one is input into a 3 x 3 convolutional layer 30 for further feature extraction.
Step thirty three: and inputting the output feature map of the step thirty-two into a convolution layer 31 of 3 x 3 to further extract features.
Thirty-four steps: the output feature map of step thirty-three is input into a 3 x 3 convolutional layer 32 for further feature extraction.
Step thirty-five: and inputting the output feature map of the step thirty-four into a convolution layer 33 of 3 x 3 to further extract features.
Step thirty-six: and inputting the output feature map in the step thirty-five into a pooling layer 2 for scaling, reducing the size of the feature map under the condition of ensuring that the resolution of the feature map is not reduced, and reducing the parameters of network training.
Step three seventeen: and inputting the output characteristic diagram of the step thirty-six into a full connection layer 1, and flattening the characteristic diagram into a one-dimensional vector.
Step thirty-eight: and inputting the output characteristic vectors obtained in the step seventeen into a softmax layer 1 for classification to obtain classification results of cholangiocellular carcinoma and hepatocellular carcinoma, and completing construction of the convolutional neural network liver cancer identification model guided by the multi-phase contrast enhanced CT image.
It will be understood by those skilled in the art that all or part of the steps in the method of the above embodiments may be implemented by hardware instructions related to a program, the program may be stored in a computer-readable storage medium, and when executed, the program includes the steps of the method of the above embodiments, and the storage medium may be: ROM/RAM, magnetic disks, optical disks, memory cards, and the like. Therefore, corresponding to the method of the present invention, the present invention also includes an automatic liver tumor classification device based on multi-stage CT image analysis, which is generally expressed in the form of functional modules corresponding to the steps of the method.
The device includes:
the acquisition and classification determination module acquires abdominal CT scanning images of cholangiocellular carcinoma and hepatocellular carcinoma patients with enhanced contrast, stores the images as three phases, namely an arterial phase, a portal venous phase and a delay phase, and determines liver cancer classifications to which all data belong to serve as a gold standard for model training;
the three-dimensional full convolution neural network segmentation model construction module is used for constructing a three-dimensional full convolution neural network segmentation model, learning is carried out by taking collected cholangiocellular carcinoma and hepatocellular carcinoma image data as input of the model, and full-automatic training learning is carried out on the internal characteristics of liver tissues in each stage through the model, so that the liver tissues are segmented from abdominal CT images and serve as interested regions of a subsequent liver cancer identification model;
the three-dimensional convolutional neural network classification model building module is used for building a three-dimensional convolutional neural network classification model, inputting the segmented liver region-of-interest image data of each stage into the three-dimensional convolutional neural network classification model for training, enabling the model to carry out combined learning and training on the characteristics of cholangiocellular carcinoma and hepatocellular carcinoma in multiple stages, then predicting the category of the cancer, comparing the prediction result with a gold standard, and supervising the training process of the model in a loss value feedback mode.
Compared with the existing liver cancer identification method, the method has the advantages that:
1. for the patient conditions with different liver cancer sizes, shapes and positions, because the model is trained by using a large number of clinically collected real patient samples, the model learns more abundant liver cancer characteristics, so that the recognition of cholangiocellular carcinoma and hepatocellular carcinoma liver cancers can be accurately realized, and the model has high robustness.
2. For the defect that the traditional machine learning method needs to manually design a liver cancer feature extraction algorithm, the method adopts a convolutional neural network technology in deep learning, automatically learns different types of liver cancer features according to data labeled by doctors, does not need to manually design a complex feature extraction algorithm, reduces the operation threshold and has higher reliability.
3. For the condition that the existing deep learning tumor identification method does not effectively utilize multi-stage contrast enhancement data, the method integrates clinical priori knowledge, and utilizes the imaging data of cholangiocellular carcinoma and hepatocellular carcinoma in the arterial stage, the portal venous stage and the delayed stage to train the model, so that the model learns and extracts the imaging characteristics of the two types of liver cancer in different stages, and the obtained model has higher identification precision.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention in any way, and all simple modifications, equivalent variations and modifications made to the above embodiment according to the technical spirit of the present invention still belong to the protection scope of the technical solution of the present invention.
Claims (6)
1. The liver tumor automatic classification method based on multi-stage CT image analysis is characterized by comprising the following steps: which comprises the following steps:
(1) collecting contrast-enhanced abdominal CT scan images of cholangiocellular carcinoma and hepatocellular carcinoma patients, storing the images into three stages of an arterial stage, a portal venous stage and a delay stage, and diagnosing liver cancer types to which all data belong as a gold standard for model training;
(2) constructing a three-dimensional full-convolution neural network segmentation model, learning by taking the cholangiocellular carcinoma and hepatocellular carcinoma image data acquired in the step (1) as the input of the model, and performing full-automatic training learning on the internal characteristics of liver tissues in each stage through the model so as to segment the liver tissues from abdominal CT images and serve as the interested region of a subsequent liver cancer identification model;
(3) and (3) constructing a three-dimensional convolutional neural network classification model, inputting the liver region-of-interest image data of each stage obtained by segmentation in the step (2) into the three-dimensional convolutional neural network classification model for training, performing combined learning and training on the characteristics of cholangiocellular carcinoma and hepatocellular carcinoma in multiple stages by the model, predicting the category of the cancer, comparing the prediction result with a gold standard, and supervising the training process of the model in a way of feeding back a loss value.
2. The method for automatically classifying liver tumor based on multi-stage CT image analysis according to claim 1, wherein: the step (3) comprises the following sub-steps:
(3.1) cascading the multi-stage liver tissue images obtained by dividing the full convolution neural network, and inputting the images into 1 convolution layer 1 of 7 × 7 to extract the characteristics of cholangiocellular carcinoma and hepatocellular carcinoma;
(3.2) inputting the output characteristic diagram in the step (3.1) into a pooling layer 1 for scaling, reducing the size of the characteristic diagram under the condition of ensuring that the resolution of the characteristic diagram extracted in the step (3.1) is not reduced, and reducing the parameter quantity of network training;
(3.3) inputting the output characteristic diagram of the step into a full connection layer 1, and flattening the characteristic diagram into a one-dimensional vector;
and (3.4) inputting the output characteristic vectors in the step into a softmax layer 1 for classification to obtain classification results of cholangiocellular carcinoma and hepatocellular carcinoma, and completing construction of a convolutional neural network liver cancer identification model guided by a multi-phase contrast enhanced CT image.
3. The method of claim 2, wherein the liver tumor is automatically classified based on multi-stage CT image analysis, and the method comprises: the following steps are further included between the steps (3.2) and (3.3):
(a) inputting the output feature map in the step (3.2) into a convolution layer 2 with 3 x 3 to further extract features;
(b) and inputting the output feature diagram in the step into a pooling layer 2 for scaling, reducing the size of the feature diagram under the condition of ensuring that the resolution of the feature diagram is not reduced, and reducing the parameters of network training.
4. The method of claim 3, wherein the liver tumor is automatically classified based on multi-stage CT image analysis, and the method comprises: said step (a) is followed by the step of: inputting the output feature map of the step (a) into a plurality of convolution layers of 3 x 3 to further extract features.
5. The method of claim 4, wherein the liver tumor is automatically classified based on multi-stage CT image analysis, and the method comprises: said step (a) is followed by 31 convolution layers 3-33 of 3 x 3.
6. Liver tumour automatic classification device based on multistage CT image analysis, its characterized in that: it includes:
the acquisition and classification determination module acquires abdominal CT scanning images of cholangiocellular carcinoma and hepatocellular carcinoma patients with enhanced contrast, stores the images as three phases, namely an arterial phase, a portal venous phase and a delay phase, and determines liver cancer classifications to which all data belong to serve as a gold standard for model training;
the three-dimensional full convolution neural network segmentation model construction module is used for constructing a three-dimensional full convolution neural network segmentation model, learning is carried out by taking collected cholangiocellular carcinoma and hepatocellular carcinoma image data as input of the model, and full-automatic training learning is carried out on the internal characteristics of liver tissues in each stage through the model, so that the liver tissues are segmented from abdominal CT images and serve as interested regions of a subsequent liver cancer identification model;
the three-dimensional convolutional neural network classification model building module is used for building a three-dimensional convolutional neural network classification model, inputting the segmented liver region-of-interest image data of each stage into the three-dimensional convolutional neural network classification model for training, enabling the model to carry out combined learning and training on the characteristics of cholangiocellular carcinoma and hepatocellular carcinoma in multiple stages, then predicting the category of the cancer, comparing the prediction result with a gold standard, and supervising the training process of the model in a loss value feedback mode.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911159730.5A CN110929789A (en) | 2019-11-22 | 2019-11-22 | Liver tumor automatic classification method and device based on multi-stage CT image analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911159730.5A CN110929789A (en) | 2019-11-22 | 2019-11-22 | Liver tumor automatic classification method and device based on multi-stage CT image analysis |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110929789A true CN110929789A (en) | 2020-03-27 |
Family
ID=69851822
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911159730.5A Pending CN110929789A (en) | 2019-11-22 | 2019-11-22 | Liver tumor automatic classification method and device based on multi-stage CT image analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110929789A (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111127467A (en) * | 2020-03-31 | 2020-05-08 | 上海联影智能医疗科技有限公司 | Image quantization method, computer device, and storage medium |
| CN111696084A (en) * | 2020-05-20 | 2020-09-22 | 平安科技（深圳）有限公司 | Cell image segmentation method, cell image segmentation device, electronic equipment and readable storage medium |
| CN112241766A (en) * | 2020-10-27 | 2021-01-19 | 西安电子科技大学 | Liver CT image multi-lesion classification method based on sample generation and transfer learning |
| CN112862785A (en) * | 2021-02-10 | 2021-05-28 | 昆明同心医联科技有限公司 | CTA image data identification method, device and storage medium |
| CN113129297A (en) * | 2021-04-30 | 2021-07-16 | 复旦大学附属中山医院 | Automatic diameter measurement method and system based on multi-phase tumor images |
| CN113657503A (en) * | 2021-08-18 | 2021-11-16 | 上海交通大学 | Malignant liver tumor classification method based on multi-modal data fusion |
| US20220138931A1 (en) * | 2020-10-30 | 2022-05-05 | International Business Machines Corporation | Lesion Detection Artificial Intelligence Pipeline Computing System |
| WO2022116914A1 (en) * | 2020-12-03 | 2022-06-09 | Ping An Technology (Shenzhen) Co., Ltd. | Method, device, and storage medium for pancreatic mass segmentation, diagnosis, and quantitative patient management |
| CN115690411A (en) * | 2022-09-22 | 2023-02-03 | 推想医疗科技股份有限公司 | Liver segment segmentation method and device, electronic equipment and storage medium |
| US11587236B2 (en) | 2020-10-30 | 2023-02-21 | International Business Machines Corporation | Refining lesion contours with combined active contour and inpainting |
| US11688517B2 (en) | 2020-10-30 | 2023-06-27 | Guerbet | Multiple operating point false positive removal for lesion identification |
| US11688063B2 (en) | 2020-10-30 | 2023-06-27 | Guerbet | Ensemble machine learning model architecture for lesion detection |
| US11694329B2 (en) | 2020-10-30 | 2023-07-04 | International Business Machines Corporation | Logistic model to determine 3D z-wise lesion connectivity |
| CN116416239A (en) * | 2023-04-13 | 2023-07-11 | 中国人民解放军海军军医大学第一附属医院 | Pancreatic CT image classification method, image classification model, electronic equipment and medium |
| US11749401B2 (en) | 2020-10-30 | 2023-09-05 | Guerbet | Seed relabeling for seed-based segmentation of a medical image |
| CN117558414A (en) * | 2023-11-23 | 2024-02-13 | 之江实验室 | System, electronic device and medium for predicting early recurrence of multi-tasking hepatocellular carcinoma |
| CN117558414B (en) * | 2023-11-23 | 2024-05-24 | 之江实验室 | System, electronic device and medium for predicting early recurrence of multi-tasking hepatocellular carcinoma |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106056596A (en) * | 2015-11-30 | 2016-10-26 | 浙江德尚韵兴图像科技有限公司 | Fully-automatic three-dimensional liver segmentation method based on local apriori information and convex optimization |
| CN109635306A (en) * | 2016-06-29 | 2019-04-16 | 潍坊学院 | Rotary machinery fault diagnosis method based on wavelet decomposition and spectrum kurtosis |
| CN109783887A (en) * | 2018-12-25 | 2019-05-21 | 西安交通大学 | A kind of intelligent recognition and search method towards Three-dimension process feature |
| CN109961443A (en) * | 2019-03-25 | 2019-07-02 | 北京理工大学 | Liver neoplasm dividing method and device based on the guidance of more phase CT images |
| CN110265141A (en) * | 2019-05-13 | 2019-09-20 | 上海大学 | A kind of liver neoplasm CT images computer aided diagnosing method |
| CN110288574A (en) * | 2019-06-13 | 2019-09-27 | 南通市传染病防治院(南通市第三人民医院) | A kind of adjuvant Ultrasonographic Diagnosis hepatoncus system and method |
-
2019
- 2019-11-22 CN CN201911159730.5A patent/CN110929789A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106056596A (en) * | 2015-11-30 | 2016-10-26 | 浙江德尚韵兴图像科技有限公司 | Fully-automatic three-dimensional liver segmentation method based on local apriori information and convex optimization |
| CN109635306A (en) * | 2016-06-29 | 2019-04-16 | 潍坊学院 | Rotary machinery fault diagnosis method based on wavelet decomposition and spectrum kurtosis |
| CN109783887A (en) * | 2018-12-25 | 2019-05-21 | 西安交通大学 | A kind of intelligent recognition and search method towards Three-dimension process feature |
| CN109961443A (en) * | 2019-03-25 | 2019-07-02 | 北京理工大学 | Liver neoplasm dividing method and device based on the guidance of more phase CT images |
| CN110265141A (en) * | 2019-05-13 | 2019-09-20 | 上海大学 | A kind of liver neoplasm CT images computer aided diagnosing method |
| CN110288574A (en) * | 2019-06-13 | 2019-09-27 | 南通市传染病防治院(南通市第三人民医院) | A kind of adjuvant Ultrasonographic Diagnosis hepatoncus system and method |
Non-Patent Citations (3)
| Title |
|---|
| TORECAN CELIK: ""Automatic Liver segmention and visualization for CT Images"", 《中国优秀硕士学位论文全文数据库 医药卫生科技辑》 * |
| 孙宇: ""医学交叉对比神经网络MCCNN在肝细胞癌和肝内胆管细胞癌中的研究与应用"", 《中国优秀硕士学位论文全文数据库 医药卫生科技辑》 * |
| 宋红等: ""腹部CT 图像序列中的肝脏分割"", 《哈尔滨工业大学学报》 * |
Cited By (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111127467A (en) * | 2020-03-31 | 2020-05-08 | 上海联影智能医疗科技有限公司 | Image quantization method, computer device, and storage medium |
| CN111127467B (en) * | 2020-03-31 | 2020-07-14 | 上海联影智能医疗科技有限公司 | Image quantization method, computer device, and storage medium |
| CN111696084A (en) * | 2020-05-20 | 2020-09-22 | 平安科技（深圳）有限公司 | Cell image segmentation method, cell image segmentation device, electronic equipment and readable storage medium |
| CN111696084B (en) * | 2020-05-20 | 2024-05-31 | 平安科技（深圳）有限公司 | Cell image segmentation method, device, electronic equipment and readable storage medium |
| CN112241766A (en) * | 2020-10-27 | 2021-01-19 | 西安电子科技大学 | Liver CT image multi-lesion classification method based on sample generation and transfer learning |
| US11688063B2 (en) | 2020-10-30 | 2023-06-27 | Guerbet | Ensemble machine learning model architecture for lesion detection |
| US11694329B2 (en) | 2020-10-30 | 2023-07-04 | International Business Machines Corporation | Logistic model to determine 3D z-wise lesion connectivity |
| US20220138931A1 (en) * | 2020-10-30 | 2022-05-05 | International Business Machines Corporation | Lesion Detection Artificial Intelligence Pipeline Computing System |
| US20220270254A1 (en) * | 2020-10-30 | 2022-08-25 | International Business Machines Corporation | Lesion Detection Artificial Intelligence Pipeline Computing System |
| US11436724B2 (en) * | 2020-10-30 | 2022-09-06 | International Business Machines Corporation | Lesion detection artificial intelligence pipeline computing system |
| US11749401B2 (en) | 2020-10-30 | 2023-09-05 | Guerbet | Seed relabeling for seed-based segmentation of a medical image |
| US11587236B2 (en) | 2020-10-30 | 2023-02-21 | International Business Machines Corporation | Refining lesion contours with combined active contour and inpainting |
| US11688517B2 (en) | 2020-10-30 | 2023-06-27 | Guerbet | Multiple operating point false positive removal for lesion identification |
| US11688065B2 (en) * | 2020-10-30 | 2023-06-27 | Guerbet | Lesion detection artificial intelligence pipeline computing system |
| WO2022116914A1 (en) * | 2020-12-03 | 2022-06-09 | Ping An Technology (Shenzhen) Co., Ltd. | Method, device, and storage medium for pancreatic mass segmentation, diagnosis, and quantitative patient management |
| CN112862785B (en) * | 2021-02-10 | 2022-11-18 | 昆明同心医联科技有限公司 | CTA image data identification method, device and storage medium |
| CN112862785A (en) * | 2021-02-10 | 2021-05-28 | 昆明同心医联科技有限公司 | CTA image data identification method, device and storage medium |
| CN113129297A (en) * | 2021-04-30 | 2021-07-16 | 复旦大学附属中山医院 | Automatic diameter measurement method and system based on multi-phase tumor images |
| CN113129297B (en) * | 2021-04-30 | 2024-03-26 | 复旦大学附属中山医院 | Diameter automatic measurement method and system based on multi-phase tumor image |
| CN113657503A (en) * | 2021-08-18 | 2021-11-16 | 上海交通大学 | Malignant liver tumor classification method based on multi-modal data fusion |
| CN115690411A (en) * | 2022-09-22 | 2023-02-03 | 推想医疗科技股份有限公司 | Liver segment segmentation method and device, electronic equipment and storage medium |
| CN116416239B (en) * | 2023-04-13 | 2024-03-12 | 中国人民解放军海军军医大学第一附属医院 | Pancreatic CT image classification method, image classification model, electronic equipment and medium |
| CN116416239A (en) * | 2023-04-13 | 2023-07-11 | 中国人民解放军海军军医大学第一附属医院 | Pancreatic CT image classification method, image classification model, electronic equipment and medium |
| CN117558414A (en) * | 2023-11-23 | 2024-02-13 | 之江实验室 | System, electronic device and medium for predicting early recurrence of multi-tasking hepatocellular carcinoma |
| CN117558414B (en) * | 2023-11-23 | 2024-05-24 | 之江实验室 | System, electronic device and medium for predicting early recurrence of multi-tasking hepatocellular carcinoma |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110929789A (en) | Liver tumor automatic classification method and device based on multi-stage CT image analysis | |
| CN107748900B (en) | Mammary gland tumor classification device and storage medium based on discriminative convolutional neural network | |
| Kowal et al. | Computer-aided diagnosis of breast cancer based on fine needle biopsy microscopic images | |
| CN111028206A (en) | Prostate cancer automatic detection and classification system based on deep learning | |
| Reshma et al. | Detection of breast cancer using histopathological image classification dataset with deep learning techniques | |
| EP2686828A1 (en) | Histology analysis | |
| CN111415352B (en) | Cancer metastasis panoramic pathological section analysis method based on deep cascade network | |
| CN111784701A (en) | Ultrasonic image segmentation method and system combining boundary feature enhancement and multi-scale information | |
| CN112862756A (en) | Method for identifying pathological change type and gene mutation in thyroid tumor pathological image | |
| WO2021183765A1 (en) | Automated detection of tumors based on image processing | |
| CN113011257A (en) | Breast cancer immunohistochemical artificial intelligence interpretation method | |
| Hassan et al. | A dilated residual hierarchically fashioned segmentation framework for extracting gleason tissues and grading prostate cancer from whole slide images | |
| Huang et al. | Recognition of transrectal ultrasound prostate image based on HOG-LBP | |
| CN117036343B (en) | FFOCT image analysis method and device for identifying axillary lymph node metastasis | |
| Sharanyaa et al. | DCNET: A Novel Implementation of Gastric Cancer Detection System through Deep Learning Convolution Networks | |
| Mocan et al. | Automatic Detection of Tumor Cells in Microscopic Images of Unstained Blood using Convolutional Neural Networks | |
| Alzubaidi et al. | Multi-class breast cancer classification by a novel two-branch deep convolutional neural network architecture | |
| Wei et al. | Multi-feature fusion for ultrasound breast image classification of benign and malignant | |
| Dickson et al. | A Dual Channel Multiscale Convolution U-Net Methodfor Liver Tumor Segmentation from Abdomen CT Images | |
| CN116206160A (en) | Automatic identification network model and automatic sketching network model construction method for nasopharyngeal carcinoma lesion tissues based on convolutional neural network model | |
| Deng et al. | Predicting differentiation degree of gastric cancer pathology images based on mask attention R-CNN | |
| CN113689950B (en) | Method, system and storage medium for identifying blood vessel distribution pattern of liver cancer IHC staining pattern | |
| CN115471512A (en) | Medical image segmentation method based on self-supervision contrast learning | |
| CN114529554A (en) | Intelligent auxiliary interpretation method for gastric cancer HER2 digital pathological section | |
| Saumiya et al. | Unified automated deep learning framework for segmentation and classification of liver tumors |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200327 |