CN116206109B - Liver tumor segmentation method based on cascade network - Google Patents
Liver tumor segmentation method based on cascade network Download PDFInfo
- Publication number
- CN116206109B CN116206109B CN202310146446.4A CN202310146446A CN116206109B CN 116206109 B CN116206109 B CN 116206109B CN 202310146446 A CN202310146446 A CN 202310146446A CN 116206109 B CN116206109 B CN 116206109B
- Authority
- CN
- China
- Prior art keywords
- input
- output
- layer
- module
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000011218 segmentation Effects 0.000 title claims abstract description 141
- 208000014018 liver neoplasm Diseases 0.000 title claims abstract description 84
- 206010019695 Hepatic neoplasm Diseases 0.000 title claims abstract description 81
- 238000000034 method Methods 0.000 title claims abstract description 15
- 210000004185 liver Anatomy 0.000 claims abstract description 53
- 238000012549 training Methods 0.000 claims abstract description 11
- 238000011176 pooling Methods 0.000 claims description 52
- 238000005070 sampling Methods 0.000 claims description 44
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 6
- 238000013135 deep learning Methods 0.000 claims description 4
- 206010028980 Neoplasm Diseases 0.000 abstract description 21
- 230000015556 catabolic process Effects 0.000 abstract description 2
- 238000006731 degradation reaction Methods 0.000 abstract description 2
- 230000008034 disappearance Effects 0.000 abstract description 2
- 239000006185 dispersion Substances 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 6
- 201000007270 liver cancer Diseases 0.000 description 3
- 239000000203 mixture Substances 0.000 description 3
- 210000001015 abdomen Anatomy 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000003709 image segmentation Methods 0.000 description 2
- 230000003902 lesion Effects 0.000 description 2
- 210000000056 organ Anatomy 0.000 description 2
- 201000011510 cancer Diseases 0.000 description 1
- 238000003759 clinical diagnosis Methods 0.000 description 1
- 238000002591 computed tomography Methods 0.000 description 1
- 238000013170 computed tomography imaging Methods 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 210000005228 liver tissue Anatomy 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
- 210000001519 tissue Anatomy 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
- G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
- G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/52—Scale-space analysis, e.g. wavelet analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/03—Recognition of patterns in medical or anatomical images
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biodiversity & Conservation Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Apparatus For Radiation Diagnosis (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a liver tumor segmentation method based on a cascade network, which adopts a mode of cascade connection of a liver segmentation network and a liver tumor segmentation network, wherein the liver segmentation network segments liver from a CT image to extract a tumor region of interest, and then the tumor region of interest is used as input of the liver tumor segmentation network to further realize accurate segmentation of liver tumors, so that the problem of unbalanced data caused by small occupation ratio of tumors in the whole CT image is solved. The liver segmentation network and the liver tumor segmentation network both take a residual network as a framework, the residual network greatly improves the depth of the network which can be effectively trained, accelerates the convergence of the training network, and reduces the model degradation, thereby effectively avoiding the problem of gradient disappearance along with the deepening of the network layers and solving the problem of gradient dispersion caused by the over-deep network.
Description
Technical Field
The invention relates to the technical field of image segmentation, in particular to a liver tumor segmentation method based on a cascade network.
Background
Liver cancer is the most common and most fatal tumor in the world, and seriously threatens the life health of people. According to the national cancer center data, the incidence rate of liver cancer in China is 5 th in all malignant tumors, and the death rate is 2 nd higher. With the development of science and technology, the computer application technology and the medical informatization level are greatly developed, and the medical image facilities are more perfect. The computed tomography imaging (computed tomography, CT) has the characteristics of quick scanning time, high image resolution and the like, and is a diagnosis mode commonly adopted by liver lesions at present. Because the liver tumor in the CT image is usually characterized by low contrast, fuzzy boundary, unfixed size, shape, position and quantity, and the like, the current clinical liver tumor segmentation needs manual intervention, namely, a doctor with abundant experience marks the outline manually, which is time-consuming and labor-consuming, is difficult to effectively adapt to the complexity and diversity of the liver tumor, has poor segmentation accuracy and lower efficiency on the target, and cannot realize the automatic segmentation of the tumor region; and the tumor area of the liver CT image of the same patient can cause different results under the mark of different doctors, and the experience and skill of the doctors are seriously depended. Therefore, the research on the accurate and efficient automatic liver tumor segmentation method has important significance for clinical diagnosis and treatment of liver cancer.
In recent years, deep learning techniques have been rapidly developed and widely used in the field of medical image segmentation. Ronneeberger et al propose a U-shaped convolutional network (Unet) that introduces a jumping connection into the convolutional network for the first time, which achieves end-to-end semantic segmentation of images through encoding-decoding operations, an encoder downsamples extracted features to capture contextual information of the images, and a decoder upsamples extracted features to precisely locate segmented regions. Li et al propose a bottleneck-supervised Unet model (bottleneck supervised Unet, BS-Unet) which is a hybrid tightly-coupled structure that is partitioned by making full use of information between layers of the network. Schlemper et al incorporate the attention mechanism into the Unet network and propose an A-Unet (attention Unet) model that automatically learns regional features related to segmentation tasks and suppresses irrelevant features. Although these networks are widely used in the field of liver tumor segmentation, there are still problems of inaccurate boundary segmentation, difficult detection of small tumors, difficult tumor segmentation due to unbalanced data types, and the like.
Disclosure of Invention
The invention aims to solve the problems of difficult segmentation and inaccurate boundary segmentation in the existing liver tumor segmentation and provides a liver tumor segmentation method based on a cascade network.
In order to solve the problems, the invention is realized by the following technical scheme:
a liver tumor segmentation method based on cascade network comprises the following steps:
firstly, constructing a liver tumor segmentation model based on a cascade network; the liver tumor segmentation model based on the cascade network consists of a liver segmentation network, a liver tumor segmentation network and a characteristic addition layer; the input of the liver segmentation network is used as the input of a liver tumor segmentation model based on a cascade network, the input and the output of the liver segmentation network are simultaneously connected with the input of the feature addition layer, the output of the feature addition layer is connected with the input of the liver tumor segmentation network, and the output of the liver tumor segmentation network is used as the output of the liver tumor segmentation model based on the cascade network;
then, constructing a training sample set by utilizing CT images which have been segmented into liver tumors in advance, and performing deep learning training on the constructed liver tumor segmentation model based on the cascade network by utilizing the training sample set to obtain a trained liver tumor segmentation model based on the cascade network;
and finally, sending the CT image which is currently required to be segmented into a trained liver tumor segmentation model based on a cascade network, and obtaining the CT image of segmented liver tumor.
The liver segmentation network and the liver tumor segmentation network are both separable convolution residual segmentation networks based on the mixed depth; the separable convolution residual error segmentation network based on the mixed depth consists of 1 input layer, 2 convolution layers, 5 first residual error modules, 4 second residual error modules, 4 pooling modules, 4 up-sampling modules and 1 output layer; the input of the input layer is used as the input of the separable convolution residual segmentation network based on the mixed depth; the output of the input layer is connected with the input of a first residual module through a first convolution layer, the output of the first residual module is connected with the input of a first pooling module and the first input of a fourth upsampling module, the output of the first pooling module is connected with the input of a second first residual module, the output of the second first residual module is connected with the input of a second pooling module and the first input of a third upsampling module, the output of the second pooling module is connected with the input of a third first residual module, the output of the third first residual module is connected with the input of a third pooling module and the first input of a second upsampling module, the output of the third pooling module is connected with the input of a fourth first residual module, the output of the fourth pooling module is connected with the input of a fourth pooling module and the first input of the first upsampling module, and the output of the fourth pooling module is connected with the input of a fifth first residual module; the output of the fifth first residual module is connected with the second input of the first up-sampling module, the output of the first up-sampling module is connected with the input of the first second residual module, the output of the first second residual module is connected with the second input of the second up-sampling module, the output of the second up-sampling module is connected with the second input of the third up-sampling module, the output of the third up-sampling module is connected with the input of the third second residual module, the output of the third up-sampling module is connected with the second input of the fourth up-sampling module, the output of the fourth up-sampling module is connected with the input of the fourth third residual module, and the output of the fourth second residual module is connected with the input of the output layer through the second convolution layer; the output of the output layer is output as a separable convolutional residual segmentation network based on the blend depth.
The first residual module of the liver segmentation network is different from the first residual module of the liver tumor segmentation network. The first residual error module of the liver segmentation network consists of 2 mixed depth separable convolution layers, 2 convolution layers and 1 characteristic addition layer; the input of the first mixed depth separable convolution layer is used as the input of a first residual error module of the liver segmentation network, the output of the first mixed depth separable convolution layer is connected with the input of the second mixed depth separable convolution layer, and the output of the second mixed depth separable convolution layer is connected with the input of the first convolution layer; the input of the second convolution layer is connected with the input of the first mixed depth separable convolution layer; the outputs of the first convolution layer and the second convolution layer are simultaneously connected with the input of the feature addition layer, and the output of the feature addition layer is used as the output of a first residual error module of the liver segmentation network. The first residual error module of the liver tumor segmentation network consists of 2 mixed depth separable convolution layers, 2 convolution layers, 1 coordinate attention mechanism layer and 1 characteristic addition layer; the input of the first mixed depth separable convolution layer is used as the input of a first residual error module of the liver segmentation network, the output of the first mixed depth separable convolution layer is connected with the input of the second mixed depth separable convolution layer, the output of the second mixed depth separable convolution layer is connected with the input of the coordinate attention mechanism layer, and the output of the coordinate attention mechanism layer is connected with the input of the first convolution layer; the input of the second convolution layer is connected with the input of the first mixed depth separable convolution layer; the outputs of the first convolution layer and the second convolution layer are simultaneously connected with the input of the feature addition layer, and the output of the feature addition layer is used as the output of a first residual error module of the liver segmentation network.
The second residual error module consists of 3 convolution layers and 1 characteristic addition layer; the input of the first convolution layer is used as the input of the second residual error module, and the output of the first convolution layer is connected with the input of the second convolution layer; the input of the third convolution layer is connected with the input of the first convolution layer; the outputs of the second convolution layer and the third convolution layer are simultaneously connected with the input of the characteristic addition layer; the output of the feature addition layer serves as the output of the second residual block.
The pooling module consists of 1 maximum pooling layer, 1 convolution layer and 1 splicing layer; the input of the maximum pooling layer and the output of the convolution layer are jointly used as the input of the pooling module, the output of the maximum pooling layer and the output of the convolution layer are jointly connected with the input of the splicing layer, and the output of the splicing layer is used as the output of the pooling module.
The up-sampling module consists of 1 bilinear interpolation layer and 1 splicing layer; the input of the bilinear interpolation layer is used as the first input of the up-sampling module, the output of the bilinear interpolation layer is connected with one input of the splicing layer, the other input of the splicing layer is used as the second input of the up-sampling module, and the output of the splicing layer is used as the output of the up-sampling module.
Compared with the prior art, the invention has the following characteristics:
1. the liver segmentation network and the liver tumor segmentation network are adopted to carry out cascading, the liver segmentation network segments the liver from the CT image firstly, a tumor region of interest is extracted, then the tumor region of interest is used as the input of the liver tumor segmentation network, the accurate segmentation of the liver tumor is further realized, and the problem of unbalanced data caused by small proportion of the tumor in the whole CT image is solved;
2. the liver segmentation network and the liver tumor segmentation network both take a residual network as a framework, the residual network greatly improves the depth of the network which can be effectively trained, accelerates the convergence of the training network, and reduces the model degradation, thereby effectively avoiding the problem of gradient disappearance along with the deepening of the network layers and solving the problem of gradient dispersion caused by the over-deep network;
3. the mixed depth separable convolution is operated on different channels by using convolution kernels with different sizes, and the multi-scale convolution kernels are fused into a single convolution operation, so that characteristic modes with different resolutions are captured, and edge details and deeper small target characteristics are extracted; through strengthening the receptive field of the segmented network feature map and fully utilizing the channel and space structure information, pixel level detail and space information can be captured better, so that the segmentation performance of the network on medical images is improved.
4. The coordinate attention mechanism can capture cross-channel information so that the model can more accurately locate and identify the lesion area.
Drawings
Fig. 1 is a schematic diagram of a liver tumor segmentation model based on a cascade network.
Fig. 2 is a schematic diagram of a separable convolutional residual partitioning network based on hybrid depth (CMDCRA-UNet).
Fig. 3 is a schematic diagram of a first Residual block (a) of a liver segmentation network and (b) of a liver tumor segmentation network.
Fig. 4 is a schematic diagram of a second Residual block 2.
Fig. 5 is a schematic diagram of a pooling module (Pool).
Fig. 6 is a schematic diagram of an Up sampling module (Up Sample).
Detailed Description
The present invention will be further described in detail with reference to specific examples in order to make the objects, technical solutions and advantages of the present invention more apparent.
A liver tumor segmentation method based on cascade network, firstly constructing a liver tumor segmentation model based on cascade network; then constructing a training sample set by utilizing CT images which have been segmented into liver tumors in advance, and performing deep learning training on the constructed liver tumor segmentation model based on the cascade network by utilizing the training sample set to obtain a trained liver tumor segmentation model based on the cascade network; and finally, sending the CT image which is currently required to be segmented into a trained liver tumor segmentation model based on a cascade network, and obtaining the CT image of segmented liver tumor.
Although the liver tumor density is different from the normal liver tissue density, the liver tumor density is similar to the tissue density of other organs in the abdomen, so that the ideal effect is difficult to obtain by directly dividing the tumor by using a network, and the liver interested region can ensure that the original CT image only keeps the liver region, thereby effectively avoiding the interference of other organs in the abdomen on the division of the liver tumor. Therefore, the liver segmentation network is firstly utilized to extract the liver region in the CT image, and then the liver tumor segmentation network is utilized to extract the tumor region in the liver region. That is, the liver tumor segmentation model based on the cascade network constructed by the invention consists of a liver segmentation network, a liver tumor segmentation network and a characteristic addition layer. As shown in fig. 1. The input of the liver segmentation network is used as the input of the liver tumor segmentation model based on the cascade network, the input and the output of the liver segmentation network are simultaneously connected with the input of the feature addition layer, the output of the feature addition layer is connected with the input of the liver tumor segmentation network, and the output of the liver tumor segmentation network is used as the output of the liver tumor segmentation model based on the cascade network. The liver segmentation network segments the liver from the original CT image, extracts a tumor region of interest, and inputs the tumor region of interest and the tumor region of interest as the liver tumor segmentation network, so that the liver tumor segmentation network further realizes accurate segmentation of liver tumors.
The liver segmentation network and the tumor segmentation network are both separable convolution residual segmentation networks based on the mixed depth. Based on the mixed depth separable convolution residual error segmentation network, the whole adopts an encoding-decoding architecture, and the residual error network is used as a framework. The separable convolution residual segmentation network based on the mixed depth consists of 1 input layer, 2 convolution layers, 5 first residual modules, 4 second residual modules, 4 pooling modules, 4 up-sampling modules and 1 output layer. As shown in fig. 2. The input of the input layer serves as the input to the separable convolutional residual segmentation network based on the blend depth. The output of the input layer is connected with the input of a first residual module through a first convolution layer, the output of the first residual module is connected with the input of a first pooling module and the first input of a fourth upsampling module, the output of the first pooling module is connected with the input of a second first residual module, the output of the second first residual module is connected with the input of a second pooling module and the first input of a third upsampling module, the output of the second pooling module is connected with the input of a third first residual module, the output of the third first residual module is connected with the input of a third pooling module and the first input of a second upsampling module, the output of the third pooling module is connected with the input of a fourth first residual module, the output of the fourth pooling module is connected with the input of a fourth pooling module and the first input of the first upsampling module, and the output of the fourth pooling module is connected with the input of a fifth first residual module. The output of the fifth first residual module is connected with the second input of the first up-sampling module, the output of the first up-sampling module is connected with the input of the first second residual module, the output of the first second residual module is connected with the second input of the second up-sampling module, the output of the second up-sampling module is connected with the second input of the third up-sampling module, the output of the third up-sampling module is connected with the input of the third second residual module, the output of the third second residual module is connected with the second input of the fourth up-sampling module, the output of the fourth up-sampling module is connected with the input of the fourth third residual module, and the output of the fourth second residual module is connected with the input of the output layer through the second convolution layer. The output of the output layer is output as a separable convolutional residual segmentation network based on the blend depth. In a separable convolution residual segmentation network based on mixed depth, firstly, an input image (an original CT image or a tumor region of interest) is convolved and dimensionality-increased by using 3×3, a coder adopts a first residual module to carry out convolution operation and extract feature images of different layers in cooperation with pooling operation, then a decoder uses a second residual module to carry out convolution operation and cooperate with up-sampling operation, the information of downsampling deletion is complemented by fusing corresponding coding layer features, and finally, pixel-level classification is carried out by using 1×1 convolution to segment a liver region or a tumor region.
In a separable convolutional residual segmentation network based on hybrid depth, a first residual module is used to extract features on the encoding path and obtain context information. In the present invention, the first residual module of the liver segmentation network and the first residual module of the liver tumor segmentation network are slightly different.
In the liver segmentation network, a first residual module of the mixed depth separable convolution residual segmentation network consists of 2 mixed depth separable convolution layers, 2 convolution layers and 1 feature addition layer. As shown in fig. 3 (a). The input of the first mixed depth separable convolution layer is used as the input of a first residual error module of the liver segmentation network, the output of the first mixed depth separable convolution layer is connected with the input of the second mixed depth separable convolution layer, and the output of the second mixed depth separable convolution layer is connected with the input of the first convolution layer; the input of the second convolution layer is connected with the input of the first mixed depth separable convolution layer; the outputs of the first convolution layer and the second convolution layer are simultaneously connected with the input of the feature addition layer, and the output of the feature addition layer is used as the output of a first residual error module of the liver segmentation network. In a first residual error module of the liver segmentation network, an input feature map is subjected to two-time mixed depth separable convolution, then 1×1 convolution is performed, and feature addition is performed on the feature map after the 1×1 convolution and the feature map after the 1×1 convolution which is originally input as an output of the first residual error module.
In a liver tumor segmentation network, a first residual module of the mixed depth separable convolution residual segmentation network consists of 2 mixed depth separable convolution layers, 2 convolution layers, 1 coordinate attention mechanism layer and 1 feature addition layer. As shown in fig. 3 (b). The input of the first mixed depth separable convolution layer is used as the input of a first residual error module of the liver segmentation network, the output of the first mixed depth separable convolution layer is connected with the input of the second mixed depth separable convolution layer, the output of the second mixed depth separable convolution layer is connected with the input of the coordinate attention mechanism layer, and the output of the coordinate attention mechanism layer is connected with the input of the first convolution layer; the input of the second convolution layer is connected with the input of the first mixed depth separable convolution layer; the outputs of the first convolution layer and the second convolution layer are simultaneously connected with the input of the feature addition layer, and the output of the feature addition layer is used as the output of a first residual error module of the liver segmentation network. In a first residual error module of the liver tumor segmentation network, an input feature map is subjected to two-time mixed depth separable convolution, then is subjected to a coordinate attention mechanism, is subjected to 1×1 convolution, and is subjected to feature addition with a feature map obtained by carrying out 1×1 convolution on the feature map obtained after the 1×1 convolution and the feature map which is originally input, so as to be used as the output of the first residual error module.
Because the convolution receptive field range in the residual error network is limited, the image features of the high-resolution liver edge and the tumor extracted by the network are insufficient, the method adds the mixed depth separable convolution into the first residual error module, groups the channels by the mixed depth separable convolution, and convolves by using convolution kernels with different sizes, thereby obtaining the mixed receptive field and capturing the high-resolution features and the low-resolution features. In the mixed depth separable convolution, channels of an input image are uniformly divided into 4 groups, convolution kernels of {3×3,5×5,7×7,9×9} are used for convolution, and finally, four feature maps after convolution are spliced.
In a separable convolution residual segmentation network based on mixed depth, a second residual module is used for precisely positioning a liver region and a tumor region on a decoding path. In the present invention, the second residual module of the liver segmentation network is identical to the second residual module of the liver tumor segmentation network. The second residual block consists of 3 convolutional layers and 1 feature addition layer. As shown in fig. 4. The input of the first convolution layer is used as the input of the second residual error module, and the output of the first convolution layer is connected with the input of the second convolution layer; the input of the third convolution layer is connected with the input of the first convolution layer; the outputs of the second convolution layer and the third convolution layer are simultaneously connected with the input of the characteristic addition layer; the output of the feature addition layer serves as the output of the second residual block. In the second residual module, the input feature map is subjected to 3×3 convolution operation twice, and then feature addition is performed on the feature map subjected to 1×1 convolution with the feature map which is initially input, so as to serve as the output of the second residual module.
In a separable convolutional residual segmentation network based on mixed depth, the pooling module consists of 1 max pooling layer, 1 convolutional layer and 1 splice layer. As shown in fig. 5. The input of the maximum pooling layer and the output of the convolution layer are jointly used as the input of the pooling module, the output of the maximum pooling layer and the output of the convolution layer are jointly connected with the input of the splicing layer, and the output of the splicing layer is used as the output of the pooling module. In the pooling module, the size of the feature map is reduced by adopting 3×3 convolution with the maximum pooling and the step length of 2, and the pooled feature map and the convolved feature map are spliced, so that the receptive field is enlarged.
In a separable convolution residual segmentation network based on mixed depth, an up-sampling module consists of 1 bilinear interpolation layer and 1 splicing layer. As shown in fig. 6. The input of the bilinear interpolation layer is used as the first input of the up-sampling module, the output of the bilinear interpolation layer is connected with one input of the splicing layer, the other input of the splicing layer is used as the second input of the up-sampling module, and the output of the splicing layer is used as the output of the up-sampling module. In the up-sampling module, the image size is expanded by bilinear interpolation, and the expanded feature image is spliced with the corresponding feature image in the coding path, so that a better feature reconstruction effect is achieved.
It should be noted that, although the examples described above are illustrative, this is not a limitation of the present invention, and thus the present invention is not limited to the above-described specific embodiments. Other embodiments, which are apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein, are considered to be within the scope of the invention as claimed.
Claims (5)
1. The liver tumor segmentation method based on the cascade network is characterized by comprising the following steps of:
firstly, constructing a liver tumor segmentation model based on a cascade network; the liver tumor segmentation model based on the cascade network consists of a liver segmentation network, a liver tumor segmentation network and a characteristic addition layer; the input of the liver segmentation network is used as the input of a liver tumor segmentation model based on a cascade network, the input and the output of the liver segmentation network are simultaneously connected with the input of the feature addition layer, the output of the feature addition layer is connected with the input of the liver tumor segmentation network, and the output of the liver tumor segmentation network is used as the output of the liver tumor segmentation model based on the cascade network;
the liver segmentation network and the liver tumor segmentation network are both separable convolution residual segmentation networks based on the mixed depth; the separable convolution residual error segmentation network based on the mixed depth consists of 1 input layer, 2 convolution layers, 5 first residual error modules, 4 second residual error modules, 4 pooling modules, 4 up-sampling modules and 1 output layer;
the input of the input layer is used as the input of the separable convolution residual segmentation network based on the mixed depth; the output of the input layer is connected with the input of a first residual module through a first convolution layer, the output of the first residual module is connected with the input of a first pooling module and the first input of a fourth upsampling module, the output of the first pooling module is connected with the input of a second first residual module, the output of the second first residual module is connected with the input of a second pooling module and the first input of a third upsampling module, the output of the second pooling module is connected with the input of a third first residual module, the output of the third first residual module is connected with the input of a third pooling module and the first input of a second upsampling module, the output of the third pooling module is connected with the input of a fourth first residual module, the output of the fourth pooling module is connected with the input of a fourth pooling module and the first input of the first upsampling module, and the output of the fourth pooling module is connected with the input of a fifth first residual module; the output of the fifth first residual module is connected with the second input of the first up-sampling module, the output of the first up-sampling module is connected with the input of the first second residual module, the output of the first second residual module is connected with the second input of the second up-sampling module, the output of the second up-sampling module is connected with the second input of the third up-sampling module, the output of the third up-sampling module is connected with the input of the third second residual module, the output of the third up-sampling module is connected with the second input of the fourth up-sampling module, the output of the fourth up-sampling module is connected with the input of the fourth third residual module, and the output of the fourth second residual module is connected with the input of the output layer through the second convolution layer; the output of the output layer is used as the output of a separable convolution residual segmentation network based on the mixed depth;
then, constructing a training sample set by utilizing CT images which have been segmented into liver tumors in advance, and performing deep learning training on the constructed liver tumor segmentation model based on the cascade network by utilizing the training sample set to obtain a trained liver tumor segmentation model based on the cascade network;
and finally, sending the CT image which is currently required to be segmented into a trained liver tumor segmentation model based on a cascade network, and obtaining the CT image of segmented liver tumor.
2. The cascade network-based liver tumor segmentation method according to claim 1, wherein a first residual module of the liver segmentation network is different from a first residual module of the liver tumor segmentation network;
the first residual error module of the liver segmentation network consists of 2 mixed depth separable convolution layers, 2 convolution layers and 1 characteristic addition layer; the input of the first mixed depth separable convolution layer is used as the input of a first residual error module of the liver segmentation network, the output of the first mixed depth separable convolution layer is connected with the input of the second mixed depth separable convolution layer, and the output of the second mixed depth separable convolution layer is connected with the input of the first convolution layer; the input of the second convolution layer is connected with the input of the first mixed depth separable convolution layer; the output of the first convolution layer and the output of the second convolution layer are simultaneously connected with the input of the characteristic addition layer, and the output of the characteristic addition layer is used as the output of a first residual error module of the liver segmentation network;
the first residual error module of the liver tumor segmentation network consists of 2 mixed depth separable convolution layers, 2 convolution layers, 1 coordinate attention mechanism layer and 1 characteristic addition layer; the input of the first mixed depth separable convolution layer is used as the input of a first residual error module of the liver segmentation network, the output of the first mixed depth separable convolution layer is connected with the input of the second mixed depth separable convolution layer, the output of the second mixed depth separable convolution layer is connected with the input of the coordinate attention mechanism layer, and the output of the coordinate attention mechanism layer is connected with the input of the first convolution layer; the input of the second convolution layer is connected with the input of the first mixed depth separable convolution layer; the outputs of the first convolution layer and the second convolution layer are simultaneously connected with the input of the feature addition layer, and the output of the feature addition layer is used as the output of a first residual error module of the liver segmentation network.
3. The cascade network-based liver tumor segmentation method of claim 1, wherein the second residual module consists of 3 convolution layers and 1 feature addition layer; the input of the first convolution layer is used as the input of the second residual error module, and the output of the first convolution layer is connected with the input of the second convolution layer; the input of the third convolution layer is connected with the input of the first convolution layer; the outputs of the second convolution layer and the third convolution layer are simultaneously connected with the input of the characteristic addition layer; the output of the feature addition layer serves as the output of the second residual block.
4. The liver tumor segmentation method based on cascade network as claimed in claim 1, wherein the pooling module is composed of 1 maximum pooling layer, 1 convolution layer and 1 splicing layer; the input of the maximum pooling layer and the output of the convolution layer are jointly used as the input of the pooling module, the output of the maximum pooling layer and the output of the convolution layer are jointly connected with the input of the splicing layer, and the output of the splicing layer is used as the output of the pooling module.
5. The liver tumor segmentation method based on cascade network as claimed in claim 1, wherein the up-sampling module is composed of 1 bilinear interpolation layer and 1 splicing layer; the input of the bilinear interpolation layer is used as the first input of the up-sampling module, the output of the bilinear interpolation layer is connected with one input of the splicing layer, the other input of the splicing layer is used as the second input of the up-sampling module, and the output of the splicing layer is used as the output of the up-sampling module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310146446.4A CN116206109B (en) | 2023-02-21 | 2023-02-21 | Liver tumor segmentation method based on cascade network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310146446.4A CN116206109B (en) | 2023-02-21 | 2023-02-21 | Liver tumor segmentation method based on cascade network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116206109A CN116206109A (en) | 2023-06-02 |
| CN116206109B true CN116206109B (en) | 2023-11-07 |
Family
ID=86515543
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310146446.4A Active CN116206109B (en) | 2023-02-21 | 2023-02-21 | Liver tumor segmentation method based on cascade network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116206109B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111179237A (en) * | 2019-12-23 | 2020-05-19 | 北京理工大学 | Image segmentation method and device for liver and liver tumor |
| CN112686905A (en) * | 2020-12-22 | 2021-04-20 | 天津大学 | Lightweight brain tumor segmentation method based on depth separable convolution |
| CN112861727A (en) * | 2021-02-09 | 2021-05-28 | 北京工业大学 | Real-time semantic segmentation method based on mixed depth separable convolution |
| CN114240962A (en) * | 2021-11-23 | 2022-03-25 | 湖南科技大学 | CT image liver tumor region automatic segmentation method based on deep learning |
| CN114693933A (en) * | 2022-04-07 | 2022-07-01 | 天津大学 | Medical image segmentation device based on generation of confrontation network and multi-scale feature fusion |
| CN115131369A (en) * | 2022-06-15 | 2022-09-30 | 南京邮电大学 | CT image liver tumor segmentation method based on 3DA-U-Nets framework |
| CN115661538A (en) * | 2022-11-02 | 2023-01-31 | 南通大学 | Image classification method based on multi-granularity Transformer |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110084095B (en) * | 2019-03-12 | 2022-03-25 | 浙江大华技术股份有限公司 | Lane line detection method, lane line detection apparatus, and computer storage medium |
| KR102215757B1 (en) * | 2019-05-14 | 2021-02-15 | 경희대학교 산학협력단 | Method, apparatus and computer program for image segmentation |
| WO2021263038A1 (en) * | 2020-06-25 | 2021-12-30 | Arrive PTE Ltd. | System and methods for characterizing cells and microenvironments |
-
2023
- 2023-02-21 CN CN202310146446.4A patent/CN116206109B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111179237A (en) * | 2019-12-23 | 2020-05-19 | 北京理工大学 | Image segmentation method and device for liver and liver tumor |
| CN112686905A (en) * | 2020-12-22 | 2021-04-20 | 天津大学 | Lightweight brain tumor segmentation method based on depth separable convolution |
| CN112861727A (en) * | 2021-02-09 | 2021-05-28 | 北京工业大学 | Real-time semantic segmentation method based on mixed depth separable convolution |
| CN114240962A (en) * | 2021-11-23 | 2022-03-25 | 湖南科技大学 | CT image liver tumor region automatic segmentation method based on deep learning |
| CN114693933A (en) * | 2022-04-07 | 2022-07-01 | 天津大学 | Medical image segmentation device based on generation of confrontation network and multi-scale feature fusion |
| CN115131369A (en) * | 2022-06-15 | 2022-09-30 | 南京邮电大学 | CT image liver tumor segmentation method based on 3DA-U-Nets framework |
| CN115661538A (en) * | 2022-11-02 | 2023-01-31 | 南通大学 | Image classification method based on multi-granularity Transformer |
Non-Patent Citations (4)
| Title |
|---|
| 《MTU: A multi-tasking U-net with hybrid convolutional learning and attention modules for cancer classification and gland Segmentation in Colon Histopathological Images》;Manju Dabass等;《Computers in Biology and Medicine》;全文 * |
| 《Multilevel depth-wise context attention network with atrous mechanism for segmentation of COVID19 affected regions》;Abdul Qayyum等;《Neural Computing and Applications》;全文 * |
| 《基于全局深度分离卷积残差网络的高效人脸识别算法》;陈希彤 等;《武汉工程大学学报》;全文 * |
| 《多通道深度可分离卷积模型实时识别复杂背景下甜菜与杂草》;孙俊 等;《农业工程学报》;第35卷(第12期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116206109A (en) | 2023-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113012172B (en) | AS-UNet-based medical image segmentation method and system | |
| CN109615636B (en) | Blood vessel tree construction method and device in lung lobe segment segmentation of CT (computed tomography) image | |
| CN113674253B (en) | Automatic segmentation method for rectal cancer CT image based on U-transducer | |
| CN111369565B (en) | Digital pathological image segmentation and classification method based on graph convolution network | |
| WO2023071531A1 (en) | Liver ct automatic segmentation method based on deep shape learning | |
| CN111127482A (en) | CT image lung trachea segmentation method and system based on deep learning | |
| CN114092439A (en) | Multi-organ instance segmentation method and system | |
| CN111047605B (en) | Construction method and segmentation method of vertebra CT segmentation network model | |
| CN112446892A (en) | Cell nucleus segmentation method based on attention learning | |
| CN111275712B (en) | Residual semantic network training method oriented to large-scale image data | |
| CN114119977A (en) | Graph convolution-based Transformer gastric cancer canceration region image segmentation method | |
| CN112465754B (en) | 3D medical image segmentation method and device based on layered perception fusion and storage medium | |
| CN111429473A (en) | Chest film lung field segmentation model establishment and segmentation method based on multi-scale feature fusion | |
| CN108305253A (en) | A kind of pathology full slice diagnostic method based on more multiplying power deep learnings | |
| CN114037714A (en) | 3D MR and TRUS image segmentation method for prostate system puncture | |
| CN113888466A (en) | Pulmonary nodule image detection method and system based on CT image | |
| CN112734755A (en) | Lung lobe segmentation method based on 3D full convolution neural network and multitask learning | |
| CN113436173A (en) | Abdomen multi-organ segmentation modeling and segmentation method and system based on edge perception | |
| CN112150449A (en) | Cerebral apoplexy focus segmentation method and system | |
| CN111091575A (en) | Medical image segmentation method based on reinforcement learning method | |
| CN116664590B (en) | Automatic segmentation method and device based on dynamic contrast enhancement magnetic resonance image | |
| CN113643310A (en) | Context polymerization-based MRI image hepatic vessel segmentation method | |
| CN116206109B (en) | Liver tumor segmentation method based on cascade network | |
| CN117523204A (en) | Liver tumor image segmentation method and device oriented to medical scene and readable storage medium | |
| CN115731387A (en) | Multi-type pulmonary nodule accurate segmentation method and model based on mixed Transformer |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |