CN117392082A - Liver CT image segmentation method and system based on full-scale jump connection - Google Patents
Liver CT image segmentation method and system based on full-scale jump connection Download PDFInfo
- Publication number
- CN117392082A CN117392082A CN202311326888.3A CN202311326888A CN117392082A CN 117392082 A CN117392082 A CN 117392082A CN 202311326888 A CN202311326888 A CN 202311326888A CN 117392082 A CN117392082 A CN 117392082A
- Authority
- CN
- China
- Prior art keywords
- layer
- decoder
- liver
- full
- encoder
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 210000004185 liver Anatomy 0.000 title claims abstract description 35
- 238000000034 method Methods 0.000 title claims abstract description 28
- 238000003709 image segmentation Methods 0.000 title claims abstract description 25
- 210000000056 organ Anatomy 0.000 claims abstract description 12
- 238000007781 pre-processing Methods 0.000 claims abstract description 8
- 238000012549 training Methods 0.000 claims description 19
- 230000006870 function Effects 0.000 claims description 16
- 230000004913 activation Effects 0.000 claims description 11
- 238000010586 diagram Methods 0.000 claims description 9
- 238000011176 pooling Methods 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 4
- 238000005070 sampling Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 3
- 230000009191 jumping Effects 0.000 claims description 2
- 238000012545 processing Methods 0.000 abstract description 3
- 230000011218 segmentation Effects 0.000 description 11
- 230000007246 mechanism Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000002591 computed tomography Methods 0.000 description 5
- 238000011156 evaluation Methods 0.000 description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 3
- 230000002776 aggregation Effects 0.000 description 2
- 238000004220 aggregation Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000003213 activating effect Effects 0.000 description 1
- 230000004931 aggregating effect Effects 0.000 description 1
- 210000003484 anatomy Anatomy 0.000 description 1
- 238000000137 annealing Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000004204 blood vessel Anatomy 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013434 data augmentation Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 210000000232 gallbladder Anatomy 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 208000019423 liver disease Diseases 0.000 description 1
- 238000002595 magnetic resonance imaging Methods 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
- G06N3/0455—Auto-encoder networks; Encoder-decoder networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30056—Liver; Hepatic
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of image processing, in particular to a liver CT image segmentation method and a system based on full-scale jump connection, comprising the steps of collecting a liver CT image, slicing and preprocessing; taking a UNet3+ network as a backbone, and introducing a attention module between an encoder and a decoder; leading out a branch from the last layer of decoder and inputting the branch into a classification guiding module, and judging whether the feature map belongs to organ features or not; multiplying the output of each node of the decoder and sending the multiplied output to a depth supervision module; and comparing the output result of the encoder layer with the label image with the corresponding size respectively, and calculating the respective loss values of different layers. The invention solves the problem that important regional characteristics are not lost when the characteristics of the network among different scales are fused.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a liver CT image segmentation method and system based on full-scale jump connection.
Background
Liver segmentation is an important task in the field of medical image processing, and has important significance in aspects of diagnosis, operation planning, treatment monitoring and the like of liver diseases; with the continued advancement of medical imaging technology, CT scanning and MRI imaging of the liver have become common clinical examination methods, producing large amounts of liver medical image data. However, these medical images typically contain a large number of anatomical structures and tissues, including the liver, blood vessels, gall bladder, etc., and thus require accurate image segmentation methods to accurately separate the liver from surrounding tissues and structures.
With the development of convolutional neural networks, semantic segmentation has made rapid progress and plays a great role in the field of medical image segmentation, three general basic frameworks, namely a Unet, a PSPNet and a Full Convolutional Network (FCN), exist in the convolutional neural networks, the Unet is based on a classical encoder-decoder structure, a series of improved versions are widely applied to the field of medical image segmentation by combining a low-level detailed feature map from a current scale encoder and a high-level semantic feature map from a corresponding next scale decoder through jump connection, and the improved versions are rapidly become baselines of semantic segmentation tasks of most medical images, in the past, many researches are carried out on the baselines, and the Unet (UNet 3+) of the full scale jump connection is one of the most representative UNet-based structures, and provides a jump connection mode capable of obtaining enough information from the full scale feature map.
Disclosure of Invention
Aiming at the defects of the existing method, the invention solves the problem that important regional characteristics are not lost when the characteristics of the network between different scales are fused.
The technical scheme adopted by the invention is as follows: the liver CT image segmentation method and system based on full-scale jump connection comprise the following steps:
step one, acquiring a liver CT image, and slicing and preprocessing;
further, the pretreatment includes: random rotation, random horizontal flip, random vertical flip, random selection.
Further, the pretreatment also comprises setting the Hounsfield unit window width to 300Hu and the window level to 50Hu.
Step two, taking a UNet3+ network as a main trunk, and introducing a attention module between an encoder and a decoder; leading out a branch from the last layer of decoder and inputting the branch into a classification guiding module, and judging whether the feature map belongs to organ features or not; and multiplying the output of each node of the decoder and sending the multiplied output to the depth supervision module.
Further, the second step specifically includes:
step 21, constructing five layers of encoders, wherein each layer of encoder takes VGG16 as a backbone, and is subjected to convolution twice 3*3, the step length is 1, and the filling is 1; performing ReLU activation and maximum pooling operation after each convolution;
step 22, constructing five layers of encoders, wherein the decoder characteristic diagram of each layer is formed by the encoder above the layer through maximum pooling, convolution of 3*3 and jump connection between full scales; the decoder under the layer is connected with full-scale internal jump through bilinear upsampling and convolution of 3*3, and the feature graphs of the total 5 different scales are spliced firstly and then pass through a 3*3 convolution, a BN normalization layer and a ReLU activation function layer, and finally the feature graphs of the decoder nodes are output;
and step 23, adding an attention gate module between a certain layer of encoder and each layer of decoder of the backbone structure, wherein before each encoder is connected to the decoder in a jumping way, the attention gate module needs to perform feature fusion of different scales through an attention gate, and the attention gate module is provided with two input signals, namely the encoding of the current layer and the up-sampling signals of other layers of different scales, and performs convolution, activation function and convolution re-addition operation to obtain an attention coefficient alpha.
Further, the output characteristics of the attention gate model are formulated as follows:
X att ＝X encoder ×α
wherein X is encoder For encoder output, α is the attention coefficient.
And step three, comparing the output result of the encoder layer with the label image with the corresponding size respectively, and calculating the respective loss values of different layers.
Further, the calculation formula of the loss value is:
wherein Y is i ∈Y，Y i Is the label image of the i-th layer, +.>Is the output of the layer i decoder node, N represents the training batch, +.>Is focus loss, +.>Is Laplace smooth dice loss, +.>Is the Lorentz hinge loss.
Further, a liver CT image segmentation system based on full-scale jump connection includes: a memory for storing instructions executable by the processor; and the processor is used for executing instructions to realize a liver CT image segmentation method based on full-scale jump connection.
Further, a computer readable medium storing computer program code, characterized in that the computer program code, when executed by a processor, implements a liver CT image segmentation method based on full-scale jump connection.
The invention has the beneficial effects that:
1. the invention focuses on the interested area better by using the full-size characteristic diagram and the attention-directing mechanism to the maximum, thereby improving the segmentation precision of the small target or the challenging area;
2. attention to different areas is enhanced through an attention mechanism, the expression capacity of the different areas in the data is improved, and a better segmentation result is obtained;
3. the constructed mixing loss function brings a smooth gradient, reduces over-fitting, processes sample imbalance, and can capture large-scale and fine-structure clear boundaries.
Drawings
FIG. 1 is a block diagram of a liver CT image segmentation method and system based on full-scale jump connection of the present invention;
FIG. 2 is X De 3 Schematic diagram of aggregation process of feature map at decoder layer;
FIG. 3 is a schematic diagram of an attention module mechanism;
FIG. 4 is a schematic diagram of a depth supervision;
FIG. 5 is a schematic diagram of a classification guideline;
fig. 6 shows the attention coefficients in the first level attention gate in different training cycles (0,10,30,60,90), with the model gradually learning to focus on the location of the liver.
Detailed Description
The invention will be further described with reference to the accompanying drawings and examples, which are simplified schematic illustrations showing only the basic structure of the invention and thus showing only those constructions that are relevant to the invention.
As shown in fig. 1, the liver CT image segmentation method based on full-scale jump connection includes the following steps:
step one, collecting a published liver CT scanning data set, slicing and preprocessing;
an LiTS liver CT image dataset is obtained, which comprises 131 training CT scans and 70 test CT scans, the training CT scans are marked by a professional doctor, and the following steps are carried out according to 8:1:1 is randomly divided into a training set, a testing set and a verification set; data preprocessing is carried out on medicine, and data augmentation is very important for medical image segmentation, so that overfitting generated by network training can be reduced, and therefore training samples are expanded by adopting random rotation, random horizontal overturn, random vertical overturn and random selection (brightness, contrast and saturation change); the liver is used as a positive class and the liver is used as a negative class, the Hounsfield Unit (HU) window width is set to be 300Hu window level to be 50Hu in the image preprocessing process, so that most irrelevant organs and tissues are removed, and the data set is preprocessed by maximum and minimum normalization after the window is set.
Step two, building a model structure, taking a UNet3+ network as a main trunk, and adding an attention mechanism module suitable for fusion of different scale features into the UNet3+ network; respectively between the encoder and decoder of the first layer to the fifth layer; the attention module can enable the network to pay more attention to the target segmentation area and inhibit the region which is not concerned so as to improve the segmentation accuracy of the network; the last layer of decoder outputs another path to the classifying guide module to judge whether the feature map belongs to organ feature, multiplies with each node output, and sends to the depth supervision module.
Step 21, constructing an encoder, wherein the encoder has five layers of nodes, the encoder is obtained through downsampling in sequence, the encoding process of the UNet3+ network is actually a characteristic extraction process, the structure of the encoder is similar to that of the VGG16, the VGG16 has the characteristics of a deep network, and strong characteristic representation capability is achieved, which means that abundant characteristic information in an input image can be extracted, and the encoder contributes to liver segmentation tasks, so that each layer of encoder module constructed takes the VGG16 as a backbone, and the convolution is carried out twice 3*3, wherein the step length is 1, the filling is 1, each convolution follows a ReLU activation layer, and the maximum pooling is carried out, namely the convolution with the step length of 2.
Step 22, constructing a decoder structure, wherein the decoder has five layers of nodes;
the decoder profile for each layer is defined by: encoders above this layer are connected by max pooling, convolution of 3*3, and skip between full scale; the decoder under the layer is connected with full-scale internal jump through bilinear upsampling and convolution of 3*3, and the feature graphs of the total 5 different scales are spliced firstly and then pass through a 3*3 convolution, a BN normalization layer and a ReLU activation function layer, and finally the feature graphs of the decoder nodes are output; the decoder of each layer can connect shallow layers and deep layers in a full-scale jump connection mode so as to reserve the bottom layer characteristics, avoid the performance degradation of a model caused by adding a plurality of layers, have the capability of exploring enough information from the full scale, clearly know the position and the boundary of organs, and each decoder layer is used for aggregating small-scale and same-scale feature images in the encoder and large-scale feature images in the decoder, so that fine-granularity semantic information and coarse-granularity semantic information under the full scale are obtained.
As in fig. 2, decoder layer X De 3 In the detailed process of feature map aggregation, the adopted convolution size is 3x3 due to the fact that the size and the channel number of the feature maps on each scale are different, the feature maps on each different scale are output into 64 channels, and the feature map sizes on different scales are consistent with the feature map size of a target decoder layer by pooling and up-adopted operation; as can be seen, decoder layer X De 3 The sources of the characteristic diagram are three parts: the first part being a small-scale encoder X En 1 And X En 2 Feature map, X of (2) En 1 And X En 2 Transmitting low-level semantic information of the bottom layer through non-overlapping maximum pooling downsampling, so that the size of the feature map is reduced by 4 times and 2 times respectively; the second part is a co-scale encoder X En 3 Is to directly receive X in a target decoder En 3 Is a feature map of (1); the third part is a large-scale encoder X En 5 Or decoder X De 5 、X De 4 Is a feature map of encoder X En 5 Or decoder X De 5 And X De 4 High-level semantic information is transferred by bilinear interpolation, so thatThe resolution of the feature map is respectively amplified by 4 times and 2 times; the number of channels is changed by convolution operation to reduce redundant information, so that 5 characteristic graphs with the number of channels being 64 are obtained, the characteristic graphs with the 5 scales are stacked, convolved by 320 filters with the convolution size of 3X3, and finally normalized BN layer and ReLU activation function layer operation is carried out to obtain X De 3 Is a feature map of (1).
As in fig. 1, step 23, an attention gate module is introduced between the encoder and the decoder;
an attention gate module is added between a certain layer of encoder and each layer of decoder of the backbone structure, the attention gate module is provided with two input signals which are respectively the encoding and up-sampling signals of the current layer and perform addition operation, so that the activation value of a repeated region is larger, the learning related to a target region is guided and enhanced, and the irrelevant region is restrained; as shown in FIG. 1, X De 1 From X En 1 、X De 2 To X De 5 The feature graphs output by the five different-scale codecs are stacked; wherein X is De 1 And X is En 1 The attention gate module between is composed of X De 2 As input.
X De 2 From X En 1 、X En 2 、X De 3 To X De 5 The feature graphs output by five different-scale codecs are stacked, wherein X is En 1 And X is De 2 The attention gate module between is composed of X En 2 As input; x is X En 2 And X is De 2 The attention gate module between is composed of X De 3 As input.
X De 3 From X En 1 -X En 3 、X De 4 To X De 5 Feature map stacking for five different scale codec outputsIn which X is En 1 And X is De 3 The attention gate module between is composed of X En 2 As input; x is X En 2 And X is De 3 The attention gate module between is composed of X De 3 As input; x is X En 3 And X is De 3 The attention gate module between is composed of X De 4 As input.
X De 4 From X En 1 -X En 4 、X De 5 The feature graphs output by five different-scale codecs are stacked, wherein X is En 1 And X is De 4 The attention gate module between is composed of X En 2 As input; x is X En 2 And X is De 4 The attention gate module between is composed of X De 3 As input; x is X En 3 And X is De 4 The attention gate module between is composed of X De 4 As input; x is X En 4 And X is De 4 The attention gate module between is composed of X De 5 As input; wherein X is De 5 And X is En 5 Is a common codec structure.
Attention coefficient α of the attention gate module:
α＝σ 2 (φ(σ 1 (W x X encoder +W g X upsample +b g ))+b φ )∈[0,1]
wherein X is encoder For encoder output, X upsample For upsampling the signal σ 1 Sum sigma 2 Activating functions, W, for ReLU and Sigmoid, respectively x And W is g Weight, b g And b φ For bias, convolution with phi 1*1, the attention coefficient alpha is larger in the target organ region and smaller in the background region, so as to achieve the purpose of screening to improve the accuracy of segmentation, and finally the current method is toThe encoded signal of the layer and the attention coefficient alpha are multiplied by pixels and then sent to up-sampling, and the calculation formula of the output characteristics of the attention gate module is as follows:
X att ＝X encoder ×α
FIG. 6 shows that the attention coefficients obtained from the training images change gradually during training, the attention gate initially evenly distributes weights at all locations, gradually updating and locating to the target organ relevant to the segmentation task as the training batch increases, so that the weights are enhanced, and the irrelevant tissues are suppressed; therefore, the attention introducing mechanism trains a plurality of attention gates on each image scale, so that the model can focus on a specific target organ in the learning process, and the reliability of jump links is enhanced.
As in fig. 4, the depth supervision structure: a convolution and ReLU activation function of 3x3 is connected behind each decoder output node, and the sizes of the decoder layer outputs under each scale are inconsistent, so that the size of the tag image GT is reduced in a downsampling mode; wherein the label image is from a LiTS liver CT image dataset, the downsampling is performed by MaxPool2d, dx2, dx4, dx8, dx16 represent scaling factors 2, 4, 8, 16.
Step three, comparing the output result of the encoder layer with the label image with corresponding size to calculate the respective loss value of different layers, summing the loss values of the 5 layers to obtain the loss value for training the network, and providing a mixed loss function for fully utilizing the feature graphs with different semantic levels at the nodes, wherein the mixed loss function comprises the following steps: laplace smooth dice loss, focus loss, and Lowatt hinge loss, the blend loss function can bring a smooth gradient, reduce overfitting, handle sample imbalance, and capture large scale and fine structure sharp boundaries, the blend loss function is described as follows:
wherein Y is i ∈Y，Y i Is the label image of the i-th layer, +.>Is the output of the layer i decoder node, N represents the training batch, +.>Is focus loss, +.>Is Laplace smooth dice loss, +.>Is the Lorentz hinge loss.
Classification guidance module structure: in most medical image segmentation, the occurrence of false positives in organ-free images is a necessary occurrence, which may be due to over-segmentation phenomena caused by noise information of the image background existing in the shallow layer; therefore, in order to solve the false positive problem of the non-organ image in the medical image segmentation, the invention applies the classification guidance module to a fifth layer decoder node of the network, the node takes another path of output, the output comprises dropout, convolution of 1x1, self-adaptive max pooling, activates the function Sigmoid to obtain a two-dimensional tensor, the value in the two-dimensional tensor represents the probability of whether a target organ exists, applies the Argmax function to obtain a single output as 0 or 1, and finally multiplies the value by branches output by 5 decoder nodes in the network structure.
Training of a model: the training set and the verification set are firstly downsampled into 320-320 sizes, then input into a built model for training, loss values are obtained by a mixed loss function in a depth supervision module, the loss values are respectively added into five layers of loss values with different scales, and model parameters are stored after training.
And sending the preprocessed pictures into the built model for training, calculating the optimal weight in the back propagation process by adopting an Adam optimizer, wherein all the super parameters are default values, and adjusting the learning rate through cosine annealing. After the training image is input into a network, each layer of decoder outputs a segmented image, the output of the lowest layer of decoder passes through a Classification Guide Module (CGM) to judge whether the image is false positive, if the image is an organ image, branches output by each node are multiplied by 1 and kept unchanged, five layers of decoders output segmented images with different scales and label images with different scales to calculate loss values, the loss values are transmitted to an Adam optimizer, and the Adam optimizer adjusts various parameters in a network structure according to the loss values.
The adopted hardware system is Windows10 professional version, the processor is Intel (R) Xeon (R) Gold6248R CPU@3.00GHz 2.99GHz (2 processors), the machine band RAM is 128G, the software operating environment is Pycharm, the programming language python3.9, and a Pytorch framework is adopted to build a network.
Model evaluation: the following evaluation indexes are used for measuring the performance of the network, wherein the evaluation indexes comprise: the parameter number, the Dice, the IoU, the Dice and the IoU index of the network model are all in the numerical ranges of [0,1], and the closer to 1, the better the model performance is represented, and the smaller the parameter number of the network model is, the better the model performance is represented. The formulae for the Dice and IoU indices are defined as follows:
where True (TP) represents correctly classified target pixels, true Negative (TN) represents correctly classified background pixels, false Positive (FP) represents incorrectly classified background pixels and False Negative (FN) represents incorrectly classified target pixels.
Table 1 is an evaluation index for liver segmentation according to the present invention:
the unet3+ network of the present invention uses full-size skip connection to transfer the extracted features from the encoder to the target decoder, facilitating integration of the layered representation; an attention gate mechanism is adopted between the skipped connection of the encoder and the decoder, so that the features extracted on different scales are selectively combined in the extended extension path, and the accuracy of the model is improved; deep and shallow features are processed simultaneously by a attention mechanism, thereby enhancing the importance of the target region while suppressing irrelevant background regions.
With the above-described preferred embodiments according to the present invention as an illustration, the above-described descriptions can be used by persons skilled in the relevant art to make various changes and modifications without departing from the scope of the technical idea of the present invention. The technical scope of the present invention is not limited to the description, but must be determined according to the scope of claims.
Claims (8)
1. The liver CT image segmentation method based on full-scale jump connection is characterized by comprising the following steps of:
step one, acquiring a liver CT image, and slicing and preprocessing;
step two, taking a UNet3+ network as a main trunk, and introducing a attention module between an encoder and a decoder; leading out a branch from the last layer of decoder and inputting the branch into a classification guiding module, and judging whether the feature map belongs to organ features or not; multiplying the output of each node of the decoder and sending the multiplied output to a depth supervision module;
and step three, comparing the output result of the encoder layer with the label image with the corresponding size respectively, and calculating the respective loss values of different layers.
2. The liver CT image segmentation method based on full-scale jump connection as recited in claim 1, wherein the preprocessing comprises: random rotation, random horizontal flip, random vertical flip, random selection.
3. The liver CT image segmentation method based on full-scale jump connection of claim 2, wherein the preprocessing further comprises setting a Hounsfield unit window width to 300Hu and a window level to 50Hu.
4. The liver CT image segmentation method based on full-scale jump connection of claim 1, wherein the step two specifically comprises:
step 21, constructing five layers of encoders, wherein each layer of encoder takes VGG16 as a backbone, and is subjected to convolution twice 3*3, the step length is 1, and the filling is 1; performing ReLU activation and maximum pooling operation after each convolution;
step 22, constructing five layers of encoders, wherein the decoder characteristic diagram of each layer is formed by the encoder above the layer through maximum pooling, convolution of 3*3 and jump connection between full scales; the decoder under the layer is connected with full-scale internal jump through bilinear upsampling and convolution of 3*3, and the feature graphs of the total 5 different scales are spliced firstly and then pass through a 3*3 convolution, a BN normalization layer and a ReLU activation function layer, and finally the feature graphs of the decoder nodes are output;
and step 23, adding an attention gate module between a certain layer of encoder and each layer of decoder of the backbone structure, wherein before each encoder is connected to the decoder in a jumping way, the attention gate module needs to perform feature fusion of different scales through an attention gate, and the attention gate module is provided with two input signals, namely the encoding of the current layer and the up-sampling signals of other layers of different scales, and performs convolution, activation function and convolution re-addition operation to obtain an attention coefficient alpha.
5. The liver CT image segmentation method based on full-scale jump connection as recited in claim 4, wherein the output characteristics of the attention gate model are formulated as:
X att ＝X encoder ×α
wherein X is encoder For encoder output, α is the attention coefficient.
6. The liver CT image segmentation method based on full-scale jump connection of claim 1, wherein the calculation formula of the loss value is:
wherein Y is i ∈Y，Y i Is the label image of the i-th layer, +.>Is the output of the layer i decoder node, N represents the training batch, +.>Is focus loss, +.>Is Laplace smooth dice loss, +.>Is the Lorentz hinge loss.
7. Liver CT image segmentation system based on full-scale jump connection, characterized by comprising: a memory for storing instructions executable by the processor; a processor for executing instructions to implement a full-scale jump connection based liver CT image segmentation method as claimed in any of claims 1-6.
8. Computer readable medium storing computer program code, characterized in that the computer program code, when executed by a processor, implements a liver CT image segmentation method based on full-scale jump connection as claimed in any of the claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311326888.3A CN117392082A (en) | 2023-10-13 | 2023-10-13 | Liver CT image segmentation method and system based on full-scale jump connection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311326888.3A CN117392082A (en) | 2023-10-13 | 2023-10-13 | Liver CT image segmentation method and system based on full-scale jump connection |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117392082A true CN117392082A (en) | 2024-01-12 |
Family
ID=89471382
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311326888.3A Pending CN117392082A (en) | 2023-10-13 | 2023-10-13 | Liver CT image segmentation method and system based on full-scale jump connection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117392082A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118212241A (en) * | 2024-05-22 | 2024-06-18 | 齐鲁工业大学(山东省科学院) | Neck MRI image analysis method based on dual-stage granularity network |
| CN118470049A (en) * | 2024-07-09 | 2024-08-09 | 中国科学院苏州生物医学工程技术研究所 | Method and medium for classifying and dividing ultrasonic image of dialysis passage and calculating vascular stenosis rate |
| CN118470049B (en) * | 2024-07-09 | 2024-10-15 | 中国科学院苏州生物医学工程技术研究所 | Method and medium for classifying and dividing ultrasonic image of dialysis passage and calculating vascular stenosis rate |
-
2023
- 2023-10-13 CN CN202311326888.3A patent/CN117392082A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118212241A (en) * | 2024-05-22 | 2024-06-18 | 齐鲁工业大学(山东省科学院) | Neck MRI image analysis method based on dual-stage granularity network |
| CN118470049A (en) * | 2024-07-09 | 2024-08-09 | 中国科学院苏州生物医学工程技术研究所 | Method and medium for classifying and dividing ultrasonic image of dialysis passage and calculating vascular stenosis rate |
| CN118470049B (en) * | 2024-07-09 | 2024-10-15 | 中国科学院苏州生物医学工程技术研究所 | Method and medium for classifying and dividing ultrasonic image of dialysis passage and calculating vascular stenosis rate |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111784671B (en) | Pathological image focus region detection method based on multi-scale deep learning | |
| CN113077471B (en) | Medical image segmentation method based on U-shaped network | |
| Pinaya et al. | Unsupervised brain imaging 3D anomaly detection and segmentation with transformers | |
| CN111627019B (en) | Liver tumor segmentation method and system based on convolutional neural network | |
| Wang et al. | Dual encoding u-net for retinal vessel segmentation | |
| CN111242233B (en) | Alzheimer disease classification method based on fusion network | |
| CN112767417B (en) | Multi-modal image segmentation method based on cascaded U-Net network | |
| CN113205524B (en) | Blood vessel image segmentation method, device and equipment based on U-Net | |
| CN116739985A (en) | Pulmonary CT image segmentation method based on transducer and convolutional neural network | |
| CN114998265A (en) | Liver tumor segmentation method based on improved U-Net | |
| CN112381846A (en) | Ultrasonic thyroid nodule segmentation method based on asymmetric network | |
| CN117392082A (en) | Liver CT image segmentation method and system based on full-scale jump connection | |
| CN110930378A (en) | Emphysema image processing method and system based on low data demand | |
| Liao et al. | Joint image quality assessment and brain extraction of fetal MRI using deep learning | |
| CN116091412A (en) | Method for segmenting tumor from PET/CT image | |
| CN117934824A (en) | Target region segmentation method and system for ultrasonic image and electronic equipment | |
| CN116486156A (en) | Full-view digital slice image classification method integrating multi-scale feature context | |
| CN116433654A (en) | Improved U-Net network spine integral segmentation method | |
| CN114581459A (en) | Improved 3D U-Net model-based segmentation method for image region of interest of preschool child lung | |
| CN117523204A (en) | Liver tumor image segmentation method and device oriented to medical scene and readable storage medium | |
| Hou | High-fidelity diabetic retina fundus image synthesis from freestyle lesion maps | |
| CN118172372A (en) | Cross-modal tumor automatic segmentation method based on PET-CT medical image and storage medium | |
| CN117474933A (en) | Medical image segmentation method based on cavity convolution attention mechanism | |
| CN115410032A (en) | OCTA image classification structure training method based on self-supervision learning | |
| CN113327221A (en) | Image synthesis method and device fusing ROI (region of interest), electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |