CN114299082A - New coronary pneumonia CT image segmentation method, device and storage medium - Google Patents
New coronary pneumonia CT image segmentation method, device and storage medium Download PDFInfo
- Publication number
- CN114299082A CN114299082A CN202111561105.0A CN202111561105A CN114299082A CN 114299082 A CN114299082 A CN 114299082A CN 202111561105 A CN202111561105 A CN 202111561105A CN 114299082 A CN114299082 A CN 114299082A
- Authority
- CN
- China
- Prior art keywords
- image
- feature map
- coronary pneumonia
- new coronary
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 206010035664 Pneumonia Diseases 0.000 title claims abstract description 80
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000003709 image segmentation Methods 0.000 title claims abstract description 34
- 238000003860 storage Methods 0.000 title claims abstract description 13
- 230000011218 segmentation Effects 0.000 claims abstract description 39
- 238000012549 training Methods 0.000 claims abstract description 22
- 238000000605 extraction Methods 0.000 claims abstract description 17
- 238000010586 diagram Methods 0.000 claims description 35
- 230000006870 function Effects 0.000 claims description 22
- 210000004072 lung Anatomy 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 17
- 230000004913 activation Effects 0.000 claims description 13
- 238000012360 testing method Methods 0.000 claims description 13
- 238000011156 evaluation Methods 0.000 claims description 12
- 238000005070 sampling Methods 0.000 claims description 11
- 238000007781 pre-processing Methods 0.000 claims description 9
- 230000003321 amplification Effects 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000010276 construction Methods 0.000 claims description 4
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 4
- 230000035945 sensitivity Effects 0.000 claims description 4
- 238000005520 cutting process Methods 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 206010025080 Lung consolidation Diseases 0.000 claims description 2
- 239000005337 ground glass Substances 0.000 claims 1
- 238000005457 optimization Methods 0.000 abstract description 5
- 230000008901 benefit Effects 0.000 description 7
- 238000013527 convolutional neural network Methods 0.000 description 6
- 238000003062 neural network model Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000003745 diagnosis Methods 0.000 description 3
- 241001678559 COVID-19 virus Species 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000007596 consolidation process Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 238000005056 compaction Methods 0.000 description 1
- 238000013170 computed tomography imaging Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 210000001519 tissue Anatomy 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Landscapes
- Apparatus For Radiation Diagnosis (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a new coronary pneumonia CT image segmentation method, equipment, a device and a computer storage medium. The invention provides a new coronary pneumonia CT image segmentation method, which comprises the following steps: inputting an image to be detected into a coding part in a TRUNET network after pre-training and optimization, obtaining a combined feature map by utilizing convolution of a residual error structure module, inputting the combined feature map into a Transformer network structure to obtain a shallow feature map, inputting the shallow feature map into a plurality of residual error structure modules for convolution to obtain a deep feature map, inputting the deep feature map into a decoding part in the TRUNET network to construct an image to obtain a new coronary pneumonia feature map, and converting the new coronary pneumonia feature map into a single channel to obtain a new coronary pneumonia feature segmentation map. The long-distance dependency relationship is considered, the local feature extraction is reserved, and the segmentation precision is improved.
Description
Technical Field
The invention relates to the technical field of medical image segmentation, in particular to a new coronary pneumonia CT image segmentation method, equipment, a device and a computer storage medium.
Background
With the higher and higher resolution of CT imaging, more tissue information can be detected, and the change of the disease condition of a patient in a short period is quicker and needs to be reviewed for many times, this undoubtedly brings great reading workload to imaging doctors, increases the probability of misjudgment and missed judgment, therefore, there is a need for an efficient and accurate method for CT image segmentation to assist diagnosis and treatment, the main subject faced by CT image segmentation is how to accurately and rapidly identify the lung fields and lesion regions in the CT image, and currently, deep learning has many excellent neural network models, in the new data, such as CT images of coronary pneumonia, however, most studies are directed to classification tasks, the method is characterized in that a normal CT image and a case CT image are distinguished, the research on the segmentation focus area is less at present, and the sudden appearance of new coronary pneumonia makes it difficult to collect enough data with labels in a short time to train a model; the full convolution network adopted by the prior art in the field of medical images is limited by the limitation of convolution, so that only local features can be focused when extracting features, and long-distance dependency relationship is ignored, so that the segmentation precision is general, and therefore, how to provide a new coronary pneumonia CT image segmentation method which considers the long-distance dependency relationship and keeps local feature extraction is a problem to be solved at present.
Disclosure of Invention
Therefore, the technical problem to be solved by the invention is to overcome the problem that the accuracy of segmentation is general because only local features can be focused and long-distance dependency is ignored when extracting features in the prior art.
In order to solve the above technical problem, the present invention provides a new coronary pneumonia CT image segmentation method, apparatus, device and computer storage medium, comprising:
inputting an image to be detected into a coding part in a TRUNET network which is trained and optimized in advance, wherein the TRUNET network comprises: the system comprises a Transformer network structure, a residual error structure module and a UNet network structure, wherein the residual error structure module comprises a convolution layer and an activation function (FRElu) layer;
utilizing a residual error structure module to carry out convolution to obtain a combined feature map;
inputting the combined characteristic diagram into the Transformer network structure to obtain a shallow characteristic diagram;
inputting the shallow feature map into a plurality of residual error structure modules to be convoluted layer by layer to obtain a deep feature map;
inputting the deep characteristic map into a decoding part in the TRUNET network, and performing up-sampling for multiple times to restore the original resolution to obtain a new coronary pneumonia characteristic map;
and outputting the new coronary pneumonia feature map into a single channel by using the convolutional layer to obtain a new coronary pneumonia feature segmentation map.
Preferably, before inputting the image to be detected into the coding part in the previously trained and optimized trunk network, the method comprises the following steps:
pre-processing the collected data set, the pre-processing comprising: removing image noise, cutting an image black area and enhancing the image;
and performing data amplification on the preprocessed data set, and dividing a training set and a testing set according to a preset proportion.
Preferably, the dataset is a four-class segmented dataset, the four-class segmented dataset pixels comprising four classes, namely background, lung fields, frosted glass-like shadows and lung compaction.
Preferably, before inputting the image to be detected into the coding part in the previously trained and optimized trunk network, the method further includes:
training TRUNET network parameters by using the training set;
verifying the trained TRUNET network parameters by using the test set, and calculating a target evaluation index to obtain a target evaluation index result, wherein the target evaluation index comprises a Dice coefficient, sensitivity and specificity;
and adjusting the TRUNET network parameters according to the target evaluation index result.
Preferably, the combined feature map includes original image information and feature information extracted after convolution.
Preferably, the inputting the combined feature map into the Transformer network structure to obtain a shallow feature map includes:
inputting the combined feature map into the Transformer network structure to obtain hidden features, wherein the Transformer network structure has 12 layers in total;
and reconstructing the hidden features into required dimensions and channels to obtain the shallow feature map.
Preferably, the inputting the deep feature map into the decoding part in the trunk network for performing multiple upsampling to restore to the original resolution, and obtaining a new coronary pneumonia feature map includes:
the deep characteristic diagram is subjected to up-sampling for multiple times and is spliced with the characteristic diagram with the same dimension of the coding part to form long-jump connection;
and constructing an image in the decoding part until the dimension of the original image is restored to obtain the new coronary pneumonia feature map.
The invention also provides a device for segmenting the new coronary pneumonia CT image, which comprises:
an input module, configured to input the to-be-detected image into the pre-trained and optimized coding portion in the trunk network, where the trunk network includes: the Transformer network structure, the residual structure and the UNet network structure, wherein the residual structure module comprises a convolution layer and an activation function (FRElu) layer;
the image information combination module is used for obtaining the combination characteristic graph by utilizing the convolution of the residual error structure module;
the shallow feature extraction module is used for inputting the combined feature map into the transform network structure to obtain the shallow feature map;
the deep characteristic extraction module is used for inputting the shallow characteristic diagram into the residual structure modules and carrying out layer-by-layer convolution to obtain a deep characteristic diagram;
the image decoding construction module is used for inputting the deep feature map into a decoding part in the TRUNET network to perform multi-time upsampling and restore the deep feature map to the original resolution so as to obtain a new coronary pneumonia feature map;
and the image conversion module is used for outputting the new coronary pneumonia feature map into a single channel by using the convolution layer to obtain a new coronary pneumonia feature segmentation map.
The invention also provides a device for segmenting the new coronary pneumonia CT image, which comprises:
a memory for storing a computer program;
and the processor is used for realizing the step of the new coronary pneumonia CT image segmentation when executing the computer program.
The invention also provides a computer readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the method for segmenting a new coronary pneumonia CT image.
Compared with the prior art, the technical scheme of the invention has the following advantages:
the invention discloses a new coronary pneumonia CT image segmentation method, which comprises the following steps: inputting an image to be detected into a coding part in a TRUNET network which is trained and optimized in advance, wherein the TRUNET network comprises: the method comprises the steps that a Transformer network structure, a residual error structure module and a UNet network structure, wherein the residual error structure module comprises a convolution layer and an activation function (FRElu) layer, a combined characteristic diagram is obtained by utilizing convolution of the residual error structure module, and the Relu function layer in the original residual error structure is replaced by the FRElu function by the residual error structure, so that the influence of the obvious improvement of a visual task due to the insensitivity of the space of the original activation function is avoided; inputting the combined feature map into the Transformer network structure to obtain a shallow feature map, wherein the Transformer network structure has the advantage of global context extraction in consideration of long-distance dependency, and improves the limitation of the limited convolution in the prior art; inputting the shallow feature map into a plurality of residual structure modules to perform convolution layer by layer to obtain a deep feature map, inputting the deep feature map into a decoding part in the TRUNET network to perform multi-time up-sampling to restore the original resolution to obtain a new coronary pneumonia feature map, and outputting the new coronary pneumonia feature map into a single channel by using a convolution layer to obtain a new coronary pneumonia feature segmentation map; the invention combines the Transformer used for sequence-to-sequence prediction with the residual error structure in the convolutional neural network, thereby not only fully playing the advantages of the Transformer for extracting the global context, but also keeping the extraction of the convolutional neural network for local characteristics, and in addition, the used residual error structure can greatly reduce the calculated amount in the training process, thereby being beneficial to the further optimization of the network.
Drawings
In order that the present disclosure may be more readily and clearly understood, reference is now made to the following detailed description of the present disclosure taken in conjunction with the accompanying drawings, in which:
FIG. 1 is a flow chart of the new coronary pneumonia CT image segmentation implementation of the present invention;
FIG. 2 is a diagram of the main body structure of the TRUNET neural network model of the present invention;
FIG. 3 is a block diagram of a modified residual structure of the present invention;
FIG. 4 is a schematic diagram of the structure of the non-linear activation function FRELU;
FIG. 5 is a flow chart of a new coronary pneumonia CT image segmentation algorithm;
fig. 6 is a block diagram of a new coronary pneumonia CT image segmentation apparatus according to an embodiment of the present invention.
Detailed Description
The core of the invention is to provide a method, a device, equipment and a computer storage medium for segmenting a new coronary pneumonia CT image, which not only consider the long-distance dependency relationship, but also reserve local feature extraction and improve the image segmentation precision.
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a flowchart illustrating an implementation of the new coronary pneumonia-based CT image segmentation according to the present invention; the specific operation steps are as follows:
s101: inputting the image to be detected into a coding part in a TRUNET network which is trained and optimized in advance;
the TRUNET network comprises: please refer to fig. 2, wherein fig. 2 is a main structure diagram of a trunk neural network model according to the present invention;
the residual error structure module includes a 3 × 3 convolution layer and an activation function FRelu layer, please refer to fig. 3 and 4, fig. 3 is a schematic diagram of the improved residual error structure module of the present invention, and fig. 4 is a schematic diagram of a structure of a nonlinear activation function FRelu;
s102: utilizing convolution of one residual error structure module to obtain a combined feature map;
the combined feature map comprises original image information and feature information extracted after convolution;
s103, inputting the combined characteristic diagram into the Transformer network structure to obtain a shallow characteristic diagram;
inputting the combined feature map into the Transformer network structure to obtain hidden features, wherein the Transformer network structure has 12 layers in total;
reconstructing the hidden features into required dimensions and channels to obtain the shallow feature map;
s104, inputting the shallow feature map into three residual error structure modules to perform layer-by-layer convolution to obtain a deep feature map;
s105: inputting the deep characteristic map into a decoding part in the TRUNET network, and performing up-sampling for multiple times to restore the original resolution to obtain a new coronary pneumonia characteristic map;
the deep characteristic diagram is subjected to up-sampling for multiple times and is spliced with the characteristic diagram with the same dimension of the coding part to form long-jump connection;
constructing an image in the decoding part until the dimension of the original image is restored to obtain the new coronary pneumonia feature map;
the decoding part adopts cascade upsampling and has a residual error structure.
S106: and outputting the new coronary pneumonia feature map into a single channel by using a 1 × 1 convolutional layer to obtain a new coronary pneumonia feature segmentation map.
The invention discloses a new coronary pneumonia CT image segmentation method, which comprises the following steps: inputting an image to be detected into a coding part in a TRUNET network which is trained and optimized in advance, wherein the TRUNET network comprises: the method comprises the steps that a Transformer network structure, a residual error structure and a UNet network structure are utilized, a combined characteristic diagram is obtained by convolution of a residual error structure module, the residual error structure module comprises a convolution layer and an activation function FRElu layer, the Relu function layer in the original residual error structure is replaced by the FRElu function in the residual error structure, and the influence that the visual task is obviously improved due to the fact that the space of the original activation function is insensitive is avoided; inputting the combined feature map into the Transformer network structure to obtain a shallow feature map, wherein the Transformer network structure has the advantage of global context extraction in consideration of long-distance dependency, and improves the limitation of the limited convolution in the prior art; inputting the shallow feature map into a plurality of residual error structure modules to perform convolution layer by layer to obtain a deep feature map, inputting the deep feature map into a decoding part in the TRUNET network to perform up-sampling for a plurality of times to restore the original resolution, and obtaining a new coronary pneumonia feature map; the long jump connection of the UNet network structure does not directly carry out supervision and loss back transmission on high-level semantic features, so that the finally recovered feature graph is ensured to be fused with more low-level features, the features of different dimensions are fused, multi-scale prediction and deep supervision can be carried out, and the information such as the recovery edge of a segmentation graph is finer by multiple times of up-sampling; outputting the new coronary pneumonia feature map into a single channel by using a convolutional layer to obtain a new coronary pneumonia feature segmentation map; the invention combines the Transformer used for sequence-to-sequence prediction with the residual error structure in the convolutional neural network, thereby not only fully playing the advantages of the Transformer for extracting the global context, but also keeping the extraction of the convolutional neural network for local characteristics, and in addition, the used residual error structure can greatly reduce the calculated amount in the training process, thereby being beneficial to the further optimization of the network.
Referring to fig. 5, fig. 5 is a schematic flow chart of a new coronary pneumonia CT image segmentation algorithm;
based on the above embodiments, this embodiment describes the new coronary pneumonia CT image segmentation algorithm in detail, which is specifically as follows:
s201: collecting a data set, preprocessing the data set, amplifying the data, and dividing the data set into a training set and a test set according to a preset proportion;
the data set used in the embodiment is a two-class segmentation data set;
pre-processing the collected data set, the pre-processing comprising: removing image noise, cutting an image black area and enhancing the image;
and performing data amplification on the preprocessed data set, and dividing a training set and a testing set according to a preset proportion.
The data set was selected from a sample of raw data from the 2019 novel coronavirus information repository (2019nCoVR) from the national bioinformatics center, which contained 750 CT images of the lungs of a new coronary pneumonia case with 4 categories of labeled image data, and a label from the stawanese doctor at the second hospital affiliated to the suzhou university, the raw CT images of the lungs being from the 2019 novel coronavirus information repository (2019 nCoVR). The resolution of both the CT image and the corresponding tag image data is 512x512, and each pixel of the tag file is a number, from 0-3, of the Background (BG), Lung Field (LF), frosted glass-like shadow (GGO), and lung real Change (CO), respectively. Due to the influence of factors such as acquisition equipment, environment and the like, some pictures have serious noise, and the noise mainly comprises the following types: the first is that the CT image is a cross-sectional image and can be overlapped with diaphragm muscle when reaching the lung bottom, so that a part of the image has large shadow, and the identification of lung fields and focuses is seriously influenced; the second is noise caused by overexposure due to too strong light; the third is that there is little valid information in the image. Directly deleting images with serious noise to ensure that the trained images have higher quality so as not to influence the performance of the model; the black background in the image belongs to an invalid region, no useful characteristic information exists, when a focus is segmented, a lung field region is a region in which a network model needs to learn and extract image characteristics, the black region usually occupies a certain resolution, and the black region is cut, so that useless pixel points can be reduced to a certain extent; meanwhile, in order to facilitate the network to better extract the characteristics of the image, the image is enhanced by adopting a method of limiting the comparison self-adaptive histogram equalization, and the enhanced image is subjected to Gaussian smoothing filtering to inhibit the image noise. And finally, performing data amplification on the image by methods of randomly rotating a certain angle (0-90 degrees), horizontally or vertically mirroring and the like to increase the number of training data, improve the generalization capability of the model and avoid the phenomenon of overfitting. And finally, carrying out data comparison according to the following steps of 8: scale of 2 demarcates the training set and the test set.
S202: constructing a TRUNET network model and training by using the data set;
training the TRUNET network parameters by using the training set, taking the data of the training set in S201 as the input of the neural network, setting the initial learning rate to be 0.0001, iterating for 500 epochs, and performing a learning rate attenuation rule as follows: respectively reducing the learning rate to 10% of the current learning rate when every 100 epochs, optimizing by using a cross EntrophyLoss loss function through optimization of an Adam optimizer, and regularizing each parameter in the network by using L2;
s203: verifying and optimizing a TRUNET model;
verifying the trained TRUNET network parameters by using the test set, and calculating a target evaluation index to obtain a target evaluation index result;
adjusting the TRUNET network parameters according to the target evaluation index result;
the target evaluation indexes of the network comprise a Dice coefficient, a Sensitivity (SE) and a Specificity (SP), and the indexes are respectively defined as:
wherein TP, FP, TN, FN represent the actual segmentation region and correctly detected segmentation region, the actual background region and incorrectly detected segmentation region, the actual background region and correctly detected background region, and the actual segmentation region and incorrectly detected background region, respectively;
the trained TRUNET model is evaluated on a test set, and the method achieves a Dice coefficient of 0.976 on a segmentation task of a lung field and achieves a Dice coefficient of 0.783 on a segmentation task of a focus;
verifying the model generated by the training of the method in the S202 by using the test set data in the S201, and continuously adjusting the parameters of the model according to the verification result to realize parameter optimization of the TRUNET model established in the S202;
s204: testing the TRUNET model optimized finally in S203 in a test set to respectively obtain a lung field segmentation image and a focus segmentation image with the same resolution;
s205: and comprehensively analyzing the lung field segmentation image and the focus segmentation image.
And calculating the area ratio of the lung field segmentation image to the area ratio of the focus segmentation image to the lung, and comprehensively obtaining the area ratio of the focus to the lung in the CT image, thereby providing the most accurate data for doctors and facilitating subsequent diagnosis and treatment.
The new coronary pneumonia CT image segmentation method provided by the invention can segment a focus region and a lung field region, provides accurate data for doctors conveniently for subsequent diagnosis and treatment by calculating respective area ratios, is more favorable for research of new coronary pneumonia compared with a neural network model aiming at classification tasks in the prior art, namely, distinguishing normal CT images and case CT images, and collects a data set for preprocessing and data expansion, and trains and optimizes the model so as to ensure that the model identification is more accurate.
Based on the above embodiments, the present embodiment adjusts the data set to make four types of segmented data sets, so that the model training is more refined, and the model segmentation accuracy is improved. The adjusted data set pixels include four categories, namely Background (BG), Lung Fields (LF), ground glass-like shadows (GGO), and lung Consolidation (CO), and other steps are the same as in the first embodiment, with the method test set yielding an mIOU of 0.785.
The invention discloses a new coronary pneumonia CT image segmentation method, which comprises the following steps: inputting an image to be detected into a coding part in a TRUNET network which is trained and optimized in advance, wherein the TRUNET network comprises: the method comprises the steps of collecting a data set, preprocessing the data set, expanding the data, adjusting a conventional two-class segmentation data set, and manufacturing a four-class segmentation data set training and optimizing model, so that the model training is more precise, and the model segmentation accuracy is improved; the method comprises the steps that a combined characteristic diagram is obtained by utilizing convolution of a residual structure module, wherein the residual structure module comprises a convolution layer and an activation function (FRElu) layer; inputting the combined feature map into the Transformer network structure to obtain a shallow feature map, wherein the Transformer network structure has the advantage of global context extraction in consideration of long-distance dependency, and improves the limitation of the limited convolution in the prior art; inputting the shallow feature map into a plurality of residual error structure modules to perform convolution layer by layer to obtain a deep feature map, inputting the deep feature map into a decoding part in the TRUNET network to perform up-sampling for a plurality of times to restore the original resolution, and obtaining a new coronary pneumonia feature map; the long jump connection of the UNet network structure does not directly carry out supervision and loss back transmission on high-level semantic features, so that the finally recovered feature graph is ensured to be fused with more low-level features, the features of different dimensions are fused, multi-scale prediction and deep supervision can be carried out, and the information such as the recovery edge of a segmentation graph is finer by multiple times of up-sampling; outputting the new coronary pneumonia feature map into a single channel by using a convolutional layer to obtain a new coronary pneumonia feature segmentation map; the invention provides a neural network new coronary pneumonia CT image segmentation method based on a Transformer and a residual structure, wherein the TRUNET neural network combines the Transformer used for sequence-to-sequence prediction with the residual structure in a convolutional neural network, so that the advantages of the Transformer for extracting global context are fully exerted, the extraction of local features by the convolutional neural network is retained, and the used residual structure can greatly reduce the calculated amount in the training process. The accuracy of segmentation is improved to a certain extent. The network model has good robustness and strong adaptability on the segmentation of the new coronary pneumonia CT image. Meanwhile, the invention also promotes the development and application of the combination of the Transformer and the convolution network in the field of medical image analysis to a certain extent.
Referring to fig. 6, fig. 6 is a block diagram illustrating a new coronary pneumonia CT image segmentation apparatus according to an embodiment of the present invention; the specific device may include:
an input module 100, configured to input the to-be-detected image into the pre-trained and optimized coding part in the trunk network, where the trunk network includes: the Transformer network structure, the residual structure module and the UNet network structure, wherein the residual structure module comprises a convolution layer and an activation function (FRElu) layer;
the image information combination module 200 is used for obtaining the combination characteristic graph by utilizing the convolution of the residual error structure module;
a shallow feature extraction module 300, configured to input the combined feature map into the transform network structure to obtain the shallow feature map;
the deep feature extraction module 400 is configured to input the shallow feature map into the plurality of residual structure modules and perform convolution layer by layer to obtain a deep feature map;
the image decoding construction module 500 is configured to input the deep feature map into a decoding portion in the trunk network, perform multiple upsampling on the deep feature map, and restore the deep feature map to an original resolution, so as to obtain a new coronary pneumonia feature map;
and an image conversion module 600, configured to output the new coronary pneumonia feature map as a single channel by using the convolutional layer, so as to obtain a new coronary pneumonia feature segmentation map.
The new coronary pneumonia CT image segmentation apparatus of the present embodiment is configured to implement the new coronary pneumonia CT image segmentation method, and therefore specific embodiments in the new coronary pneumonia CT image segmentation apparatus may be seen in the previous embodiments of the new coronary pneumonia CT image segmentation method, for example, the input module 100, the image information combination module 200, the shallow feature extraction module 300, the deep feature extraction module 400, the image decoding construction module 500, and the image conversion module 600 are respectively configured to implement steps S101, S102, S103, S104, S105, and S106 in the new coronary pneumonia CT image segmentation method, so specific embodiments thereof may refer to descriptions of corresponding embodiments of each part, and are not described herein again.
The specific embodiment of the present invention further provides a new coronary pneumonia CT image segmentation apparatus, including: a memory for storing a computer program; and the processor is used for realizing the steps of the method for segmenting the new coronary pneumonia CT image when executing the computer program.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method for segmenting the new coronary pneumonia CT image are implemented.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be understood that the above examples are only for clarity of illustration and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. And obvious variations or modifications therefrom are within the scope of the invention.
Claims (10)
1. A new coronary pneumonia CT image segmentation method is characterized by comprising the following steps:
inputting an image to be detected into a coding part in a TRUNET network which is trained and optimized in advance, wherein the TRUNET network comprises: the system comprises a Transformer network structure, a residual error structure module and a UNet network structure, wherein the residual error structure module comprises a convolution layer and an activation function (FRElu) layer;
utilizing the residual error structure module to carry out convolution to obtain a combined feature map;
inputting the combined characteristic diagram into the Transformer network structure to obtain a shallow characteristic diagram;
inputting the shallow feature map into a plurality of residual error structure modules to be convoluted layer by layer to obtain a deep feature map;
inputting the deep characteristic map into a decoding part in the TRUNET network, and performing up-sampling for multiple times to restore the original resolution to obtain a new coronary pneumonia characteristic map;
and outputting the new coronary pneumonia feature map into a single channel by using the convolutional layer to obtain a new coronary pneumonia feature segmentation map.
2. The new coronary pneumonia CT image segmentation method according to claim 1, wherein the inputting the image to be examined into the coding part of the previously trained and optimized TRUNET network comprises:
pre-processing the collected data set, the pre-processing comprising: removing image noise, cutting an image black area and enhancing the image;
and performing data amplification on the preprocessed data set, and dividing a training set and a testing set according to a preset proportion.
3. The method of claim 2, wherein the data set is a four-class segmentation data set, and the four-class segmentation data set comprises four classes of pixels, namely background, lung field, ground glass shadow and lung consolidation.
4. The new coronary pneumonia CT image segmentation method according to claim 2, wherein before inputting the image to be examined into the coding part of the previously trained and optimized TRUNET network, the method further comprises:
training TRUNET network parameters by using the training set;
verifying the trained TRUNET network parameters by using the test set, and calculating a target evaluation index to obtain a target evaluation index result, wherein the target evaluation index comprises a Di ce coefficient, sensitivity and specificity;
and adjusting the TRUNET network parameters according to the target evaluation index result.
5. The method of claim 1, wherein the combined feature map includes original image information and feature information extracted after convolution.
6. The method of claim 1, wherein the inputting the combined feature map into the transform network structure to obtain a shallow feature map comprises:
inputting the combined feature map into the Transformer network structure to obtain hidden features, wherein the Transformer network structure has 12 layers in total;
and reconstructing the hidden features into required dimensions and channels to obtain the shallow feature map.
7. The method for segmenting the CT image of new coronary pneumonia according to claim 1, wherein the inputting the deep feature map into the decoding portion of the trunk network for performing multiple upsampling to restore to the original resolution to obtain the new coronary pneumonia feature map comprises:
the deep characteristic diagram is subjected to up-sampling for multiple times and is spliced with the characteristic diagram with the same dimension of the coding part to form long-jump connection;
and constructing an image in the decoding part until the dimension of the original image is restored to obtain the new coronary pneumonia feature map.
8. An apparatus for segmenting a CT image of new coronary pneumonia, comprising:
an input module, configured to input the to-be-detected image into the pre-trained and optimized coding portion in the trunk network, where the trunk network includes: the Transformer network structure, the residual structure module and the UNet network structure, wherein the residual structure module comprises a convolution layer and an activation function (FRElu) layer;
the image information combination module is used for obtaining the combination characteristic graph by utilizing the convolution of the residual error structure module;
the shallow feature extraction module is used for inputting the combined feature map into the transform network structure to obtain the shallow feature map;
the deep characteristic extraction module is used for inputting the shallow characteristic diagram into the residual structure modules and carrying out layer-by-layer convolution to obtain a deep characteristic diagram;
the image decoding construction module is used for inputting the deep feature map into a decoding part in the TRUNET network to perform multi-time upsampling and restore the deep feature map to the original resolution so as to obtain a new coronary pneumonia feature map;
and the image conversion module is used for outputting the new coronary pneumonia feature map into a single channel by using the convolution layer to obtain a new coronary pneumonia feature segmentation map.
9. An apparatus for segmentation of CT images of new coronary pneumonia, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the method of segmentation of a new coronary pneumonia CT image as claimed in any one of claims 1 to 7 when executing said computer program.
10. A computer-readable storage medium, having a computer program stored thereon, which, when being executed by a processor, implements the steps of a method for new coronary pneumonia CT image segmentation as set forth in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111561105.0A CN114299082A (en) | 2021-12-15 | 2021-12-15 | New coronary pneumonia CT image segmentation method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111561105.0A CN114299082A (en) | 2021-12-15 | 2021-12-15 | New coronary pneumonia CT image segmentation method, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114299082A true CN114299082A (en) | 2022-04-08 |
Family
ID=80967892
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111561105.0A Pending CN114299082A (en) | 2021-12-15 | 2021-12-15 | New coronary pneumonia CT image segmentation method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114299082A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114937022A (en) * | 2022-05-31 | 2022-08-23 | 天津大学 | Novel coronary pneumonia disease detection and segmentation method |
| CN115115523A (en) * | 2022-08-26 | 2022-09-27 | 中加健康工程研究院(合肥)有限公司 | CNN and Transformer fused medical image depth information extraction method |
| CN116523840A (en) * | 2023-03-30 | 2023-08-01 | 苏州大学 | Lung CT image detection system and method based on deep learning |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112233117A (en) * | 2020-12-14 | 2021-01-15 | 浙江卡易智慧医疗科技有限公司 | New coronary pneumonia CT detects discernment positioning system and computing equipment |
| CN113191285A (en) * | 2021-05-08 | 2021-07-30 | 山东大学 | River and lake remote sensing image segmentation method and system based on convolutional neural network and Transformer |
| CN113793275A (en) * | 2021-08-27 | 2021-12-14 | 西安理工大学 | Swin Unet low-illumination image enhancement method |
-
2021
- 2021-12-15 CN CN202111561105.0A patent/CN114299082A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112233117A (en) * | 2020-12-14 | 2021-01-15 | 浙江卡易智慧医疗科技有限公司 | New coronary pneumonia CT detects discernment positioning system and computing equipment |
| CN113191285A (en) * | 2021-05-08 | 2021-07-30 | 山东大学 | River and lake remote sensing image segmentation method and system based on convolutional neural network and Transformer |
| CN113793275A (en) * | 2021-08-27 | 2021-12-14 | 西安理工大学 | Swin Unet low-illumination image enhancement method |
Non-Patent Citations (1)
| Title |
|---|
| JIENENG CHEN ET AL: "TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation", 《ARXIV:2102.04306V1》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114937022A (en) * | 2022-05-31 | 2022-08-23 | 天津大学 | Novel coronary pneumonia disease detection and segmentation method |
| CN114937022B (en) * | 2022-05-31 | 2023-04-07 | 天津大学 | Novel coronary pneumonia disease detection and segmentation method |
| CN115115523A (en) * | 2022-08-26 | 2022-09-27 | 中加健康工程研究院(合肥)有限公司 | CNN and Transformer fused medical image depth information extraction method |
| CN116523840A (en) * | 2023-03-30 | 2023-08-01 | 苏州大学 | Lung CT image detection system and method based on deep learning |
| CN116523840B (en) * | 2023-03-30 | 2024-01-16 | 苏州大学 | Lung CT image detection system and method based on deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11580646B2 (en) | Medical image segmentation method based on U-Net | |
| CN114299082A (en) | New coronary pneumonia CT image segmentation method, device and storage medium | |
| CN107247971B (en) | Intelligent analysis method and system for ultrasonic thyroid nodule risk index | |
| CN110796199B (en) | Image processing method and device and electronic medical equipment | |
| CN115018824B (en) | Colonoscope polyp image segmentation method based on CNN and Transformer fusion | |
| CN115661144B (en) | Adaptive medical image segmentation method based on deformable U-Net | |
| CN112733961A (en) | Method and system for classifying diabetic retinopathy based on attention mechanism | |
| CN113223005B (en) | Thyroid nodule automatic segmentation and grading intelligent system | |
| CN115019405A (en) | Multi-modal fusion-based tumor classification method and system | |
| CN112037212A (en) | Pulmonary tuberculosis DR image identification method based on deep learning | |
| CN113129310B (en) | Medical image segmentation system based on attention routing | |
| CN116563285B (en) | Focus characteristic identifying and dividing method and system based on full neural network | |
| CN111325282B (en) | Mammary gland X-ray image identification method and device adapting to multiple models | |
| CN112861881A (en) | Honeycomb lung recognition method based on improved MobileNet model | |
| CN111554384A (en) | Adenocarcinoma pathological image analysis method based on prior perception and multitask learning | |
| CN116862885A (en) | Segmentation guide denoising knowledge distillation method and device for ultrasonic image lesion detection | |
| CN116228759A (en) | Computer-aided diagnosis system and apparatus for renal cell carcinoma type | |
| CN115762721A (en) | Medical image quality control method and system based on computer vision technology | |
| Yang et al. | Label refinement with an iterative generative adversarial network for boosting retinal vessel segmentation | |
| CN115409812A (en) | CT image automatic classification method based on fusion time attention mechanism | |
| CN113192076B (en) | MRI brain tumor image segmentation method combining classification prediction and multi-scale feature extraction | |
| CN115294093A (en) | U-shaped pneumonia chest CT image segmentation method based on embedded residual convolution | |
| CN114419000A (en) | Femoral head necrosis index prediction system based on multi-scale geometric embedded convolutional neural network | |
| CN114565617A (en) | Pruning U-Net + + based breast tumor image segmentation method and system | |
| CN111598144A (en) | Training method and device of image recognition model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220408 |