CN114743000A - Multitask pneumothorax medical image semantic segmentation model system and method based on Unet - Google Patents

Multitask pneumothorax medical image semantic segmentation model system and method based on Unet Download PDF

Info

Publication number
CN114743000A
CN114743000A CN202210312808.8A CN202210312808A CN114743000A CN 114743000 A CN114743000 A CN 114743000A CN 202210312808 A CN202210312808 A CN 202210312808A CN 114743000 A CN114743000 A CN 114743000A
Authority
CN
China
Prior art keywords
model
pneumothorax
module
image
semantic segmentation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210312808.8A
Other languages
Chinese (zh)
Inventor
沈旭东
吴湘莲
楼平
雷英栋
陶冶博
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiaxing Vocational and Technical College
Original Assignee
Jiaxing Vocational and Technical College
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiaxing Vocational and Technical College filed Critical Jiaxing Vocational and Technical College
Priority to CN202210312808.8A priority Critical patent/CN114743000A/en
Publication of CN114743000A publication Critical patent/CN114743000A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)

Abstract

The invention discloses a multitask pneumothorax medical image semantic segmentation model system and a multitask pneumothorax medical image semantic segmentation model method based on Unet, wherein the multitask pneumothorax medical image semantic segmentation model method based on Unet comprises the following steps of S1: acquiring a pneumothorax CT image and carrying out coding processing on the pneumothorax CT image through a first model and a second model so as to obtain an image second classification. The invention discloses a multitask pneumothorax medical image semantic segmentation model system and a multitask pneumothorax medical image semantic segmentation method based on Unet, wherein the traditional Unet semantic segmentation is improved, Resnet34 is used as a frame in a coding stage, an SCSE module is introduced for correcting image characteristic information, global information of an image is acquired from two directions of space and a channel, a loss function is learned by adopting a multitask strategy of combining classification and segmentation, and the semantic segmentation is carried out on the pneumothorax medical image.

Description

Multitask pneumothorax medical image semantic segmentation model system and method based on Unet
Technical Field
The invention belongs to the technical field of pneumothorax image analysis, and particularly relates to a multitask pneumothorax medical image semantic segmentation model system based on Unet and a multitask pneumothorax medical image semantic segmentation model method based on Unet.
Background
Pneumothorax means that gas enters a pleural cavity to cause an pneumatosis state, the pneumothorax is a great risk to human life, the accuracy of general judgment is 50%, and the threat to human is great, wherein a general method is to diagnose through an X-ray film, but the X-ray film can be judged only by doctors with abundant experience, currently, with the continuous development of deep learning technology, whether patients suffer from pneumothorax diseases can be quickly identified through a deep learning model, the positions of focuses can be judged, whether the patients suffer from pneumothorax diseases is predicted through a network model, the burden of the doctors is greatly relieved, and an effective auxiliary means can be provided.
In the process of semantic segmentation by using a traditional Unet model, the first problem is that the recall rate of a positive sample is low, and an X-ray film has pneumothorax focus, but the positive sample is not identified in the process of prediction by using the semantic segmentation model; the second problem is whether to design a better network structure on the basis of the original structure and further improve the precision of semantic segmentation.
Therefore, the above problems are further improved.
Disclosure of Invention
The invention mainly aims to provide a multitask pneumothorax medical image semantic segmentation model system and a multitask pneumothorax medical image semantic segmentation method based on Unet, wherein the traditional Unet semantic segmentation is improved, Resnet34 is used as a frame in a coding stage, an SCSE module is introduced for correcting image characteristic information, global information of an image is obtained from two directions of space and a channel, a loss function is learned by adopting a multitask strategy of combining classification and segmentation, and the pneumothorax medical image is subjected to semantic segmentation, so that the semantic segmentation method has certain advantages in segmentation accuracy compared with the traditional Unet semantic segmentation method.
In order to achieve the above purposes, the invention provides a multitask pneumothorax medical image semantic segmentation model method based on Unet, which is used for performing semantic segmentation on a pneumothorax medical image and comprises the following steps:
step S1: acquiring a pneumothorax CT image, and carrying out coding processing on the pneumothorax CT image through a first model (an Unet model with a frame of Resnet34 and pre-training parameters) and a second model (an SCSE module structure) so as to obtain an image second classification;
step S2: during decoding processing, fusing characteristic information of each layer in the encoding processing stage, performing up-sampling, and combining a second model to continuously fuse the information of each layer subjected to up-sampling so as to obtain a semantic segmentation image;
step S3: and fusing the loss function of the image secondary classification and the loss function of the semantic segmentation image, learning the loss function by adopting a multi-task strategy of classification and segmentation fusion, fusing the local pixel points and the global pixel points in the classification, and obtaining the global loss function after adjusting the weight relation so as to realize the semantic segmentation of the pneumothorax CT image.
As a further preferable embodiment of the above technical means, step S1 is specifically implemented as the following steps:
step S1.1: the first model is used as a basic feature extraction network of the pneumothorax CT image and carries out basic feature extraction by a method of adding a residual error network in the network;
step S1.2: the second model corrects the extracted basic features through a space compression module and a channel compression module, so that correction data of each layer corresponding to the first model are obtained;
step S1.3: and carrying out coding processing on the pneumothorax CT image through the first model and the second model so as to obtain a second classification of the image.
As a further preferable embodiment of the above-mentioned technical means, step S2 is specifically implemented as the following steps:
step S2.1: a first module (64, 32, 32) in the encoding process upsamples (Bn1+ Relu + Maxpool) a module in which a largest pooling process in the encoding process is located to obtain a first feature map and performs a first modification process on the first feature map by a second model to obtain first modified data;
step S2.2: a second module (64, 64, 64) in the coding process up-samples the first correction data obtained by the first module and fuses correction data corresponding to a module in which the fifth layer of the first model in the coding process is located to obtain a second feature map and performs a second correction process on the second feature map through the second model to obtain second correction data;
step S2.3: a third module (64, 128, 128) in the coding process performs up-sampling on the second correction data obtained by the second module and fuses correction data corresponding to a module where a fourth layer of the first model in the coding process is located to obtain a third feature map and performs a third correction process on the third feature map through the second model to obtain third correction data;
step S2.4: a fourth module (64, 256, 256) in the encoding process up-samples the third correction data obtained by the third module and fuses the correction data corresponding to the module in which the third layer of the first model in the encoding process is located to obtain a fourth feature map and performs a fourth correction process on the fourth feature map through the second model to obtain fourth correction data;
step S2.5: a fifth module (64, 512, 512) in the encoding process performs up-sampling on the fourth correction data obtained by the fourth module and fuses correction data corresponding to a module in which the second layer of the first model in the encoding process is located to obtain a fifth feature map and performs a fifth correction process on the fifth feature map through the second model to obtain fifth correction data;
step S2.6: and fusing the corresponding correction data obtained by each module in the encoding process so as to complete the encoding process and further obtain the semantic segmentation image.
As a more preferable embodiment of the above solution, the processing of performing the correction by the space compression module and the channel compression module of the second model in step S1 and step S2 is:
suppose the input pneumothorax CT image is X ∈ RH×W×CAssume that the basic feature obtained in the encoding process or the feature map obtained in the decoding process (collectively referred to as the feature map for the sake of description of the aspect) is U e RH×W×C'H and W represent the height and width of the space of the image, C and C 'represent the number of input and output channels, after convolution and nonlinear transformation operation, U contains the information of the space and channel in X, after each encoding or decoding process, a second model is introduced to modify U into U',
for the spatial compression module, assume the input feature map U ═ U1, U2, … UC]Wherein u isi∈RH×WRepresenting the feature map of each channel, obtaining the vector z ∈ R through the operation of Global Average Pooling (GAP)1×1×CFor the kth channel profile, the transformation is as follows:
Figure BDA0003567765480000041
the transformation introduces global spatial feature information into z, and then passes z ═ W1(δ(W2z)) to give z', wherein
Figure BDA0003567765480000042
For two fully-connected networks, delta (·) is a ReLU operation, and then a normalized correction coefficient sigma (z'). epsilon. [0,1 ] is obtained through sigmod transformation]Finally, the correction coefficient is multiplied by the input feature map U to obtain U'cSE，
U′cSE＝[σ(z′1)u1,σ(z′2)u2,…σ(z′C)uC]；
(by the space compression module, information in the unimportant channels is reduced and suppressed, while information in the important channels is kept almost unchanged, so that the phase change is enhanced;
the index for measuring the importance of the channel can be obtained by compressing the information of the channel, and the index for measuring the importance of the space position can be obtained by compressing the information of the channel
For the channel compression module, assume the input signature U ═ U1, U2, … UC]Converting to obtain U ═ U1,1,u1 ,1…ui,j…uH,W]Wherein u isi,j∈R1×1×CRepresents the corresponding spatial position (i, j), by q ═ WsqThe U convolution operation realizes the space extrusion operation, and then the normalized correction coefficient sigma (q) is obtained by sigma-mod transformation, wherein the sigma (q) belongs to [0,1 ]]Finally, the correction coefficient is multiplied by the input feature map U to obtain U'sSE，
U′sSE＝[σ(q1,1)u1,1,σ(q1,2)u1,1,…σ(qh,w)uH,W]；
Finally, the fusion information of the two is obtained by addition, and the importance of the space information and the channel information is weighted to obtain the output U'sCSE＝U′cSE+U′sSE。
In order to achieve the above object, the present invention further provides a multitask pneumothorax medical image semantic segmentation model system based on the uet, which includes a memory, a processor and a computer program stored in the memory and operable on the processor, wherein the processor implements the steps of the multitask pneumothorax medical image semantic segmentation model method based on the uet when executing the computer program.
Drawings
FIG. 1 is a schematic diagram of a multitask pneumothorax medical image semantic segmentation model system based on Unet and a method thereof.
FIG. 2 is a block diagram of a second model of the Unet-based multitask pneumothorax medical image semantic segmentation model system and method thereof of the present invention.
FIG. 3 is a diagram of a multitask learning structure of the multitask pneumothorax medical image semantic segmentation model system and method thereof based on the Unet of the present invention.
Detailed Description
The following description is provided to disclose the invention so as to enable any person skilled in the art to practice the invention. The preferred embodiments in the following description are given by way of example only, and other obvious variations will occur to those skilled in the art. The basic principles of the invention, as defined in the following description, may be applied to other embodiments, variations, modifications, equivalents, and other technical solutions without departing from the spirit and scope of the invention.
In a preferred embodiment of the present invention, those skilled in the art should note that the pneumothorax CT image and the like to which the present invention relates may be regarded as prior art.
Preferred embodiments.
The invention discloses a multitask pneumothorax medical image semantic segmentation model method based on Unet, which is used for performing semantic segmentation on a pneumothorax medical image and comprises the following steps of:
step S1: acquiring a pneumothorax CT image and carrying out coding processing on the pneumothorax CT image through a first model (a Unet model with a frame of Resnet34 and pre-training parameters) and a second model (an SCSE module structure), thereby acquiring an image second classification;
step S2: during decoding processing, fusing characteristic information of each layer in the encoding processing stage, performing up-sampling, and combining a second model to continuously fuse the information of each layer subjected to up-sampling so as to obtain a semantic segmentation image;
step S3: and fusing the loss function of the image secondary classification and the loss function of the semantic segmentation image, learning the loss function by adopting a multi-task strategy of classification and segmentation fusion, fusing the local pixel points and the global pixel points in the classification, and obtaining the global loss function after adjusting the weight relation so as to realize the semantic segmentation of the pneumothorax CT image.
Specifically, step S1 is implemented as the following steps:
step S1.1: the first model is used as a basic feature extraction network of the pneumothorax CT image and carries out basic feature extraction by a method of adding a residual error network in the network;
step S1.2: the second model corrects the extracted basic features through a space compression module and a channel compression module, so that correction data of each layer corresponding to the first model are obtained;
step S1.3: and carrying out coding processing on the pneumothorax CT image through the first model and the second model so as to obtain a second classification of the image.
More specifically, step S2 is specifically implemented as the following steps:
step S2.1: a first module (64, 32, 32) in the encoding process upsamples (Bn1+ Relu + Maxpool) a module in which a largest pooling process in the encoding process is located to obtain a first feature map and performs a first modification process on the first feature map through a second model to obtain first modified data;
step S2.2: a second module (64, 64, 64) in the coding process up-samples the first correction data obtained by the first module and fuses correction data corresponding to a module in which the fifth layer of the first model in the coding process is located to obtain a second feature map and performs a second correction process on the second feature map through the second model to obtain second correction data;
step S2.3: a third module (64, 128, 128) in the coding process performs up-sampling on the second correction data obtained by the second module and fuses correction data corresponding to a module where a fourth layer of the first model in the coding process is located to obtain a third feature map and performs a third correction process on the third feature map through the second model to obtain third correction data;
step S2.4: a fourth module (64, 256, 256) in the encoding process up-samples the third correction data obtained by the third module and fuses the correction data corresponding to the module in which the third layer of the first model in the encoding process is located to obtain a fourth feature map and performs a fourth correction process on the fourth feature map through the second model to obtain fourth correction data;
step S2.5: a fifth module (64, 512, 512) in the coding process up-samples the fourth correction data obtained by the fourth module and fuses the correction data corresponding to the module where the second layer of the first model in the coding process is located to obtain a fifth feature map and performs a fifth correction process on the fifth feature map through the second model to obtain fifth correction data;
step S2.6: and fusing the corresponding correction data obtained by each module in the coding process so as to complete the coding process and further obtain the semantic segmentation image.
Further, the processing of performing the correction by the space compression module and the channel compression module of the second model in step S1 and step S2 is:
suppose the input pneumothorax CT image is X ∈ RH×W×CAssume that the basic feature obtained in the encoding process or the feature map obtained in the decoding process (collectively referred to as the feature map for the sake of description of the aspect) is U e RH×W×C'H and W represent the height and width of the space of the image, C and C 'represent the number of input and output channels, after convolution and nonlinear transformation operation, U contains the information of the space and channel in X, after each encoding or decoding process, a second model is introduced to modify U into U',
for the spatial compression module, assume that the input eigenmap U ═ U1, U2, … UC]Wherein u isi∈RH×WRepresenting the feature map of each channel, obtaining the vector z ∈ R through the operation of Global Average Pooling (GAP)1×1×CFor the kth channel profile, the transformation is as follows:
Figure BDA0003567765480000081
the transformation introduces global spatial feature information into z, and then passes z ═ W1(δ(W2z)) to give z', wherein
Figure BDA0003567765480000082
For two fully connected networks, δ (-) is the ReLU operation, then through sigmod transformation to obtain normalized correction coefficient sigma (z') ∈ [0,1 ]]Finally, the correction coefficient is multiplied by the input feature map U to obtain U'cSE，
U′cSE＝[σ(z′1)u1,σ(z′2)u2,…σ(z′C)uC]；
(by the space compression module, information in the unimportant channels is reduced and suppressed, while information in the important channels remains almost unchanged, so that the phasing is enhanced;
the index for measuring the importance of the channel can be obtained by compressing the information of the channel, and the index for measuring the importance of the space position can be obtained by compressing the information of the channel
For the channel compression module, assume the input signature U ═ U1, U2, … UC]Converting to obtain U ═ U1,1,u1 ,1…ui,j…uH,W]Wherein u isi,j∈R1×1×CRepresents the corresponding spatial position (i, j), by q ═ WsqU convolution operation is used for achieving space extrusion operation, then the normalized correction coefficient sigma (q) belongs to [0,1 ] is obtained through sigmod transformation, and finally the correction coefficient is multiplied by the input feature graph U to obtain U'sSE，
U′sSE＝[σ(q1,1)u1,1,σ(q1,2)u1,1,…σ(qh,w)uH,W]；
Finally, obtaining the fusion information of the two through addition, and carrying out weight processing on the importance of the space information and the channel information, U'sCSE＝U′cSE+U′sSE。
The invention also discloses a multitask pneumothorax medical image semantic segmentation model system based on Unet, which comprises a memory, a processor and a computer program which is stored in the memory and can run on the processor, wherein the processor realizes the steps of the multitask pneumothorax medical image semantic segmentation model method based on Unet when executing the computer program.
Preferably, as for the first problem in the background art, the reason is that the Unet is a semantic segmentation model for point-to-point pixel levels, and whether a lesion exists or not cannot be judged from a global perspective, so that for a small lesion, a fine prediction result cannot be obtained by simply using the Unet network for segmentation. For the second problem, the traditional solution is to increase the volume of the network, but the number of neurons is increased to design a deeper network structure, which will result in occupying very large hardware resources and also make training very difficult.
Preferably, as shown in fig. 1, the invention discloses an end-to-end semantic segmentation model for pneumothorax CT images, which adopts a coding and decoding network structure, wherein a basic feature extraction network in a coding stage is an important part when a two-class network and a semantic segmentation network are designed, and a pneumathorax CT image with a large amount of difficulty is obtained, the invention adopts a pnet model with a pretraining parameter and using Resnet34 as a frame as a feature extraction network because a method of adding a residual error network in the network solves the problems that the depth of the network reaches a certain degree, the error is increased, the effect is deteriorated, the gradient disappearance phenomenon is more obvious, and the minimum loss is difficult to realize by network back propagation. In the encoding stage, an SCSE module structure is introduced to obtain image two-classification loss, each layer of feature information in the encoding stage is fused during decoding, up-sampling is carried out, each layer of feature information which is up-sampled is fused again to obtain a semantic segmentation image, loss functions of image two-classification and image semantic segmentation are fused, the weight relation is adjusted, and a global loss function is obtained and optimized.
In fig. 1, Conv represents convolution, Bn1 represents batch normalization, Relu represents activation function, Resnet _ L1 represents Resnet34 first layer, SCSE represents Spatial-Channel sequence & Excitation, and upsampling represents upsampling.
Preferably, as shown in fig. 3, when performing semantic segmentation on a pneumothorax CT image, in order to reduce noise interference, when the summation of pixel points of the segmented image is less than a certain number, we predict that the image is free of lesion features, so that it is equivalent to performing two classification operations by using a semantic segmentation algorithm, but the semantic segmentation model and the loss function only consider the classification of a single pixel point, and do not consider the overall classification of the summation of all pixel points of the whole image[8-9]Therefore, in the optimization process, only local optimization is performed, and overall optimization is not considered, for this problem, a multi-task learning strategy is introduced, under the condition that the whole network structure is guaranteed to be unchanged, a plurality of tasks of 'classification' and 'segmentation' are set, for two classifications, a logistic loss function loss1 is introduced, for segmentation tasks, a cross entropy loss function loss2 is introduced, and finally, a fused loss function loss1+ b loss2 is obtained. Wherein a and b are weighted values of the two.
Preferably, the invention adopts a pyrrch frame for design when designing related experiments, and a graphic workstation used for the experiments is configured as follows: 4-core CPU, memory 16G, GeForceGTX1080Ti GPU, video memory 16G, operating system Ubuntu 16.04, and the network model configuration is as follows: the optimizer adopts random gradient descent, momentum is 0.9, weight _ decade is 0.0001, Learningrate is 0.002, class weight a is 0.1, semantic segmentation loss weight b is 0.9, as a single picture is 512, the picture is large, and the batch processing size is set to 4, in order to increase the batch processing size, the invention adopts gradient accumulation to realize 'display memory expansion', 1 batch processing data is obtained each time, the gradient is calculated by a gradient descent method, the gradient is not cleared and is continuously accumulated, after a certain number of times of accumulation, network parameters are updated according to the accumulated gradient, then the gradient is cleared, and the next cycle is carried out. Under certain conditions, the larger the batch size is, the better the training effect is, the gradient accumulation realizes the phase change expansion of batch processing amount, and the accumulation times are set to be 4 times, namely, the batch processing size is set to be 16.
Preferably, when the model designed by the invention is used for predicting pneumothorax CT images, the score of the Dice similarity coefficient reaches 85.3%, taking 3 patients as an example for prediction.
The results of comparative analysis of different network models can be known, the Resnet34 is used as a coding frame, the SCSE module is introduced to improve the prediction accuracy by about 4% compared with a single Unet network, and the prediction accuracy by adopting the binary classification and semantic segmentation double tasks is improved by 1.2% compared with the prediction accuracy by adopting a single task.
Preferably, the deep learning model is a very powerful tool in the field of medical image detection, the semantic segmentation is used for predicting the focus position of a pneumothorax CT image to play a good auxiliary role in judging diseases for doctors, the semantic segmentation model designed by the invention adopts the resnet34 as a frame, the problem that the gradient disappears when the network depth reaches a certain degree is solved, the SCSE module is introduced, the information of the image is obtained from two directions of space and channel, the semantic segmentation precision is improved, the loss function adopts a classification and segmentation multi-task strategy for learning, the classification of local pixel points and the global image is fused, and the semantic segmentation precision is further improved.
It should be noted that the technical features such as the pneumothorax CT image and the like related to the patent application of the present invention should be regarded as the prior art, and the specific structure, the operation principle, the control mode and the spatial arrangement mode of the technical features may be selected conventionally in the field, and should not be regarded as the invention point of the patent application, and the patent application is not further specifically described in detail.
It will be apparent to those skilled in the art that modifications and equivalents can be made to the embodiments described above, or some features of the embodiments described above, and any modifications, equivalents, improvements, and the like, which fall within the spirit and principle of the present invention, are intended to be included within the scope of the present invention.

Claims (5)

1. A multitask pneumothorax medical image semantic segmentation model method based on Unet is used for performing semantic segmentation on pneumothorax medical images and is characterized by comprising the following steps:
step S1: acquiring a pneumothorax CT image and coding the pneumothorax CT image through a first model and a second model so as to obtain a second classification of the image;
step S2: during decoding processing, fusing characteristic information of each layer in the encoding processing stage, performing up-sampling, and combining a second model to continuously fuse the information of each layer subjected to up-sampling so as to obtain a semantic segmentation image;
step S3: and fusing the loss function of the image secondary classification and the loss function of the semantic segmentation image, learning the loss function by adopting a multi-task strategy of classification and segmentation fusion, fusing the local pixel points and the global pixel points in the classification, and obtaining the global loss function after adjusting the weight relation so as to realize the semantic segmentation of the pneumothorax CT image.
2. The Unet-based multitask pneumothorax medical image semantic segmentation model method according to claim 1, characterized in that step S1 is embodied as the following steps:
step S1.1: the first model is used as a basic feature extraction network of the pneumothorax CT image and carries out basic feature extraction by a method of adding a residual error network in the network;
step S1.2: the second model corrects the extracted basic features through a space compression module and a channel compression module, so that correction data of each layer corresponding to the first model are obtained;
step S1.3: and carrying out encoding processing on the pneumothorax CT image through the first model and the second model so as to obtain an image second classification.
3. The Unet-based multitask pneumothorax medical image semantic segmentation model method according to claim 1, characterized in that step S2 is embodied as the following steps:
step S2.1: a first module in the coding process performs upsampling on a module where the largest pooling process in the coding process is located to obtain a first feature map and performs a first correction process on the first feature map through a second model to obtain first correction data;
step S2.2: a second module in the coding process performs up-sampling on the first correction data obtained by the first module and fuses correction data corresponding to a module where a fifth layer of the first model in the coding process is located to obtain a second feature map, and performs second correction processing on the second feature map through the second model to obtain second correction data;
step S2.3: a third module in the coding process performs up-sampling on second correction data obtained by the second module and fuses correction data corresponding to a module where a fourth layer of the first model in the coding process is located to obtain a third feature map, and performs third correction processing on the third feature map through the second model to obtain third correction data;
step S2.4: a fourth module in the encoding process performs up-sampling on the third correction data obtained by the third module and fuses correction data corresponding to a module where the third layer of the first model in the encoding process is located to obtain a fourth feature map, and performs a fourth correction process on the fourth feature map through the second model to obtain fourth correction data;
step S2.5: a fifth module in the encoding process performs up-sampling on fourth correction data obtained by the fourth module and fuses correction data corresponding to a module where the second layer of the first model in the encoding process is located, so as to obtain a fifth feature map, and performs fifth correction processing on the fifth feature map through the second model, so as to obtain fifth correction data;
step S2.6: and fusing the corresponding correction data obtained by each module in the encoding process so as to complete the encoding process and further obtain the semantic segmentation image.
4. The method for semantic segmentation of multitask pneumothorax medical image based on Unet claimed in claim 3, wherein the modification process by the space compression module and the channel compression module of the second model in steps S1 and S2 is:
suppose the input pneumothorax CT image is X ∈ RH×W×CAssume that the basic feature obtained in the encoding process or the feature map obtained in the decoding process is U ∈ RH×W×C'H and W represent the height and width of the space of the image, C and C 'represent the number of input and output channels, after convolution and nonlinear transformation operation, U contains the information of the space and channel in X, after each encoding or decoding process, a second model is introduced to modify U into U',
for the spatial compression module, assume the input feature map U ═ U1, U2, … UC]Wherein u isi∈RH×WThe characteristic diagram representing each channel is subjected to global average pooling operation to obtain a vector z epsilon R1×1×CFor the kth channel profile, the transformation is as follows:
Figure FDA0003567765470000031
the transformation introduces global spatial feature information into z, and then passes z ═ W1(δ(W2z)) to give z', wherein
Figure FDA0003567765470000032
For two fully-connected networks, delta (·) is a ReLU operation, and then a normalized correction coefficient sigma (z') ∈ [0,1 ] is obtained through sigmod transformation]Finally, the correction coefficient is multiplied by the input feature map U to obtain U'cSE，
U′cSE＝[σ(z′1)u1,σ(z'2)u2,…σ(z'C)uC]；
For the channel compression module, assume the input signature U ═ U1, U2, … UC]Converting to obtain U ═ U1,1,u1,1…ui,j…uH,W]Wherein u isi,j∈R1×1×CRepresents the corresponding spatial position (i, j), by q ═ WsqThe U convolution operation realizes the space extrusion operation, then the normalized correction coefficient sigma (q) is obtained from [0,1 ] through the sigmod transformation, and finally the correction coefficient is multiplied by the input characteristic diagramU to obtain U'sSE，
U′sSE＝[σ(q1,1)u1,1,σ(q1,2)u1,1,…σ(qh,w)uH,W]；
Finally, obtaining the fusion information of the two through addition, and carrying out weight processing on the importance of the space information and the channel information, U'sCSE＝U′cSE+U′sSE。
5. A Unet-based multitask pneumothorax medical image semantic segmentation model system, comprising a memory, a processor and a computer program stored in said memory and operable on said processor, wherein said computer program when executed by said processor implements the steps of the Unet-based multitask pneumothorax medical image semantic segmentation model method according to any one of claims 1-4.
CN202210312808.8A 2022-03-28 2022-03-28 Multitask pneumothorax medical image semantic segmentation model system and method based on Unet Pending CN114743000A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210312808.8A CN114743000A (en) 2022-03-28 2022-03-28 Multitask pneumothorax medical image semantic segmentation model system and method based on Unet

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210312808.8A CN114743000A (en) 2022-03-28 2022-03-28 Multitask pneumothorax medical image semantic segmentation model system and method based on Unet

Publications (1)

Publication Number Publication Date
CN114743000A true CN114743000A (en) 2022-07-12

Family

ID=82277610

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210312808.8A Pending CN114743000A (en) 2022-03-28 2022-03-28 Multitask pneumothorax medical image semantic segmentation model system and method based on Unet

Country Status (1)

Country Link
CN (1) CN114743000A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116109663A (en) * 2023-04-04 2023-05-12 山东大学第二医院 Stomach CT image segmentation method based on multi-threshold segmentation

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116109663A (en) * 2023-04-04 2023-05-12 山东大学第二医院 Stomach CT image segmentation method based on multi-threshold segmentation
CN116109663B (en) * 2023-04-04 2023-06-23 山东大学第二医院 Stomach CT image segmentation method based on multi-threshold segmentation

Similar Documents

Publication Publication Date Title
CN110189334B (en) Medical image segmentation method of residual error type full convolution neural network based on attention mechanism
CN113673307B (en) Lightweight video action recognition method
JP7417640B2 (en) Real-time video ultra-high resolution
CN110992270A (en) Multi-scale residual attention network image super-resolution reconstruction method based on attention
CN110675462B (en) Gray image colorization method based on convolutional neural network
CN112132959A (en) Digital rock core image processing method and device, computer equipment and storage medium
CN112258526A (en) CT (computed tomography) kidney region cascade segmentation method based on dual attention mechanism
CN112837224A (en) Super-resolution image reconstruction method based on convolutional neural network
CN111583285A (en) Liver image semantic segmentation method based on edge attention strategy
CN114663440A (en) Fundus image focus segmentation method based on deep learning
CN115564649B (en) Image super-resolution reconstruction method, device and equipment
CN113870124B (en) Weak supervision-based double-network mutual excitation learning shadow removing method
CN115375711A (en) Image segmentation method of global context attention network based on multi-scale fusion
CN115565043A (en) Method for detecting target by combining multiple characteristic features and target prediction method
CN116071300A (en) Cell nucleus segmentation method based on context feature fusion and related equipment
CN117058307A (en) Method, system, equipment and storage medium for generating heart three-dimensional nuclear magnetic resonance image
CN114943721A (en) Neck ultrasonic image segmentation method based on improved U-Net network
CN115439470A (en) Polyp image segmentation method, computer-readable storage medium, and computer device
CN114743000A (en) Multitask pneumothorax medical image semantic segmentation model system and method based on Unet
CN111986210B (en) Medical image small focus segmentation method
CN110827963A (en) Semantic segmentation method for pathological image and electronic equipment
CN117575915A (en) Image super-resolution reconstruction method, terminal equipment and storage medium
CN116912268A (en) Skin lesion image segmentation method, device, equipment and storage medium
Lai et al. Generative focused feedback residual networks for image steganalysis and hidden information reconstruction
CN111626296A (en) Medical image segmentation system, method and terminal based on deep neural network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination