CN115908463A - 3D coronary artery image segmentation method based on semi-supervised consistency learning - Google Patents
3D coronary artery image segmentation method based on semi-supervised consistency learning Download PDFInfo
- Publication number
- CN115908463A CN115908463A CN202310007319.6A CN202310007319A CN115908463A CN 115908463 A CN115908463 A CN 115908463A CN 202310007319 A CN202310007319 A CN 202310007319A CN 115908463 A CN115908463 A CN 115908463A
- Authority
- CN
- China
- Prior art keywords
- model
- representing
- label
- data
- coronary artery
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a 3D coronary artery image segmentation method based on semi-supervised consistency learning, which comprises the following steps: acquiring original 3D cardiac CT image data, preprocessing the acquired original 3D cardiac CT image data, and expanding and enhancing the data by adopting random rotation, contrast enhancement and random cutting modes; introducing consistency learning, and constructing a 3D coronary artery segmentation model of a two-stage semi-supervised training mode; inputting a 3D heart CT image to be segmented into a 3D coronary artery segmentation model at a first stage for prediction to obtain a pseudo label feature map, and inputting the pseudo label feature map into a 3D coronary artery segmentation model at a second stage to obtain a segmentation result. By adopting the semi-supervised consistency learning method, a large amount of non-label data can be efficiently utilized on the premise of realizing the complete supervision performance, the label data and the non-label data in the proportion of 1:4 are input into the 3D coronary artery segmentation model, and the non-label data are marked by using the label data, so that the training data are effectively increased.
Description
Technical Field
The invention relates to the technical field of medical image segmentation, in particular to a 3D coronary artery image segmentation method based on semi-supervised consistency learning.
Background
Coronary computed tomography angiography is the primary means of diagnosing coronary artery disease. Accurate and robust coronary artery segmentation from 3D cardiac CT images plays a crucial role in clinical diagnosis and therapy. The method is influenced by the image quality, the labeling precision, few effective 3D heart CT image pixels at the tail end of the blood vessel and the interference of vein tissue structures, and has certain challenge on the accurate segmentation of the coronary artery. Common vessel segmentation algorithms are: a segmentation method based on statistics, a segmentation method based on an active contour model, a region-based segmentation method, a boundary-based tracking method, a threshold-based method. The methods have the defects of poor adaptability to different cases, high calculation cost, easy over-segmentation of images and the like. Therefore, there is a problem that the rupture or the loss of the coronary end results in incomplete division structure, and the vein or other tissues are mistaken as the coronary artery.
With the emergence of Convolutional Neural Networks (CNNs), a series of supervised learning methods emerge, and the methods finish end-to-end semantic segmentation of medical images according to labels labeled in advance, so that the overall performance of image segmentation is greatly improved. CNNs have been commonly used in image segmentation tasks. Compared with the traditional method, the convolutional neural network has good characteristic expression capability and does not need to artificially extract image characteristics. Therefore, the method can well deal with the medical image segmentation task with a complex tissue structure, and does not need to carry out excessive preprocessing operation on the image. CNNs generally have five hierarchical structures of an input layer, a convolutional layer, an active layer, a pooling layer, and an output layer. The preprocessing operation on the image generally occurs in an input layer, and the image is subjected to a series of preprocessing operations on the input layer and then is subjected to feature extraction and local perception through convolution operation of a convolutional layer to obtain a feature map. The activation layer is arranged to facilitate the enhancement of network expression capability, generally speaking, the activation layer is a nonlinear mapping of the output result of the convolutional layer, and commonly used activation functions are sigmoid, tanh, relu, leak relu and the like. The Pooling layer is also called an undersampling or downsampling layer, and aims to reduce feature dimension, compress data volume, reduce overfitting and improve the generalization of a model, and the commonly used Pooling layer is maximum Pooling (Max Pooling) and Average Pooling (Average Pooling). The output layer is also called a full connection layer, and is connected with a classifier, such as a softmax classifier and the like.
However, coronary CT images are complex and segmenting blood vessels in 3D medical images still presents a significant challenge. One of the challenges is the presence of false positives, the inability to ensure that the results of the segmentation are spatially connected, and the potential for holes or burrs. Furthermore, the long-distance topology of coronary vessels is complex, and imaging artifacts often exist in CT images, so that the CNN-based segmentation algorithm is prone to lose some segments of the vessels, thereby causing discontinuity of the extracted vessel center line. The supervised deep learning approach of medical imaging also relies on a large amount of annotation data, and manual collection of coronary data is time consuming while requiring expertise, especially in 3D images.
Disclosure of Invention
The invention aims to provide a 3D coronary artery image segmentation method based on semi-supervised consistency learning, aiming at the technical defects in the prior art.
The technical scheme adopted for realizing the purpose of the invention is as follows:
a method of 3D coronary artery image segmentation based on semi-supervised consistency learning, the method comprising:
and 3, inputting the 3D heart CT image to be segmented into the 3D coronary artery segmentation model at the first stage for prediction to obtain a pseudo label feature map, and inputting the pseudo label feature map into the 3D coronary artery segmentation model at the second stage to obtain a segmentation result.
In the above technical solution, the 3D cardiac CT image data includes a training data set and a testing data set, the training data set is composed of N + M training samples, that is, there are N label data and M label-free data, and the label data set is represented as
An unlabeled data set is represented as->
Wherein it is present>
Representing a 3D cardiac CT input image, y i ∈{0,1} H×W×D Representing the true label, H, W and D represent the height, width and thickness, respectively, of the 3D cardiac CT input image; 20% of the labeled data and 80% of the unlabeled data are taken for 3D coronary artery segmentation model training.
In the above technical solution, each stage of the 3D coronary artery segmentation model includes a set of teacher-student models.
In the above technical solution, the expression of the pseudo tag feature map is:
z i ＝f(x i ；θ 1 ,η)
in the formula (I), the compound is shown in the specification,
z i representing spatial features with the same size as the 3D cardiac CT input image, f (-) represents a neural network, x i Representing a 3D cardiac CT input image, theta 1 Represents the weights of the first stage 3 dresinet model, and η represents the regularization and interference performed on the 3D cardiac CT input image.
In the above technical solution, the expression of the segmentation result is:
in the formula (I), the compound is shown in the specification,
representing a spatial feature whose segmentation result has the same size as the 3D cardiac CT input image, f (-) represents a neural network，z i Representing a characteristic diagram of a pseudo tag, theta 2 Representing the weights of the second stage model and ξ represents the noise added to the pseudo-tag.
In the technical scheme, a semi-supervised training process of the 3D coronary artery segmentation model in the first stage adopts a consistency learning mode of a student model and two teacher models, the consistency of the model in a prediction process or an intermediate feature extraction process is enhanced by adding disturbance, the image is randomly expanded in the disturbance process for the same unlabeled data, so that the model obtains two different prediction results in forward propagation, and consistency constraint is added between predictions of the enhanced image; the semi-supervised training process of the 3D coronary artery segmentation model of the second stage adopts a pseudo-label training mode and introduces an attention mechanism.
In the above technical solution, the teacher-student model training total loss function expression is as follows:
l total (.)＝l sup (.)+λl con (.)
in the formula I total (. Phi.) represents the overall loss function of teacher-student model training, l sup (-) represents the loss of supervision between the coronary segmentation prediction map and the true label,/ con And lambda represents a slope weighting coefficient, and is used for controlling the balance between the supervision loss and the unsupervised loss, ensuring that the overall loss function is dominated by the supervision loss at the beginning and avoiding the model from being degraded in the training process, wherein
The specific expression of the slope weighting coefficient is as follows:
in the formula, λ represents a slope weighting coefficient, t represents the current iteration number of the model, and t max Representing the maximum number of iterations of the model.
In the technical scheme, the label data is input into a student model to obtain a coronary artery segmentation prediction graph, and the supervision loss between the coronary artery segmentation prediction graph and a real label is calculated based on Cross Entropy (CE) and DICE coefficient (DICE), wherein the expression of the supervision loss is as follows:
l sup (.)＝0.5×(l CE (y i ,p i )+l DICE (y i ,p i ))
in the formula I sup (-) represents the loss of supervision between the coronary segmentation prediction map and the true label,/ CE (y i ,p i ) Respectively representing the cross entropy loss between the coronary segmentation prediction map and the true label, l DICE (y i ,p i ) Respectively representing the die coefficient loss between the coronary segmentation prediction map and the true label.
In the above technical solution, for unlabelled data, a semi-supervised training strategy of mutual teaching is adopted, and the same unlabelled data is respectively input into a teacher model and a student model to generate two prediction results, where the two prediction results are respectively:
in the formula (I), the compound is shown in the specification,
representing the prediction result generated by inputting the same label-free data into the teacher model, f (-) representing the neural network, x u Representing unlabeled data, theta t Represents teacher model weight,. Sup/or->
Representing the prediction result, theta, produced by inputting the same unlabeled data into the student model s Representing student model powerWeighing;
the teacher model is used for providing pseudo labels for the student model, the student model is used for providing pseudo labels for the teacher model, and the pseudo labels provided by the student model for the teacher model and the pseudo labels provided by the teacher model for the student model have the following specific expressions:
in the formula (I), the compound is shown in the specification,
represents a pseudo label provided by the student model for the teacher model, argmax () represents a function that solves the parameter set for the function, <' > H>
Representing the result of prediction generated by inputting the same unlabeled data into the student model, respectively>
Represents a dummy label provided by the teacher model for the student model>
Representing the predicted result generated by inputting the same piece of unlabeled data into the teacher model respectively.
In the above technical solution, the expression of the unsupervised consistency loss is as follows:
in the formula I con () represents an unsupervised loss of consistency,
representing a two-cross entropy loss of the teacher model's prediction with a pseudo label, based on the predicted value of the teacher model, and based on the predicted value of the pseudo label, based on the predicted value of the teacher model>
Representing the two-cross entropy loss of the prediction result and the pseudo label of the student model.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention utilizes the 3D DRESUNet network to extract abundant global features and spatial information in the 3D heart CT image, takes the extracted features as a stage training result, and leads the model to pay more attention to interested information through the segmentation result of the self-attention mechanism constraint model of the second stage and the introduction of the self-attention mechanism, thereby overcoming some limitations in the traditional neural network, such as the performance reduction of the system along with the increase of the input size, low calculation efficiency, the lack of the extraction and the strengthening of the features of the system, and the like.
2. The method for learning the semi-supervised consistency can efficiently utilize a large amount of non-label data on the premise of realizing the complete supervision performance, combines supervised learning and non-supervised learning, uses a small part of labeled examples and a large amount of non-label data, and learns from the model and predicts a new example. The basic process involves using existing label data to label the remaining unlabeled data, effectively helping to augment the training data. And more data disturbance and model disturbance are introduced on the basis of a teacher-student model to construct the consistency of the same input under different disturbance. The two-stage semi-supervised learning is more beneficial to keeping the connectivity of the coronary artery segmentation result and meeting the shape constraint on the basis of learning from the unlabeled data to a large amount of useful information
3. The invention improves the accuracy of coronary artery segmentation to a certain extent, can carry out analysis and processing of different levels on effective information in a complex coronary image, and can better serve the aspect of medical clinic. A large amount of unlabeled data are fully utilized in the segmentation process, the dependence of the model on the labeled data is relieved to a great extent, and the labeling time and labor cost of a doctor on the 3D data are reduced. In a general clinical scenario, medical image acquisition equipment is used to acquire images of diseased parts of patients, and subjective analysis of doctors in a professional field is required. Accurate segmentation of coronary arteries is an important prerequisite for quantitative description of diseases and three-dimensional reconstruction of blood vessels. The method provided by the invention plays a role in assisting the clinical diagnosis and treatment of doctors, researches an intelligent image segmentation method, and has important significance in clinical medical diagnosis.
Drawings
Fig. 1 is a flowchart of a 3D coronary artery image segmentation method based on semi-supervised consistency learning according to the present invention.
Fig. 2 is a schematic structural diagram of 3 dresinet of the present invention.
Detailed Description
The present invention will be described in further detail with reference to specific examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Example 1
A method for 3D coronary artery image segmentation based on semi-supervised consistency learning, see fig. 1, the method comprising:
The pretreatment comprises the following steps: the size of the acquired original 3D heart CT image is uniformly adjusted to 256 × 128, all the scanning data, namely a heart area, are used as the center to be cut out, lung noise is extracted, meanwhile, the Hu value of the 3D heart CT image is limited to be between [ -90,410], and each image is normalized to be zero mean and unit variance.
The 3D cardiac CT image data comprises a training data set and a test data set, the training data set consisting of N + M training samples, i.e. N labeled data and M unlabeled data, wherein the labeled data set is represented as
Unlabeled data set representation as->
Wherein it is present>
Representing a 3D cardiac CT input image, y i ∈{0,1} H×W×D Representing the true label, H, W and D represent the height, width and thickness, respectively, of the 3D cardiac CT input image; wherein, 20% of the labeled data and 80% of the unlabeled data are selected for 3D coronary artery segmentation model training.
Ways of data set enhancement include: in the training stage, data enhancement is carried out in a combined mode, wherein the data enhancement comprises modes of random rotation, contrast enhancement, random cutting and the like; according to the characteristics of consistency learning, carrying out strong enhancement (random mask, random gray scale and random fuzziness) and weak enhancement (random noise) on the same label-free data; the inference phase performs data enhancement by using a center cropping mode, and crops the 3D cardiac CT image to be segmented into 128 × 128 pixels.
And 2, introducing consistency learning, and constructing a 3D coronary artery segmentation model of a two-stage semi-supervised training mode.
The 3D coronary artery segmentation model comprises two stages, wherein the 3D coronary artery segmentation model in the first stage and the 3D coronary artery segmentation model in the second stage both adopt a semi-supervised training mode, and each stage comprises a set of teacher-student models; in the first stage, 3 DRESUNets are selected as a backbone network, wherein the 3 DRESUNets comprise four encoder layers, four decoder layers and a certain number of residual error layers, and consistency learning is performed by adding model disturbance, data disturbance and introducing an additional teacher model; the network of the second stage comprises a convolutional layer and a self-attention layer, and certain noise is added in the training stage.
Wherein, referring to fig. 2, each encoder layer of the 3 dresinet comprises a convolutional layer, a normalization layer, an active layer, and a downsampling layer; the convolution kernel size of the convolution layer is 3 x 3, and the step size is 1; the activation layer adopts a LeakyReLU activation function, and the normalization layer adopts a BatchNorm layer normalization function; the convolution kernel size of the down-sampling layer is 3 x 3, and the step size is 2; after the convolution layer and the down-sampling layer, the number of channels of the feature map is doubled, and the resolution is reduced by half; the decoder layer replaces the down-sampling layer in the encoder with the up-sampling layer, and in order to keep more bottom layer information, a certain number of jump connections are adopted.
And 3, inputting the 3D heart CT image to be segmented into the 3D coronary artery segmentation model at the first stage for prediction to obtain a pseudo label characteristic diagram, and inputting the pseudo label characteristic diagram into the 3D coronary artery segmentation model at the second stage to obtain a segmentation result.
Specifically, a 3D cardiac CT image to be segmented is input into a 3 dresinet network for prediction, and a pseudo label feature map of a first stage is obtained, where the pseudo label feature map specifically includes:
z i ＝f(x i ；θ 1 ,η)
in the formula (I), the compound is shown in the specification,
z i representing spatial features with the same size as the 3D cardiac CT input image, f (-) represents a neural network, x i Representing a 3D cardiac CT input image, theta 1 Represents the weights of the first stage 3 dresinet model, and η represents the regularization and interference performed on the 3D cardiac CT input image.
The input of the second stage is the pseudo label characteristic diagram generated in the first stage, and the pseudo label characteristic diagram is input into the network of the stage to obtain a segmentation result, wherein the specific expression is as follows:
in the formula (I), the compound is shown in the specification,
representing the spatial feature of the segmentation result having the same size as the 3D cardiac CT input image, f (-) representing a neural network, z i Representing a characteristic diagram of the pseudo-tag, [ theta ] 2 Representing the weights of the second stage model and ξ represents the noise added to the pseudo-tag.
The first stage adopts a consistency learning mode of a student model and two teacher models, consistency of the consistency learning in a prediction process or an intermediate feature extraction process is enhanced by adding disturbance, the same unlabeled data is subjected to random image expansion in the disturbance process, so that the model obtains two different prediction results in forward propagation, and consistency constraint is added between predictions of the enhanced images, so that the model focuses more on a low-density area, and the segmentation performance is improved; in order to guarantee the connectivity constraint of the coronary artery, the second stage adopts a pseudo-label training mode and introduces an attention mechanism.
Specifically, in the semi-supervised training process, the parameters of the student model are updated through back propagation, and the parameters of the teacher model are the exponential weighted average of the student network parameters; in order to ensure that the teacher model and the student model can learn the characteristics of the label data in the consistent learning process, supervision constraint is built on the label data. Inputting the label data into a student model for training to obtain a coronary artery segmentation prediction graph, and calculating supervision loss between the coronary artery segmentation prediction graph and a real label based on Cross Entropy (CE) and DICE coefficient (DICE), wherein the expression of the supervision loss is as follows:
l sup (.)＝0.5×(l CE (y i ,p i )+l DICE (y i ,p i ))
in the formula I sup (-) represents the loss of supervision between the coronary segmentation prediction map and the true label,/ CE (y i ,p i ) Respectively representing the cross entropy loss between the coronary segmentation prediction map and the true label, l DICE (y i ,p i ) Respectively representing the die coefficient loss, y, between the coronary segmentation prediction map and the true label i Representing a genuine label, p i Representing a coronary artery segmentation prediction map.
For the non-label data, adopting a mutual teaching semi-supervised training strategy, respectively inputting the same piece of non-label data into a teacher model and a student model, and generating two prediction results, wherein the two prediction results are respectively as follows:
in the formula (I), the compound is shown in the specification,
representing the predicted result of the same label-free data input to the teacher model, f (-) represents the neural network, x u Representing unlabeled data, theta t Represents teacher model weight,. Sup/or->
Representing the prediction result, theta, produced by inputting the same unlabeled data into the student model s Representing student model weights.
In the semi-supervised training process, the pseudo label is a mode advocating generation of an artificial label for unlabeled data by using a model per se; for the non-label data, the teacher model is used for providing pseudo labels for the student model, the student model is used for providing pseudo labels for the teacher model, and the pseudo labels provided by the student model for the teacher model and the pseudo labels provided by the teacher model for the student model are expressed in the following specific expressions:
in the formula (I), the compound is shown in the specification,
represents a pseudo label provided by the student model for the teacher model, argmax () represents a function that solves the parameter set for the function, <' > H>
Representing the result of prediction generated by inputting the same unlabeled data into the student model, respectively>
A pseudo label provided for the student model on behalf of the teacher model, in combination>
Representing the predicted results of the same unlabeled data input to the teacher model, respectively.
In the semi-supervised training process, the confidence degree output after the model passes softmax represents the confidence degree of the model on the corresponding pseudo label. The higher the confidence, the higher the potential accuracy of this pseudo-tag. The method comprises the following steps of averaging prediction results of two teacher models in the same 3D cardiac CT image, taking final confidence as weight, applying punishment to output of student models, giving up a 3D cardiac CT image pixel label with low confidence, and improving overall performance by adopting Binary Cross Entropy (BCE), so that unsupervised consistency loss can be determined, wherein the expression of the unsupervised consistency loss is as follows:
in the formula I con (. -) represents an unsupervised loss of consistency,
two-cross entropy loss, representing prediction result of teacher model and false label>
Representing the two-cross entropy loss of the prediction result and the pseudo label of the student model.
Wherein, the overall performance is improved by adopting the Binary Cross Entropy (BCE), and the noise in the training process can be reduced.
Therefore, the overall loss function expression of the teacher-student model training is as follows:
l total (.)＝l sup (.)+λl con (.)
in the formula I total (-) represents the overall loss function of the teacher-student model training,/ sup (-) represents the loss of supervision between the coronary segmentation prediction map and the true label,/ con And lambda represents a slope weighting coefficient for controlling the balance between the supervised loss and the unsupervised loss, ensuring that the overall loss function is initially dominated by the supervised loss and avoiding the model from degrading in the training process, wherein
The specific expression of the slope weighting coefficient is as follows:
in the formula, λ represents a slope weighting coefficient, t represents the current iteration number of the model, and t max Representing the maximum number of iterations of the model.
In the training process, weakly enhancing the unlabeled data and then inputting the weakly enhanced unlabeled data into the teacher model, averagely weighting results obtained by predicting the two teacher models and recording the results as a first pseudo label characteristic diagram, strongly enhancing the unlabeled data and then sending the strongly enhanced unlabeled data into the student model, and recording the predicted results as a second pseudo label characteristic diagram; and recording the prediction result of the labeled data after passing through the student model as a third pseudo label characteristic diagram, wherein the expression of each pseudo label characteristic diagram is as follows:
in the formula (I), the compound is shown in the specification,
represents a first pseudo label feature map, is based on a first label feature map, and is based on a second label feature map>
Represents a second pseudo-tag signature map, based on the comparison of the characteristic map and the presence of a label in the label database>
And &>
For calculating a loss of coherence>
Representing a third pseudo-label feature map for calculating supervision loss, η' representing that the unlabeled data is weakly enhanced, η representing that the unlabeled data is strongly enhanced, f (-) representing a neural network, x u Represents unlabeled data, <' > based on>
Representing a first teacher-model of the teacher,
represents a second teacher model, and>
representing a student model, x l Representing the label image.
In the first stage of the semi-supervised training process, based on the combination of student models, the supervised loss and the unsupervised consistency loss are minimized, and the specific expression is as follows:
in the formula I sup1 Representing a mixture of first-stage cross-entropy (CE) losses and die coefficient (DICE) losses,/ con1 Representing an unsupervised loss of consistency, D L Representing a tag data set, D U Representing unlabeled datasets, f (-) representing a neural network, x i Representing a 3D cardiac CT input image,
representing a student model, y i Represents a true label, <' > based on>
Represents a weighted average of the predictions for the two teacher models, η' represents the data enhancement to different degrees, η represents the interference of unlabeled data, and λ represents the ramp weighting factor.
In the first stage of the semi-supervised training process, based on the combination of the student model minimized supervision loss and unsupervised consistency loss, the specific expression is as follows:
in the formula I sup2 Mixture representing CE loss and DICE loss,/ con2 Representing an unsupervised loss of consistency, D L Representing a tag data set, D U Representing unlabeled datasets, f (-) represents a neural network, z i Representing the pseudo-tag signature generated in the first stage,
representing student model weights, y i Represents a true label, <' > based on>
Representing the weights of the teacher model, xi' and xi represent adding different noise in the pseudo label, and lambda represents the slope weighting coefficient.
Wherein the two stages of the semi-supervised training process are performed sequentially, in the teacher-student model, the weights of the student models are updated by back propagation, and the Exponential Moving Average (EMA) of the weights of the student models is used to update the weights of the teacher model to integrate the different training sessionsAnd (5) refining the information in the step. That is, the weights of the teacher model are updated to: theta t '＝αθ t ' -1 +(1-α)θ t 。
Wherein, theta t ' weight, θ, representing teacher model completed by semi-supervised training step t update t ' -1 Weight, θ, representing the teacher model updated by the semi-supervised training step t-1 t Represents the weight of the teacher model for the semi-supervised training step t, and α represents the rate at which EMA attenuation is controlled.
Further, generating a plurality of coronary artery segmentation prediction images for the same 3D heart CT image, and performing 3D reconstruction on the generated coronary artery segmentation prediction images according to the original positions to obtain reconstructed coronary artery segmentation prediction images; and according to the reconstructed coronary artery segmentation prediction graph, calculating the average accuracy, the average recall rate, the average dice similarity coefficient and the average topological skeleton similarity coefficient of the coronary artery to evaluate the constructed 3D coronary artery segmentation model.
Example 2
On the basis of example 1, a specific 3D cardiac CT image is used for illustration.
Specifically, the size of an acquired original 3D heart CT image is adjusted to 256 × 128, the Hu value of the CT image is limited to [ -90,410], pulmonary vein noise is removed, and each image is normalized to be zero mean and unit variance; and (3) enabling the label data and the non-label data to be in a mode of 1:4, dividing according to the proportion; during semi-supervised training, the CT image is cut into images with the size of 128 by 128, and the training images are enhanced to different degrees; and after the 3D coronary artery segmentation model training is finished, taking the prediction result obtained by the two teacher model training as the final segmentation result.
Wherein tests are performed on publicly available datasets based on orcache and CASDQEF and model segmentation results are evaluated using Precision, recall, and Dice.
Claims (10)
1. A method for 3D coronary artery image segmentation based on semi-supervised consistency learning, the method comprising:
step 1, collecting original 3D cardiac CT image data, preprocessing the collected original 3D cardiac CT image data, and expanding and enhancing the data by adopting random rotation, contrast enhancement and random cutting modes;
step 2, introducing consistency learning, and constructing a 3D coronary artery segmentation model of a two-stage semi-supervised training mode;
and 3, inputting the 3D heart CT image to be segmented into the 3D coronary artery segmentation model at the first stage for prediction to obtain a pseudo label feature map, and inputting the pseudo label feature map into the 3D coronary artery segmentation model at the second stage to obtain a segmentation result.
2. The method of claim 1, wherein the 3D cardiac CT image data comprises a training dataset and a testing dataset, the training dataset consisting of N + M training samples, N labeled data and M unlabeled data, the labeled dataset represented as
Unlabeled data set representation as->
Wherein +>
Representing a 3D cardiac CT input image, y i ∈{0,1} H×W×D Representing the true label, H, W and D represent the height, width and thickness, respectively, of the 3D cardiac CT input image; 20% of the labeled data and 80% of the unlabeled data are taken for 3D coronary artery segmentation model training.
3. The method of claim 1, wherein each stage of the 3D coronary artery segmentation model comprises a set of teacher-student models.
4. The method of claim 1, wherein the pseudo tag signature graph is expressed as:
z i ＝f(x i ；θ 1 ,η)
in the formula (I), the compound is shown in the specification,
z i representing a spatial feature with the same size as the 3D cardiac CT input image, f (-) represents a neural network, x i Representing a 3D cardiac CT input image, θ 1 Represents the weights of the first stage 3D reset model, and η represents the regularization and interference performed on the 3D cardiac CT input image.
5. The method of claim 1, wherein the segmentation result is expressed by:
in the formula (I), the compound is shown in the specification,
representing the spatial feature of the segmentation result having the same size as the 3D cardiac CT input image, f (-) representing a neural network, z i Representing a characteristic diagram of the pseudo-tag, [ theta ] 2 Representing the weights of the second stage model and ξ represents the noise added to the pseudo-tag.
6. The method of claim 3, wherein the semi-supervised training process of the first-stage 3D coronary artery segmentation model adopts a consistent learning mode of one student model and two teacher models, enhances the consistency of the model in the prediction process and the intermediate feature extraction process by adding disturbance, randomly expands the image in the disturbance process for the same unlabeled data so that the model obtains two different prediction results in forward propagation, and adds a consistency constraint between the predictions of the enhanced image; the semi-supervised training process of the 3D coronary artery segmentation model in the second stage adopts a pseudo-label training mode and introduces an attention mechanism.
7. The method of claim 3, wherein the teacher-student model trains the overall loss function as follows:
l total (.)＝l sup (.)+λl con (.)
in the formula I total (-) represents the overall loss function of the teacher-student model training,/ sup (. To.) represents the supervised loss between the coronary segmentation prediction map and the true label, l con And lambda represents a slope weighting coefficient, and is used for controlling the balance between the supervision loss and the unsupervised loss, ensuring that the overall loss function is dominated by the supervision loss at the beginning and avoiding the model from being degraded in the training process, wherein
The specific expression of the slope weighting coefficient is as follows:
in the formula, λ represents a slope weighting coefficient, t represents the current iteration number of the model, and t max Representing the maximum number of iterations of the model.
8. The method of claim 7, wherein the label data is input into a student model to obtain a coronary artery segmentation prediction graph, and a supervised loss between the coronary artery segmentation prediction graph and the true label is calculated based on Cross Entropy (CE) and DICE coefficient (DICE), and is expressed as follows:
l sup (.)＝0.5×(l CE (y i ,p i )+l DICE (y i ,p i ))
in the formula I sup (-) represents the loss of supervision between the coronary segmentation prediction map and the true label,/ CE (y i ,p i ) Respectively representing the cross entropy loss between the coronary segmentation prediction map and the true label, l DICE (y i ,p i ) Respectively representing the die coefficient loss between the coronary segmentation prediction map and the true label.
9. The method of claim 3, wherein for unlabeled data, a semi-supervised training strategy of mutual teaching is adopted, and the same unlabeled data is respectively input into the teacher model and the student model to generate two prediction results, wherein the two prediction results are respectively:
in the formula (I), the compound is shown in the specification,
representing the prediction result generated by inputting the same label-free data into the teacher model, f (-) representing the neural network, x u Representing unlabeled data, theta t Represents teacher model weight, <' > in combination>
Representing the prediction result, theta, produced by inputting the same unlabeled data into the student model s Representing student model weights;
the teacher model is used for providing pseudo labels for the student model, the student model is used for providing pseudo labels for the teacher model, and the pseudo labels provided by the student model for the teacher model and the pseudo labels provided by the teacher model for the student model have the following specific expressions:
in the formula (I), the compound is shown in the specification,
represents a pseudo label provided by the student model for the teacher model, argmax () represents a function that solves the parameter set for the function, <' > H>
Representing the result of prediction generated by inputting the same unlabeled data into the student model, respectively>
Represents a dummy label provided by the teacher model for the student model>
Representing the predicted result generated by inputting the same piece of unlabeled data into the teacher model respectively.
10. The method of claim 7, wherein the unsupervised consistency loss expression is as follows:
in the formula I con () represents an unsupervised loss of consistency,
two-cross entropy loss, representing prediction result of teacher model and false label>
Representing the two-cross entropy loss of the prediction result and the pseudo label of the student model. />
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310007319.6A CN115908463A (en) | 2023-01-04 | 2023-01-04 | 3D coronary artery image segmentation method based on semi-supervised consistency learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310007319.6A CN115908463A (en) | 2023-01-04 | 2023-01-04 | 3D coronary artery image segmentation method based on semi-supervised consistency learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115908463A true CN115908463A (en) | 2023-04-04 |
Family
ID=86491491
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310007319.6A Pending CN115908463A (en) | 2023-01-04 | 2023-01-04 | 3D coronary artery image segmentation method based on semi-supervised consistency learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115908463A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116188481A (en) * | 2023-04-27 | 2023-05-30 | 深圳市大数据研究院 | Image segmentation model training method, image processing method, system and electronic equipment |
| CN116402838A (en) * | 2023-06-08 | 2023-07-07 | 吉林大学 | Semi-supervised image segmentation method and system for intracranial hemorrhage |
| CN116758096A (en) * | 2023-07-03 | 2023-09-15 | 强联智创（北京）科技有限公司 | Aneurysm segmentation method, electronic device, and storage medium |
| CN117934522A (en) * | 2024-03-25 | 2024-04-26 | 江西师范大学 | Two-stage coronary artery image segmentation method, system and computer equipment |
-
2023
- 2023-01-04 CN CN202310007319.6A patent/CN115908463A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116188481A (en) * | 2023-04-27 | 2023-05-30 | 深圳市大数据研究院 | Image segmentation model training method, image processing method, system and electronic equipment |
| CN116188481B (en) * | 2023-04-27 | 2023-10-20 | 深圳市大数据研究院 | Image segmentation model training method, image processing method, system and electronic equipment |
| CN116402838A (en) * | 2023-06-08 | 2023-07-07 | 吉林大学 | Semi-supervised image segmentation method and system for intracranial hemorrhage |
| CN116402838B (en) * | 2023-06-08 | 2023-09-15 | 吉林大学 | Semi-supervised image segmentation method and system for intracranial hemorrhage |
| CN116758096A (en) * | 2023-07-03 | 2023-09-15 | 强联智创（北京）科技有限公司 | Aneurysm segmentation method, electronic device, and storage medium |
| CN117934522A (en) * | 2024-03-25 | 2024-04-26 | 江西师范大学 | Two-stage coronary artery image segmentation method, system and computer equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wong et al. | Deep learning-based cardiovascular image diagnosis: a promising challenge | |
| CN111192245B (en) | Brain tumor segmentation network and method based on U-Net network | |
| CN115908463A (en) | 3D coronary artery image segmentation method based on semi-supervised consistency learning | |
| CN113040715B (en) | Human brain function network classification method based on convolutional neural network | |
| Ferreira et al. | End-to-end supervised lung lobe segmentation | |
| WO2023071531A1 (en) | Liver ct automatic segmentation method based on deep shape learning | |
| CN112258456B (en) | Three-dimensional image segmentation method based on convolutional neural network supervision | |
| Xiao et al. | Efficient combination of CNN and transformer for dual-teacher uncertainty-guided semi-supervised medical image segmentation | |
| CN112785551A (en) | Coronary artery segmentation method based on deep learning | |
| CN109003280B (en) | Method for segmenting intima in blood vessel by two-channel intravascular ultrasonic image | |
| CN113935976A (en) | Method and system for automatically segmenting blood vessels in internal organs by enhancing CT (computed tomography) image | |
| CN114398979A (en) | Ultrasonic image thyroid nodule classification method based on feature decoupling | |
| CN115908297A (en) | Topology knowledge-based blood vessel segmentation modeling method in medical image | |
| CN114943721A (en) | Neck ultrasonic image segmentation method based on improved U-Net network | |
| CN115984555A (en) | Coronary artery stenosis identification method based on depth self-encoder composition | |
| CN114387282A (en) | Accurate automatic segmentation method and system for medical image organs | |
| CN113421250A (en) | Intelligent fundus disease diagnosis method based on lesion-free image training | |
| CN113408603A (en) | Coronary artery stenosis degree identification method based on multi-classifier fusion | |
| CN114972291B (en) | Medical image structured automatic labeling method and system based on hybrid enhanced intelligence | |
| CN115565671A (en) | Atrial fibrillation auxiliary analysis method based on cross-model mutual teaching semi-supervision | |
| CN115471512A (en) | Medical image segmentation method based on self-supervision contrast learning | |
| CN115170584A (en) | Medical image segmentation method based on multi-scale Transformer | |
| Stephens et al. | MRI to CTA translation for pulmonary artery evaluation using CycleGANs trained with unpaired data | |
| CN113902738A (en) | Heart MRI segmentation method and system | |
| Liu et al. | Neural network for lung cancer diagnosis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |