WO2024244261A1 - 基于掩码建模的3d医学图像分割模型建立方法及其应用 - Google Patents

基于掩码建模的3d医学图像分割模型建立方法及其应用 Download PDF

Info

Publication number
WO2024244261A1
WO2024244261A1 PCT/CN2023/124922 CN2023124922W WO2024244261A1 WO 2024244261 A1 WO2024244261 A1 WO 2024244261A1 CN 2023124922 W CN2023124922 W CN 2023124922W WO 2024244261 A1 WO2024244261 A1 WO 2024244261A1
Authority
WO
WIPO (PCT)
Prior art keywords
segmentation
medical image
network
decoder
student
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/124922
Other languages
English (en)
French (fr)
Inventor
张旭明
周权
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huazhong University of Science and Technology
Original Assignee
Huazhong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huazhong University of Science and Technology filed Critical Huazhong University of Science and Technology
Priority to US18/689,426 priority Critical patent/US20250225776A1/en
Publication of WO2024244261A1 publication Critical patent/WO2024244261A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/0895—Weakly supervised learning, e.g. semi-supervised or self-supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/20—Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G06V10/765—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects using rules for classification or partitioning the feature space
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06V10/7753—Incorporation of unlabelled data, e.g. multiple instance learning [MIL]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/776—Validation; Performance evaluation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/778—Active pattern-learning, e.g. online learning of image or video features
    • G06V10/7784—Active pattern-learning, e.g. online learning of image or video features based on feedback from supervisors
    • G06V10/7792—Active pattern-learning, e.g. online learning of image or video features based on feedback from supervisors the supervisor being an automated module, e.g. "intelligent oracle"
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional [3D] objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • the present invention belongs to the field of medical image segmentation, and more specifically, relates to a method for establishing a 3D medical image segmentation model based on mask modeling and its application.
  • SSL typically uses labeled images to provide traditional supervised losses such as cross entropy and Dice loss, while unlabeled images provide unsupervised losses such as consistency regularization and minimizing entropy. In essence, the unsupervised losses all serve a common goal, which is to absorb additional knowledge from unlabeled images by training two learners and maximizing their decision consistency on the same task under certain perturbations (e.g., perturbations such as images and models).
  • SSL methods typically involve a dual-model architecture, with the two models playing the roles of teacher and student respectively. The student model is trained normally, and its weights are merged into the teacher model through a synchronization mechanism (e.g., exponential moving average) at each update.
  • the predictions of the teacher model are relatively stable and reliable, so they can be used as pseudo-labels for the student model, that is, if there are no true labels, the student's predictions can be kept consistent with the teacher.
  • the present invention provides a method for establishing a 3D medical image segmentation model based on mask modeling and its application, the purpose of which is to allow the student network and the teacher network in the semi-supervised network to input different three-dimensional images with missing partial information through a random masking method, thereby improving the diversity of tasks, so that the two networks can robustly learn related and complementary features, so that the consistency constraints at the feature level can produce effective unsupervised guidance in the entire training process, and ultimately improve the robustness, generalization and accuracy of the three-dimensional medical image segmentation model.
  • a method for establishing a 3D medical image segmentation model based on mask modeling comprising:
  • (S1) Establish a semi-supervised learning network including a student network and a teacher network;
  • the student network includes: a first encoding module and a decoding module; the first encoding module includes an encoder for extracting different scale features of the three-dimensional input image to obtain latent features; the decoding module includes a first segmentation decoder, and the first segmentation decoder is used to extract and upsample the latent features to obtain segmentation results;
  • the teacher network includes: a second encoding module having a structure consistent with the first encoding module, and a second segmentation decoder having a structure consistent with the first segmentation decoder;
  • the semi-supervised learning network is trained in the following way: freeze the weight of the teacher network, perform two random mask operations on each image, and then input them into the student network and the teacher network respectively, optimize and update the weight of the student network according to the preset training loss function, and transfer the updated weight to the teacher network;
  • the training loss function includes the prototype representation loss Lp1 , which is used to characterize the difference between the features of the corresponding area of the segmentation target in the potential features Vs and Vt extracted by the student network and the teacher network;
  • the feature p fg corresponding to the segmentation target area in its potential features is calculated as follows:
  • V represents the potential feature
  • P represents the segmentation result
  • C represents the number of channels of the potential feature
  • Vj represents the jth channel of V
  • Pj represents the jth channel of P
  • UP() represents the upsampling operation
  • N and M represent the number of annotated images and unannotated images in the 3D medical image segmentation dataset, respectively.
  • the training loss function also includes: potential feature loss L fea ; the potential feature loss L fea is used to characterize the difference between the potential features extracted by the student network and the teacher network, and its expression is:
  • Lmse represents the root mean square error
  • N and M represent the number of labeled images and unlabeled images in the 3D medical image segmentation dataset, respectively;
  • Vis and Vit represent the potential features extracted by the student network and the teacher network after the i-th image Xi in the 3D medical image segmentation dataset is input, respectively.
  • the decoding module further includes K auxiliary segmentation decoders; the auxiliary segmentation decoders are used to extract and upsample the latent features to obtain segmentation results; the upsampling methods of the K auxiliary segmentation decoders are different from each other and are all different from the first segmentation decoder;
  • the training loss function also includes: segmentation consistency loss L mc ; the segmentation consistency constraint is used to characterize the difference between the segmentation results of the K auxiliary segmentation decoders and the first segmentation decoder, and its expression is:
  • K is a positive integer, and Respectively represent the segmentation results predicted by the mth segmentation decoder and the nth segmentation decoder after the image Xi is input, express The result after sharpening processing; the segmentation decoder is an auxiliary segmentation decoder or a first segmentation decoder.
  • the decoding module further includes: a reconstruction decoder; the reconstruction decoder is used to extract and upsample the latent features to restore the original image information and obtain a reconstructed image;
  • the training loss function also includes: reconstruction loss L sup1 ; the reconstruction loss is used to characterize the difference between the reconstructed image obtained by the student network and the original image, and its expression is:
  • the encoder further includes: F HybridFormer modules connected in sequence; in the second encoding module, the encoder further includes: F HybridFormer modules connected in sequence;
  • the HybridFormer module is used to calculate self-attention in pixel space and sample dimension
  • the latent features extracted by the student network are feature images extracted by the encoder in the first encoding module and processed by F HybridFormer modules
  • the latent features extracted by the teacher network are feature images extracted by the encoder in the second encoding module and processed by F HybridFormer modules;
  • F is a positive integer.
  • the training loss function also includes: a segmentation loss L sup2 , which is used to characterize the difference between the segmentation result predicted by the first segmentation decoder and the gold standard, and its expression is:
  • N represents the number of annotated images in the 3D medical image segmentation dataset
  • Yi represents the gold standard of the segmentation image corresponding to the i-th image Xi in the 3D medical image segmentation dataset
  • Pis represents the segmentation result predicted by the first segmentation decoder after the image Xi is input
  • Lseg represents the sum of DICE loss and cross entropy loss.
  • the random mask operation includes:
  • the 3D medical image is divided into equal-sized and non-overlapping cubic blocks, a certain proportion of the cubic blocks are randomly selected, and the pixels in the corresponding area are set to zero.
  • a 3D medical image segmentation method comprising:
  • the 3D medical image to be segmented is input into the 3D medical image segmentation model established by the 3D medical image segmentation model establishment method based on mask modeling provided by the present invention, and the segmentation result is obtained from the output of the 3D medical image segmentation model.
  • a computer-readable storage medium comprising a stored computer program.
  • the device where the computer-readable storage medium is located is controlled to execute the method for establishing a 3D medical image segmentation model based on mask modeling provided by the present invention, and/or the 3D medical image segmentation method provided by the present invention.
  • the present invention When training a 3D medical image segmentation model, the present invention performs a random masking operation on the input original image to generate two different 3D mask images, which are input into the student network and the teacher network respectively, so that the input images of the two models are incomplete, but they jointly contain complementary and global information. Due to the randomness of the masking strategy, the image segmentation tasks of the two networks also have great diversity, so that relevant and complementary features can be robustly learned, so that the consistency constraints at the feature level produce effective unsupervised guidance in the entire training process, thereby achieving the effect of improving robustness, generalization and segmentation accuracy.
  • the training loss function includes the prototype representation loss Lp1 , which can constrain the feature space and prototype representation, thereby obtaining better segmentation performance.
  • the features of the latent space i.e., the latent features extracted by the encoding module
  • the prediction map output by the segmentation decoder are used to extract the prototype representation of its foreground, which can effectively explore the connection between the feature map and the probability map and further improve the training effect of the model.
  • a latent feature loss is designed in the training loss function to characterize the difference between the latent features extracted by the student network and the teacher network.
  • the introduction of this loss can enhance the similarity of the latent features extracted by the student network and the teacher network, thereby effectively realizing information complementarity when the inputs of the two networks use independent masks.
  • an auxiliary segmentation decoder is also introduced into the student network, and consistency constraints are set between the auxiliary segmentation decoders and the original first segmentation decoders, so that these segmentation decoders can learn from each other. Since the upsampling methods used by the auxiliary segmentation decoders are different from each other and different from the original first segmentation decoder, the mutual learning between students is a decoupled student-to-student learning, which enables the first segmentation decoder to receive additional decoupled knowledge and obtain more constructive unsupervised guidance, providing the teacher model with error suspicion monitoring and correction capabilities, and effectively avoiding the impact of prediction uncertainty on network robustness, generalization and accuracy.
  • a reconstruction decoder for restoring the original image information is also introduced into the student network, so that the student network can complete the segmentation task and the reconstruction task at the same time, thereby realizing a joint learning task.
  • the reconstruction loss is correspondingly introduced into the training loss function. Since the two tasks in the joint learning task share the same encoder structure, the introduction of the reconstruction decoder and reconstruction loss will enable the encoder structure to accurately extract potential features even when part of the image information is missing, thereby ensuring the prediction accuracy of the segmentation decoder.
  • one or more HybridFormer modules are introduced after the encoder to calculate the self-attention in the pixel space and sample dimension, thereby further improving the feature representation capability in the latent space.
  • FIG1 is a schematic diagram of a framework of a semi-supervised learning model provided by an embodiment of the present invention.
  • FIG2 is a schematic diagram of a method for establishing a 3D medical image segmentation model based on mask modeling provided by an embodiment of the present invention
  • Figure 3 is a schematic diagram comparing the segmentation results of the 3D medical image segmentation method provided by an embodiment of the present invention and the existing method on the same left atrium dataset; wherein, (a) is the segmentation gold standard, (b) is the segmentation result of the existing MT method, (c) is the segmentation result of the existing UA-MT method, (d) is the segmentation result of the existing SSASNet method, (e) is the segmentation result of the existing DTC method, (f) is the segmentation result of the existing URPC method, (g) is the segmentation result of the existing MCNet+ method, and (h) is the segmentation result of the 3D medical image segmentation method provided by an embodiment of the present invention.
  • the present invention provides a 3D medical image segmentation model establishment method based on mask modeling and its application, the overall idea of which is: by performing a random mask operation on the input original image, two different mask images are generated, which are input into the student network and the teacher network respectively, so that the input images of the two models are incomplete, but they jointly contain the overall information, and due to the randomness of the mask strategy, the image segmentation tasks of the two networks also have great diversity, so that relevant and complementary features can be robustly learned, so that the consistency constraints at the feature level produce effective unsupervised guidance in the entire training process, so as to achieve the effect of improving robustness, generalization and segmentation accuracy.
  • the present invention also proposes two new learning methods, namely, diverse joint task learning (DJL) and decoupled student-to-student learning (DIL), and instantiates them into an enhanced teacher-student architecture for robust 3D medical image segmentation.
  • DJL diverse joint task learning
  • DIL decoupled student-to-student learning
  • the student network will complete the task of restoring the original image information while completing the 3D image segmentation task.
  • the two tasks will share the same encoder structure.
  • the teacher and student models learn to jointly segment the same target but restore different image contents. Due to the randomness of the mask strategy, this joint task also has great diversity.
  • the student model in DIL is also attached with one or more auxiliary decoding branches, which can be equivalently regarded as other "students" by using different upsampling designs.
  • the auxiliary branches learn from the original branches by optimizing the pairwise consistency constraints of the output level, but their weights are independent of the teacher-student synchronization. The students thus receive additional decoupled knowledge and obtain more constructive unsupervised guidance, avoiding errors in the knowledge learned by the original branches that affect the teacher network's learning of correct knowledge, and providing the teacher model with error suspicion monitoring and correction capabilities.
  • Embodiment 1 is a diagrammatic representation of Embodiment 1:
  • a method for establishing a 3D medical image segmentation model based on mask modeling, as shown in FIG1 and FIG2, includes:
  • the student network includes: a first encoding module and a decoding module; the first encoding module includes an encoder for extracting different scale features of a three-dimensional input image to obtain latent features; the decoding module includes a first segmentation decoder, and the first segmentation decoder is used to extract and upsample the latent features to obtain segmentation results;
  • the encoder in the student network is composed of multiple convolutional layers and downsampling layers;
  • the first segmentation decoder is correspondingly composed of multiple convolutional layers and upsampling layers, and the upsampling is specifically implemented by transposed convolution;
  • the encoder in the first encoding module, also includes: F HybridFormer modules connected in sequence, where F is a positive integer; the HybridFormer module is used to calculate the self-attention of the pixel space and sample dimension; and the potential features extracted by the student network are the feature images extracted by the encoder in the first encoding module after being processed by F HybridFormer modules;
  • the calculation of pixel space self-attention and sample dimension self-attention is completed by two parts respectively, and the two parts include a multi-layer convolution layer and a downsampling layer respectively; in practical applications, the number F of HybridFormer modules can be flexibly set according to actual needs;
  • the potential features output by the first encoding module are input into the first segmentation decoder.
  • the low-order output feature map is concatenated with the feature map extracted by the corresponding large-scale convolution kernel of the previous level along the channel direction.
  • the large-scale features are further extracted using 3 ⁇ 3 convolution, and finally the segmentation prediction map is obtained.
  • the teacher network includes: a second encoding module and a second segmentation decoder; a second encoding
  • the structure of the module is consistent with the first encoding module in the student network, including an encoder and F HybridFormer modules after the encoder, which are used to extract the potential features of the input mask image;
  • the structure of the second segmentation decoder is consistent with the structure of the first segmentation decoder in the student network, which is used to extract and upsample the potential features to obtain the segmentation results.
  • HybridFormer module in the student network and the teacher network effectively improves the feature representation capability in the latent space.
  • the decoding module in the student network includes two types of branches, one is a reconstruction decoder and the other is a segmentation decoder.
  • the reconstruction decoder is used to extract and upsample the latent features to restore the original image information and obtain the reconstructed image; the segmentation decoder predicts the segmentation result.
  • the segmentation decoders specifically include three, one of which is the first segmentation decoder and the other two are auxiliary segmentation decoders.
  • the two auxiliary segmentation decoders are also used to extract and upsample the latent features to obtain the segmentation result.
  • the upsampling methods used by the two auxiliary segmentation decoders are different from each other and are different from the upsampling method of the first segmentation decoder, which are linear interpolation and neighboring interpolation respectively; the auxiliary segmentation decoder cooperates with the first segmentation decoder to achieve decoupled student-to-student learning (DIL).
  • DIL student-to-student learning
  • auxiliary segmentation decoders and the upsampling method used here are only exemplary descriptions and should not be understood as the only limitation to the present invention. In practical applications, they can be flexibly adjusted according to actual needs.
  • this embodiment correspondingly proposes a model training method, specifically:
  • the training method is as follows: freeze the weights of the teacher network, perform two random mask operations on each image, and then input them into the student network and the teacher network respectively.
  • the weights of the student network are optimized and updated, and the updated weights are migrated to the teacher network; optionally, in this embodiment, the student network is migrated to the teacher network, which is specifically completed through an exponential moving average (EMA).
  • EMA exponential moving average
  • Xi ⁇ ⁇ H ⁇ W ⁇ D and Yi ⁇ ⁇ 0,1 ⁇ H ⁇ W ⁇ D are used to represent the input image and the segmentation label, where H, W and D represent the height, width and depth of the three-dimensional image.
  • i N+1,...N+M ⁇ , respectively, where N and M represent the number of labeled images and unlabeled images, respectively.
  • X; ⁇ ) is the probability map of the input image, which is used to represent the probability of each pixel belonging to each type of segmentation target.
  • the specific manner of performing the mask operation on the three-dimensional image includes:
  • the 3D medical image is divided into cube blocks of equal size (for example, 4 ⁇ 4 ⁇ 4) and non-overlapping.
  • a certain proportion of the cube blocks are randomly masked by sampling from a uniform distribution, and the pixels in the masked area are set to zero.
  • the two mask images input into the student network and the teacher network are missing some information. Since the random mask operation is random, the mask areas of the two mask images are different, and the features in the latent space will be destroyed by the random mask, which is crucial for the medical image segmentation task. At the same time, the two mask images together contain complete image information.
  • the mask image input to the student network is processed by its encoding module to obtain the potential feature V s , and then predicted by the first segmentation decoder to obtain the segmentation result P s_s_Tconv .
  • the segmentation results P s_s_Tri , P s_s_Nea are obtained by two auxiliary segmentation decoders.
  • the reconstruction decoder restores the original image information, the reconstructed image is obtained.
  • the mask image input to the teacher network is processed by its encoding module to obtain the potential feature V t , which is then predicted by the first segmentation decoder to obtain the segmentation result P s_t .
  • pixel space information is extracted only by forcing the probability map of the student network to the probability map of the teacher network, while ignoring the constraints of the potential feature space and the mining of information in unlabeled data.
  • the mask operation used in this embodiment reduces redundant information and creates a more robust and reliable image than noise or transformation. Challenging feature representation tasks. The same image with different random masks may lead to different predictions, especially when the target area to be segmented is masked. Directly aligning the prediction results of the teacher-student network is too strict and may lead to prediction collapse.
  • the training loss function includes a prototype representation loss Lp1 , which is used to characterize the difference between the features of the corresponding area of the segmentation target in the latent features Vs and Vt extracted by the student network and the teacher network; the prototype, that is, the feature of the corresponding area of the segmentation target in the latent feature, is specifically determined by the latent features extracted by the encoding module and the segmentation result predicted by the segmentation decoder. Therefore, in this embodiment, the design of the training loss function can constrain the feature space and the prototype representation to train the established model to obtain better segmentation performance.
  • this embodiment averages the features along the channel dimension when calculating the prototype. Accordingly, for the student network or the teacher network, the features of the region corresponding to the segmentation target in the potential features are calculated as follows:
  • V represents the potential feature
  • P represents the segmentation result
  • C represents the number of channels of the potential feature
  • Vj represents the jth channel of V
  • Pj represents the jth channel of P
  • UP() represents the upsampling operation
  • Pfg ⁇ [0,1] H ⁇ W ⁇ D represents the foreground prototype.
  • N and M represent the number of annotated images and unannotated images in the 3D medical image segmentation dataset, respectively.
  • the training loss function also includes: potential feature loss Lfea ; the potential feature loss Lfea is used to characterize the difference between the potential features extracted by the student network and the teacher network, and its expression is:
  • Lmse represents the root mean square error
  • N and M represent the number of labeled images and unlabeled images in the 3D medical image segmentation dataset, respectively;
  • Vis and Vit represent the potential features extracted by the student network and the teacher network after the i-th image Xi in the 3D medical image segmentation dataset is input, respectively.
  • this embodiment introduces the auxiliary segmentation decoder, which makes the auxiliary segmentation decoder and the original first segmentation decoder pass consistency constraints, and uses the sharpening function to reduce the influence of pixels that are easily misclassified.
  • Pis represents the predicted segmentation result after the i-th image Xi is input, and the result of the prediction result after the sharpening function is processed is expressed as:
  • T represents a hyperparameter for controlling the degree of sharpening
  • P i s_sharp is the result after being processed by the sharpening function, which will be used as a pseudo-label in the consistency constraint between the segmentation decoders, so that the segmentation result predicted by the segmentation decoder can be close to the result after sharpening processing;
  • the consistency constraint between the segmentation decoders is characterized by the segmentation consistency loss L mc.
  • the training loss function also includes: the segmentation consistency loss L mc ; the segmentation consistency constraint is used to characterize the difference between the segmentation results of the two auxiliary segmentation decoders and the first segmentation decoder, and its expression is:
  • the segmentation decoder is an auxiliary segmentation decoder or a first segmentation decoder; this embodiment can constrain the segmentation results of each segmentation decoder to remain consistent by introducing a segmentation consistency constraint in the training loss function.
  • the joint task includes not only the segmentation task but also the reconstruction task, and the two tasks share the same encoding structure; when the mask image information is accurate, the original, undamaged voxel information can be reconstructed through the reconstruction task, so that the encoder can extract features more effectively.
  • the training loss function also includes: reconstruction loss L sup1 ; the reconstruction loss is used to characterize the difference between the reconstructed image obtained by the student network and the original image, and its expression is:
  • the training loss function also includes: a segmentation loss L sup2 , which is used to characterize the difference between the segmentation result predicted by the first segmentation decoder and the gold standard, and its expression is:
  • N represents the number of annotated images in the 3D medical image segmentation dataset
  • Yi represents the gold standard of the segmented image corresponding to the i-th image Xi in the 3D medical image segmentation dataset
  • Pis represents the segmentation result predicted by the first segmentation decoder after the image Xi is input
  • Lseg represents the DICE loss and the cross The sum of the fork entropy losses.
  • this embodiment can obtain a student network with excellent segmentation performance.
  • this embodiment further includes: (S3) extracting a first encoding module and a first decoder to connect and form a 3D medical image segmentation model.
  • this embodiment ensures the diversity of tasks through a random mask strategy.
  • DJL joint task learning
  • DIL decoupled student-to-student learning
  • the student network and teacher network in the semi-supervised learning network can robustly learn relevant and complementary features, so that the consistency constraints at the feature level can generate effective unsupervised guidance throughout the training process, and the student network can receive decoupled knowledge to obtain more constructive unsupervised guidance, providing the teacher network with error suspicion monitoring and correction capabilities.
  • the 3D medical image segmentation model established in this embodiment has stronger robustness and generalization, as well as higher segmentation accuracy.
  • Embodiment 2 is a diagrammatic representation of Embodiment 1:
  • a 3D medical image segmentation method comprising:
  • the 3D medical image to be segmented is input into the 3D medical image segmentation model established by the 3D medical image segmentation model establishment method based on mask modeling provided in the above-mentioned embodiment 1, and the segmentation result is obtained from the output of the 3D medical image segmentation model.
  • this embodiment can obtain high-precision segmentation results in various three-dimensional medical image segmentation scenarios.
  • Embodiment 3 is a diagrammatic representation of Embodiment 3
  • a computer readable storage medium comprising a stored computer program, the computer program being processed
  • the device where the computer-readable storage medium is located is controlled to execute the 3D medical image segmentation model establishment method based on mask modeling provided in the above-mentioned embodiment 1, and/or the 3D medical image segmentation method provided in the above-mentioned embodiment 2.
  • the following uses the segmentation results on the three-dimensional GE-MRI dataset from the left atrium segmentation challenge to further verify the beneficial effects of the method provided by the present invention.
  • the dataset was acquired using a clinical whole-body MRI scanner with a data resolution of 0.625 ⁇ 0.625 ⁇ 0.625mm 3 , of which 154 scan data had labels given by experts. 123 scan data were randomly selected for training and 31 data samples were tested. Before network training, all scan data were preprocessed, the pixel intensity was standardized to unit variance and zero mean, and the samples were randomly cropped to 112 ⁇ 112 ⁇ 80mm 3 .
  • the segmentation results of the V-Net based on supervised training at 10% and 20% labels are used as the baseline. It can be observed from the table that, relative to the baseline, all semi-supervised methods can provide more effective guidance on the segmentation results at 10% label amount, which reveals a large amount of diverse and useful information for model training contained in the unlabeled data.
  • the supervised V-Net with 100% labeled data outperforms all semi-supervised methods.
  • the quantification results also comprehensively reflect that the segmentation method provided by the present invention has more substantial improvements than other semi-supervised methods.
  • the mainstream semi-supervised methods achieved greater improvements than the corresponding supervised training V-Net on both 20% and 10% labeled data.
  • the method of the present invention outperforms the compared methods on all four quantitative indicators.
  • MC-Net+ is slightly worse than the method of the present invention.
  • the method of the present invention can still provide significantly improved segmentation index results.
  • 2D and 3D views of the segmentation results generated by all comparison methods under 20% labeled data are visualized in Figure 3.
  • the method of the present invention retains more details and sharpens most isolated areas.
  • the method of the present invention does not achieve the best results in terms of the number of parameters and calculations, the difference with other comparison algorithms is not significant, so given the powerful computing power of modern computers, it will not bring too much computational burden.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Computer Graphics (AREA)
  • Architecture (AREA)
  • Computer Hardware Design (AREA)
  • Geometry (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

基于掩码建模的3D医学图像分割模型建立方法及其应用，属于医学图像分割领域，包括：建立包括学生网络和教师网络的半监督学习网络，学生网络包括用于提取潜在特征的编码模块和用于预测分割结果的分割解码器，教师网络包括与学生网络结构一致的编码模块和分割解码器；对半监督学习网络进行训练，训练时，将每个图像进行两次随机掩码操作后分别输入两个网络，根据训练损失函数对学生网络的权重进行优化更新后，迁移至教师网络；训练损失函数包括原型表征损失，用于表征两个网络提取生成的原型之间的差异；学生网络中还可包括重建解码器以及辅助分割解码器。本方法能够在样本稀少的情况下提高三维医学图像分割模型的鲁棒性、泛化性以及准确性。

Description

基于掩码建模的3D医学图像分割模型建立方法及其应用 【技术领域】
本发明属于医学图像分割领域，更具体地，涉及基于掩码建模的3D医学图像分割模型建立方法及其应用。
【背景技术】
从三维医学图像中准确、鲁棒地分割器官或病变在诊断和治疗等临床应用中起着至关重要的作用。与传统的分割算法相比，基于深度学习的方法，如卷积神经网络，由于其具有学习高级语义图像特征的良好能力而被广泛应用于各种视觉任务中。然而，3D医学图像分割方法大多依赖大量标签数据进行全监督训练，人工标注劳动强度大且成本高，对全监督分割方法的性能产生负面影响。相比之下，半监督学习(SSL)方法直接从已标注和未标注图像中学习，达到比单独利用有标注数据更好的分割精度。
SSL通常使用标记图像来提供传统的监督损失，如交叉熵和Dice损失，而未标记图像提供无监督的损失，如一致性正则化和最小化熵。本质上，无监督损失都服务于一个共同的目标，即通过训练两个学习者并在某些扰动(例如，图像、模型等扰动)下最大化他们在同一任务上的决策一致性，从没标签的图像中吸收额外的知识。实际上，SSL方法通常涉及一个双模型架构，让这两个模型分别扮演教师和学生的角色。学生模型经过正常训练，每次更新时其权重通过同步机制(例如指数移动平均)合并到教师模型中。教师模型的预测是相对稳定和可靠的，因此可以用作学生模型的伪标签，即如果没有真正的标签，可让学生的预测与教师保持一致。
虽然SSL方法已经取得了很大的成功，但对于三维医学图像分割来说，存在两个主要的挑战：一方面，即使是未标记的三维医学数据量也非常稀 少，而分割目标遵循清晰的解剖先验知识，这使得以一致性约束为目的任务缺乏多样性，因此教师和学生很快就达成一致，从一致性约束中获得的指导性知识也变少。另一方面，两个模型之间的同步使它们的知识紧密耦合，随着训练的进行，这种耦合必然会变得越来越严格，并且当学生网络学习到知识存在差错时，教师网络无法感知。这使得前面提到的一致性约束存在的问题变得更糟，从而无法为学生模型提供建设性的指导。以上两方面的挑战导致最终基于SSL方法训练得到的三维医学图像分割模型的鲁棒性和泛化性较差，分割准确性也得不到保证。
【发明内容】
针对现有技术的缺陷和改进需求，本发明提供了基于掩码建模的3D医学图像分割模型建立方法及其应用，其目的在于，通过随机掩码的方式使得半监督网络中学生网络和教师网络输入不同的、缺失部分信息的三维图像，提高任务的多样性，从而使两个网络能够鲁棒地学习相关且互补的特征，使得特征级的一致性约束在整个训练过程中产生有效的无监督指导，最终提高三维医学图像分割模型的鲁棒性、泛化性以及准确性。
为实现上述目的，按照本发明的一个方面，提供了一种基于掩码建模的3D医学图像分割模型建立方法，包括：
(S1)建立包括学生网络和教师网络的半监督学习网络；
学生网络包括：第一编码模块和解码模块；第一编码模块包括编码器，用于提取三维输入图像的不同尺度特征，得到潜在特征；解码模块包括第一分割解码器，第一分割解码器用于对潜在特征进行特征提取和上采样，得到分割结果；
教师网络包括：与第一编码模块结构一致的第二编码模块，以及与第一分割解码器结构一致的第二分割解码器；
(S2)利用包括有标注图像和未标注图像的3D医学图像分割数据集对 半监督学习网络进行训练，训练方式如下：冻结教师网络的权重，将每个图像进行两次随机掩码操作后，分别输入至学生网络和教师网络，根据预设的训练损失函数对学生网络的权重进行优化更新，并将更新后的权重迁移至教师网络；训练损失函数包括原型表征损失Lp1，用于表征学生网络和教师网络提取的潜在特征Vs和Vt中分割目标对应区域的特征之间的差异；
(S3)提取第一编码模块和第一解码器连接构成3D医学图像分割模型。
进一步地，对于学生网络或教师网络，其潜在特征中分割目标对应区域的特征pfg的计算方式如下：
其中，V表示潜在特征，P表示分割结果；C表示潜在特征的通道数，Vj表示V的第j个通道，Pj表示P的第j个通道；UP()表示上采样操作；
并且，原型表征损失Lp1的表达式为：
其中，和分别表示潜在特征Vs和Vt中分割目标对应区域的特征，Lmse表示均方根误差，N和M分别表示3D医学图像分割数据集中有标注图像和未标注图像的数量。
进一步地，训练损失函数还包括：潜在特征损失Lfea；潜在特征损失Lfea用于表征学生网络和教师网络提取的潜在特征之间的差异，其表达式为：
其中，Lmse表示均方根误差，N和M分别表示3D医学图像分割数据集中有标注图像和未标注图像的数量；Vi s和Vi t分别表示3D医学图像分割数据集中的第i个图像Xi输入后，学生网络和教师网络提取的潜在特征。
进一步地，学生网络中，解码模块还包括K个辅助分割解码器；辅助分割解码器用于对潜在特征进行特征提取和上采样，得到分割结果；K个辅助分割解码器的上采样方式互不相同，且均不同于第一分割解码器；
并且，训练损失函数还包括：分割一致性损失Lmc；分割一致性约束用于表征K个辅助分割解码器及第一分割解码器的分割结果之间的差异，其表达式为：
其中，K为正整数，和分别表示图像Xi输入后，第m个分割解码器和第n个分割解码器预测的分割结果，表示经锐化处理后的结果；分割解码器为辅助分割解码器或第一分割解码器。
进一步地，学生网络中，解码模块还包括：重建解码器；重建解码器用于对潜在特征进行特征提取和上采样，以恢复原始图像信息，得到重建图像；
并且，训练损失函数还包括：重建损失Lsup1；重建损失用于表征学生网络重建得到的重建图像与原始图像之间的差异，其表达式为：
其中，表示图像Xi输入后学生网络重建得到的重建图像，α表示平衡参数。
进一步地，第一编码模块中，编码器之后还包括：F个依次连接的HybridFormer模块；第二编码模块中，编码器之后还包括：F个依次连接的HybridFormer模块；
HybridFormer模块用于计算像素空间和样本维度的自注意力；
并且，学生网络提取的潜在特征为第一编码模块中的编码器提取的特征图像经F个HybridFormer模块处理后的特征图像，教师网络提取的潜在特征为第二编码模块中的编码器提取的特征图像经F个HybridFormer模块处理后的特征图像；
其中，F为正整数。
进一步地，训练损失函数还包括：分割损失Lsup2，用于表征第一分割解码器预测的分割结果与金标准之间的差异，其表达式为：
其中，N表示3D医学图像分割数据集中有标注图像数量，Yi表示3D医学图像分割数据集中的第i个图像Xi对应的分割图像的金标准，Pi s表示图像Xi输入后，第一分割解码器预测的分割结果；Lseg表示DICE损失和交叉熵损失之和。
进一步地，随机掩码操作包括：
将3D医学图像分割成大小相等且互不重叠的立方体块，随机选取一定比例的立方体块，将相应区域像素置零。
按照本发明的又一个方面，提供了一种3D医学图像分割方法，包括：
将待分割的3D医学图像输入至由本发明提供的基于掩码建模的3D医学图像分割模型建立方法建立得到的3D医学图像分割模型，从3D医学图像分割模型的输出获取分割结果。
按照本发明的又一个方面，提供了一种计算机可读存储介质，包括存储的计算机程序，计算机程序被处理器执行时，控制计算机可读存储介质所在设备执行本发明提供的基于掩码建模的3D医学图像分割模型建立方法，和/或，本发明提供的3D医学图像分割方法。
总体而言，通过本发明所构思的以上技术方案，能够取得以下有益效果：
(1)本发明在对3D医学图像分割模型进行训练时，对输入的原始图像进行随机掩码操作生成两个不同的3D掩码图像，分别输入学生网络和教师网络，使得两个模型的输入图像都不完整，但它们共同包含了互补和全局信息，由于掩码策略的随机性，使得两个网络的图像分割任务也存在巨大的多样性，从而可以鲁棒地学习相关且互补的特征，使得特征级的一致性约束在整个训练过程中产生有效的无监督指导，达到提高鲁棒性、泛化性以及分割精度的效果，同时，训练损失函数中包含了原型表征损失Lp1，能够对特征空间和原型表征进行约束，从而获得更好的分割性能。
(2)在本发明的优选方案中，利用潜在空间的特征(即编码模块提取的潜在特征)和分割解码器输出的预测图来提取其前景的原型表示，能够有效探索特征图和概率图之间的联系，进一步提高模型的训练效果。
(3)在本发明的优选方案中，在训练损失函数中还设计了用于表征学生网络和教师网络提取的潜在特征之间的差异的潜在特征损失，该损失的引入，能够增强学生网络和教师网络所提取潜在特征的相似性，从而在两个网络的输入采用独立掩码的情况下，有效实现信息互补。
(4)在本发明的优选方案中，在学生网络中还引入了辅助分割解码器，并在辅助分割解码器和原有的第一分割解码器两两之间设置了一致性约束，使得这些分割解码器之间相互学习，由于辅助分割解码器所采用的上采样方式互不相同且不同于原有的第一分割解码器，因此，该学生间的相互学习是一种解耦的学生间学习，使得第一分割解码器能够接收到额外的解耦知识从而得到更有建设性的无监督指导，为老师模型提供错误怀疑监控和纠正能力，能够有有效避免预测不确定性对网络鲁棒性、泛化性和准确性的影响。
(5)在本发明的优选方案中，学生网络中还引入了用于恢复原始图像信息的重建解码器，使得学生网络同时完成分割任务和重建任务，由此实现了一种联合学习任务，同时，在训练损失函数中相应引入了重建损失， 由于联合学习任务中的两个任务共用相同的编码器结构，重建解码器及重建损失的引入，将使得该编码器结构在图像信息部分缺失的情况下也能准确提取到潜在特征，保证分割解码器的预测准确性。
(6)在本发明的优选方案中，学生网络和教师网络的编码模块中，在编码器之后还引入了一个或多个HybridFormer模块，用于计算像素空间和样本维度的自注意力，由此能够进一步提高潜在空间中的特征表征能力。
【附图说明】
图1为本发明实施例提供的半监督学习模型的框架示意图；
图2为本发明实施例提供的基于掩码建模的3D医学图像分割模型建立方法示意图；
图3为本发明实施例提供的3D医学图像分割方法与现有方法在相同左心房数据集上的分割结果对比示意图；其中，(a)为分割金标准，(b)为现有的MT方法的分割结果，(c)为现有的UA-MT方法的分割结果，(d)为现有的SSASNet方法的分割结果，(e)为现有的DTC方法的分割结果，(f)为现有的URPC方法的分割结果，(g)为现有的MCNet+方法的分割结果，(h)为本发明实施例提供的3D医学图像分割方法的分割结果。
【具体实施方式】
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本发明，并不用于限定本发明。此外，下面所描述的本发明各个实施方式中所涉及到的技术特征只要彼此之间未构成冲突就可以相互组合。
在本发明中，本发明及附图中的术语“第一”、“第二”等(如果存 在)是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。
为了解决现有的基于双模型架构的SSL方法应用于三维医学图像分割时，由于样本量稀少而导致鲁棒性、泛化性差且分割精度不高的技术问题，本发明提供了基于掩码建模的3D医学图像分割模型建立方法及其应用，其整体思路在于：通过对输入的原始图像进行随机掩码操作生成两个不同的掩码图像，分别输入学生网络和教师网络，使得两个模型的输入图像都不完整，但它们共同包含了整体信息，由于掩码策略的随机性，使得两个网络的图像分割任务也存在巨大的多样性，从而可以鲁棒地学习相关且互补的特征，使得特征级的一致性约束在整个训练过程中产生有效的无监督指导，达到提高鲁棒性、泛化性以及分割精度的效果。
在以上掩码策略的基础上，为了进一步提高三维医学图像分割的性能，本发明还提出了两种新的学习方式，即多样的联合任务学习(DJL)和解耦的学生间学习(DIL)，并将其实例化为增强的师生架构，用于鲁棒的三维医学图像分割。在多样的联合任务学习中，学生网络在完成三维图像分割任务的同时，还会完成恢复原始图像信息的任务，这两个任务将共用相同的编码器结构，基于掩码策略生成的不同的掩码图像，教师和学生模型学习联合分割相同目标，但恢复不同图像内容的任务，由于掩码策略的随机性，这种联合任务也具有巨大的多样性。为了便于上述DJL，DIL中的学生模型还附加了一个或多个辅助解码分支，通过使用不同的上采样设计，可以等效地将其视为其他“学生”，通过优化输出级的成对一致性约束，辅助分支与原分支相互学习，但它们的权重与师生同步无关，学生由此接收到额外的解耦知识从而得到更有建设性的无监督指导，避免因原分支学习到的知识出现差错，而影响老师网络学习到正确的知识，为老师模型提供错误怀疑监控和纠正能力。
以下为实施例。
实施例1：
一种基于掩码建模的3D医学图像分割模型建立方法，如图1和图2所示，包括：
(S1)建立包括学生网络和教师网络的半监督学习网络，该半监督学习网络的结构如图1所示；
参阅图1，学生网络包括：第一编码模块和解码模块；第一编码模块包括编码器，用于提取三维输入图像的不同尺度特征，得到潜在特征；解码模块包括第一分割解码器，第一分割解码器用于对潜在特征进行特征提取和上采样，得到分割结果；
可选地，本实施例中，学生网络中的编码器由多层卷积层和下采样层组成；第一分割解码器则相对应由多层卷积层和上采样层组成，且上采样具体通过转置卷积实现；
由于本实施例中，输入学生网络和教师网络为经过随机掩码操作之后的图像，图像中会存在部分信息的缺失，针对这一情况，本实施例中，第一编码模块中，编码器之后还包括：F个依次连接的HybridFormer模块，F为正整数；HybridFormer模块用于计算像素空间和样本维度的自注意力；并且，学生网络提取的潜在特征为第一编码模块中的编码器提取的特征图像经F个HybridFormer模块处理后的特征图像；
可选地，HybridFormer模块中，像素空间自注意力和样本维度自注意力的计算分别由两个部分完成，这两个部分分别包括多层卷积层和下采样层；在实际应用中，HybridFormer模块的数量F可根据实际的需要灵活设置；
第一编码模块输出的潜在特征输入第一分割解码器中，通过跨越连接的方式，将低阶输出特征图与其对应的上一级大尺度卷积核提取的特征图沿通道方向拼接，使用3×3卷积进一步提取大尺度的特征，最后得到分割预测图。
参阅图1，教师网络包括：第二编码模块和第二分割解码器；第二编码 模块的结构与学生网络中的第一编码模块结构一致，包括编码器以及编码器之后的F个HybridFormer模块，用于提取输入的掩码图像的潜在特征；第二分割解码器的结构与学生网络中的第一分割解码器的结构一致，用于对潜在特征进行特征提取和上采样，得到分割结果。
学生网络和教师网络中，HybridFormer模块的引入，有效提高了潜在空间中的特征表征能力。
参阅图1，为了实现多样的联合任务学习(DJL)，以更好地学习掩码图像对中的互补信息，学生网络中的解码模块包括两种类型的分支，一种是重建解码器，另一种是分割解码器，重建解码器用于对潜在特征进行特征提取和上采样，以恢复原始图像信息，得到重建图像；分割解码器预测分割结果。同时，为了降低分割结果预测的不确定性，本实施例中，分割解码器具体包括三个，其中一个为第一分割解码器，另两个为辅助分割解码器，这两个辅助分割解码器也用于对潜在特征进行特征提取和上采样，得到分割结果，但是，这两个辅助分割解码器所采用的上采样方式互不相同，且均与第一分割解码器的上采样方式不同，分别为线性插值和临近插值；辅助分割解码器与第一分割解码器相互配合，实现了解耦的学生间学习(DIL)。
应当说明的是，此处关于辅助分割解码器的数量以及所采用的上采样方式，仅为示例性的描述，不应理解为对本发明唯一的限定，在实际应用中，可根据实际需要灵活调整。
对于所建立的半监督学习模型，本实施例相应提出了模型的训练方式，具体为：
(S2)利用包括有标注图像和未标注图像的3D医学图像分割数据集对半监督学习网络进行训练；
训练方式如下：冻结教师网络的权重，将每个图像进行两次随机掩码操作后，分别输入至学生网络和教师网络，根据预设的训练损失函数对学 生网络的权重进行优化更新，并将更新后的权重迁移至教师网络；可选地，本实施例中，学生网络迁移至教师网络，具体通过指数滑动平均(EMA)完成。
本实施例中，使用Xi∈□H×W×D和Yi∈{0,1}H×W×D来表示输入图像和分割标签，其中H、W和D表示三维图像的高度、宽度和深度。标注数据集和未标注数据集分别表示为DaL＝{Xi,Yi|i＝1,...N}和DaU＝{Xi|i＝N+1,...N+M}，其中N和M分别表示已标注图像和未标注图像的数量。P(Ypred|X；θ)是输入图像的概率图，用于表示每个像素属于各类分割目标的概率。
可选地，本实施例中，对三维图像进行掩码操作的具体方式包括：
将3D医学图像分割成大小相等(例如4×4×4)且互不重叠的立方体块，从均匀分布中随采样，随机遮盖一定比例的立方体块，被遮盖的区域像素置零；通过掩码操作，输入学生网络和教师网络的两个掩码图像缺失了部分信息；由于该随机掩码操作具有随机性，因此，两个掩码图像的掩码区域不同，特征在潜在空间会被随机掩码破坏，这对于医学图像分割任务至关重要，同时，这两个掩码图像共同包含了完整的图像信息。
基于步骤(S1)所建立的半监督学习网络，输入学生网络的掩码图像经其编码模块处理后，得到潜在特征Vs，之后经过第一分割解码器预测得到分割结果Ps_s_Tconv，同时，经两个辅助分割解码器得到分割结果Ps_s_Tri，Ps_s_Nea，经重建解码器恢复原始图像信息后，得到重建图像输入教师网络的掩码图像经其编码模块处理后，得到潜在特征Vt，之后经过第一分割解码器预测得到分割结果Ps_t。
现有双模型架构中，仅仅通过强制学生网络的概率图与教师网络的概率图来提取像素空间信息，而忽略了潜在特征空间的约束和未标记数据中信息的挖掘。虽然一致性学习在半监督分割任务中起到十分重要的作用，但是本实施例采用的掩码操作减少了冗余信息，并创建了比噪声或变换更 具挑战性的特征表示任务。具有不同随机掩码的同一图像可能导致不同的预测，特别是当要分割的目标区域被掩盖时。直接对齐师生网络的预测结果过于严格，可能会导致预测崩溃。受原型学习的启发，本实施例利用潜在空间的特征和预测图来提取其前景的原型表示，以探索特征图和概率图之间的联系，相应地，本实施例中，训练损失函数包括原型表征损失Lp1，用于表征学生网络和教师网络提取的潜在特征Vs和Vt中分割目标对应区域的特征之间的差异；原型，即潜在特征中分割目标对应区域的特征，具体通过编码模块提取的潜在特征和分割解码器预测的分割结果共同决定，因此，本实施例中，训练损失函数的设计能够对特征空间和原型表征进行约束来训练所建立的模型，以获得更好的分割性能。
为了提取多通道特征信息，本实施例在计算原型时，沿通道维度对特征进行平均，相应地，对于学生网络或教师网络，其潜在特征中分割目标对应区域的特征的计算方式如下：
其中，V表示潜在特征，P表示分割结果；C表示潜在特征的通道数，Vj表示V的第j个通道，Pj表示P的第j个通道；UP()表示上采样操作；Pfg∈[0,1]H×W×D即表示前景原型，基于该原型生成方式，原型表征损失Lp1的表达式为：
其中，和分别表示潜在特征Vs和Vt中分割目标对应区域的特征，Lmse表示均方根误差，N和M分别表示3D医学图像分割数据集中有标注图像和未标注图像的数量。
本实施例进一步考虑到学生网络和教师网络的掩码是独立的，因此通 过对训练损失函数做了进一步改进，从而加强二者所提取的潜在特征(Vs,Vt)的相似性从而实现信息互补。相应地，本实施例中，训练损失函数还包括：潜在特征损失Lfea；潜在特征损失Lfea用于表征学生网络和教师网络提取的潜在特征之间的差异，其表达式为：
其中，Lmse表示均方根误差，N和M分别表示3D医学图像分割数据集中有标注图像和未标注图像的数量；Vi s和Vi t分别表示3D医学图像分割数据集中的第i个图像Xi输入后，学生网络和教师网络提取的潜在特征。
为了有效避免预测不确定性对网络鲁棒性、泛化性和准确性的影响，本实施例在引入辅助分割解码器的基础上，会使辅助分割解码器连同原有的第一分割解码器两两之间通过一致性约束，并使用锐化函数减轻容易被错误分类像素的影响，对于任意一个分割解码器，以Pi s表示第i个图像Xi输入后其所预测的分割结果，则该预测结果经锐化函数处理后的结果表示为：
其中，T表示用于控制锐化程度的超参数；Pi s_sharp即为经锐化函数处理后的结果，在分割解码器两两之间的一致性约束中，将作为伪标签使用，由此能够使分割解码器预测的分割结果接近于锐化处理之后的结果；
本实施例中，分割解码器两两之间的一致性约束通过分割一致性损失Lmc进行表征，相应地，训练损失函数还包括：分割一致性损失Lmc；分割一致性约束用于表征2个辅助分割解码器及第一分割解码器的分割结果之间的差异，其表达式为：
其中，和分别表示图像Xi输入后，第m个分割解码器和第n个分割解码器预测的分割结果，表示经锐化处理后的结果；分割解码器为辅助分割解码器或第一分割解码器；本实施例通过在训练损失函数中引入分割一致性约束，能够约束各分割解码器的分割结果保持一致。
由于本实施例实现了联合任务学习，该联合任务除了包括分割任务，还包括重建任务，并且两个任务共用同一个编码结构；在掩码图像信息确实的情况下，通过重建任务，能够重建原始的、未被破坏的体素信息，从而使编码器能够更有效地提取特征。为了保证重建任务的准确性，本实施例中，训练损失函数还包括：重建损失Lsup1；重建损失用于表征学生网络重建得到的重建图像与原始图像之间的差异，其表达式为：
其中，表示图像Xi输入后学生网络重建得到的重建图像，α表示平衡参数。
同时，为了保证分割任务的准确性，本实施例中，训练损失函数还包括：分割损失Lsup2，用于表征第一分割解码器预测的分割结果与金标准之间的差异，其表达式为：
其中，N表示3D医学图像分割数据集中有标注图像的数量，Yi表示3D医学图像分割数据集中的第i个图像Xi对应的分割图像的金标准，Pi s表示图像Xi输入后，第一分割解码器预测的分割结果；Lseg表示DICE损失和交 叉熵损失之和。
基于以上分析，本实施例中，总体的训练损失函数可表示如下：
Ltotal＝Lp1+Lfea+Lmc+Lsup1+Lsup2。
在随机掩码的基础上，基于以上半监督学习网络结构及相应的训练损失函数，本实施例能够得到分割性能优良的学生网络。
在以上步骤(S1)和(S2)的基础上，本实施例还包括：(S3)提取第一编码模块和第一解码器连接构成3D医学图像分割模型。
总的来说，本实施例通过随机掩码策略保证任务的多样性，在此基础上，通过多样的联合任务学习(DJL)和解耦的学生间学习(DIL)，使得半监督学习网络中的学生网络和教师网络可以鲁棒地学习相关且互补的特征，使得特征级的一致性约束在整个训练过程中产生有效的无监督指导，并使得学生网络接收到解耦知识从而得到更有建设性的无监督指导，为教师网络提供错误怀疑监控和纠正能力。最终本实施例所建立的3D医学图像分割模型，具有更强的鲁棒性和泛化性，以及更高的分割精度。
实施例2：
一种3D医学图像分割方法，包括：
将待分割的3D医学图像输入至由上述实施例1提供的基于掩码建模的3D医学图像分割模型建立方法建立得到的3D医学图像分割模型，从3D医学图像分割模型的输出获取分割结果。
由于本实施例1所建立的3D医学图像分割模型，具有更强的鲁棒性和泛化性，以及更高的分割精度，基于该3D医学图像分割模型，本实施例能够在各种三维医学图像分割场景下，均能获得高精度的分割结果。
实施例3：
一种计算机可读存储介质，包括存储的计算机程序，计算机程序被处 理器执行时，控制计算机可读存储介质所在设备执行上述实施例1提供的基于掩码建模的3D医学图像分割模型建立方法，和/或，上述实施例2提供的3D医学图像分割方法。
以下使用来自左心房分割挑战赛中的三维GE-MRI数据集上分割结果对本发明所提供方法的有益效果做进一步的验证。数据集是使用临床全身MRI扫描仪获取的，数据分辨率为0.625×0.625×0.625mm3，其中154个扫描数据具有专家给出的标签。随机选择123个扫描数据进行训练，31个数据样本进行测试。在网络训练之前，对所有扫描数据进行预处理，将像素强度标准化为单位方差和零均值，并将样本随机裁剪为112×112×80mm3。在该数据集上，本发明提供的3D医学图像分割方法与其他现有分割方法在左心房数据集上的图像分割可视化结果如图3所示；图3中，(a)为分割金标准，(b)～(g)分别为现有的MT、UA-MT、SSASNet、DTC、URPC、MCNet+方法的分割结果，(h)为本发明实施例提供的3D医学图像分割方法的分割结果。10％和20％标签量下的分割定量结果如表1所示，评估的指标包括分割准确率(Dice,Jaccard)，95％豪斯多夫距离(95HD)、平均表面距离(ASD)，其中，Dice和Jaccard越大，表明网络性能越好，95HD和ASD越小，表明网络性能越好。表1中，“Ours”即表示本发明所提供的分割方法。
如表1所示，基于监督训练的V-Net在10％和20％标签下的分割结果作为基线。从表中可以观察到，相对于基线，所有半监督方法都可以在10％标签量下对分割结果提供更有成效的指导，这揭示了包含在未标记数据中的大量多样且对于模型训练有用的信息。特别地，如图3所示，具有100％标记数据的监督V-Net优于所有半监督方法。然而，本发明方法与100％标记数据的监督V-Net之间仅存在微小的差异。此外，所有的量化结果也综合反映了本发明所提供的分割方法比其他半监督方法有更实质的改进。
具体来说，主流的半监督方法在20％和10％的标记数据上都取得了比相应的监督训练下V-Net更大的改进。特别地，本发明方法在四个量化指标上均优于所比较的方法。在比较的方法中，MC-Net+略差于本发明方法。随着标注数据的增加，所有半监督方法分割结果都得到改善。在极其有限的标注数据下，相较于其他最先进的半监督算法，本发明方法仍然能提供明显提升的分割指标结果。为了直观地揭示本专利方法的优越性，在图3中可视化了20％标记数据下所有对比方法生成的分割结果的2D和3D视图。显然，本发明方法保留了更多的细节并锐化了大多数孤立区域。虽然本发型方法在参数数量和计算方面没有达到最佳结果，但与其他比较算法的差异并不显著，因此鉴于现代计算机的强大算力，不会带来太多的计算负担。
表1左心房分割任务中各种评价方法的定量比较
本领域的技术人员容易理解，以上所述仅为本发明的较佳实施例而已，并不用以限制本发明，凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本发明的保护范围之内。

Claims (10)

  1. 一种基于掩码建模的3D医学图像分割模型建立方法，其特征在于，包括：
    (S1)建立包括学生网络和教师网络的半监督学习网络；
    所述学生网络包括：第一编码模块和解码模块；所述第一编码模块包括编码器，用于提取三维输入图像的不同尺度特征，得到潜在特征；所述解码模块包括第一分割解码器，所述第一分割解码器用于对潜在特征进行特征提取和上采样，得到分割结果；
    所述教师网络包括：与所述第一编码模块结构一致的第二编码模块，以及与所述第一分割解码器结构一致的第二分割解码器；
    (S2)利用包括有标注图像和未标注图像的3D医学图像分割数据集对所述半监督学习网络进行训练，训练方式如下：冻结所述教师网络的权重，将每个图像进行两次随机掩码操作后，分别输入至所述学生网络和所述教师网络，根据预设的训练损失函数对所述学生网络的权重进行优化更新，并将更新后的权重迁移至所述教师网络；所述训练损失函数包括原型表征损失Lp1，用于表征所述学生网络和所述教师网络提取的潜在特征Vs和Vt中分割目标对应区域的特征之间的差异；
    (S3)提取第一编码模块和第一解码器连接构成3D医学图像分割模型。
  2. 如权利要求1所述的基于掩码建模的3D医学图像分割模型建立方法，其特征在于，对于学生网络或教师网络，其潜在特征中分割目标对应区域的特征pfg的计算方式如下：
    其中，V表示潜在特征，P表示分割结果；C表示潜在特征的通道数，Vj表示V的第j个通道，Pj表示P的第j个通道；UP()表示上采样操作；
    并且，所述原型表征损失Lp1的表达式为：
    其中，和分别表示潜在特征Vs和Vt中分割目标对应区域的特征，Lmse表示均方根误差，N和M分别表示所述3D医学图像分割数据集中有标注图像和未标注图像的数量。
  3. 如权利要求1所述的基于掩码建模的3D医学图像分割模型建立方法，其特征在于，所述训练损失函数还包括：潜在特征损失Lfea；所述潜在特征损失Lfea用于表征所述学生网络和所述教师网络提取的潜在特征之间的差异，其表达式为：
    其中，Lmse表示均方根误差，N和M分别表示所述3D医学图像分割数据集中有标注图像和未标注图像的数量；Vi s和Vi t分别表示所述3D医学图像分割数据集中的第i个图像Xi输入后，所述学生网络和所述教师网络提取的潜在特征。
  4. 如权利要求3所述的基于掩码建模的3D医学图像分割模型建立方法，其特征在于，所述学生网络中，解码模块还包括K个辅助分割解码器；所述辅助分割解码器用于对潜在特征进行特征提取和上采样，得到分割结果；所述K个辅助分割解码器的上采样方式互不相同，且均不同于所述第一分割解码器；
    并且，所述训练损失函数还包括：分割一致性损失Lmc；所述分割一致性约束用于表征所述K个辅助分割解码器及所述第一分割解码器的分割结果之间的差异，其表达式为：
    其中，K为正整数，和分别表示图像Xi输入后，第m个分割解码器和第n个分割解码器预测的分割结果，表示经锐化处理后的结果；所述分割解码器为辅助分割解码器或所述第一分割解码器。
  5. 如权利要求4所述的基于掩码建模的3D医学图像分割模型建立方法，其特征在于，所述学生网络中，解码模块还包括：重建解码器；所述重建解码器用于对潜在特征进行特征提取和上采样，以恢复原始图像信息，得到重建图像；
    并且，所述训练损失函数还包括：重建损失Lsup1；所述重建损失用于表征所述学生网络重建得到的重建图像与原始图像之间的差异，其表达式为：
    其中，表示图像Xi输入后所述学生网络重建得到的重建图像，α表示平衡参数。
  6. 如权利要求5所述的基于掩码建模的3D医学图像分割模型建立方法，其特征在于，所述第一编码模块中，编码器之后还包括：F个依次连接的HybridFormer模块；所述第二编码模块中，编码器之后还包括：F个依次连接的HybridFormer模块；
    所述HybridFormer模块用于计算像素空间和样本维度的自注意力；
    并且，所述学生网络提取的潜在特征为所述第一编码模块中的编码器提取的特征图像经F个HybridFormer模块处理后的特征图像，所述教师网 络提取的潜在特征为所述第二编码模块中的编码器提取的特征图像经F个HybridFormer模块处理后的特征图像；
    其中，F为正整数。
  7. 如权利要求1～6任一项所述的基于掩码建模的3D医学图像分割模型建立方法，其特征在于，所述训练损失函数还包括：分割损失Lsup2，用于表征所述第一分割解码器预测的分割结果与金标准之间的差异，其表达式为：
    其中，N表示所述3D医学图像分割数据集中有标注图像的数量，Yi表示所述3D医学图像分割数据集中的第i个图像Xi对应的分割图像的金标准，Pi s表示图像Xi输入后，所述第一分割解码器预测的分割结果；Lseg表示DICE损失和交叉熵损失之和。
  8. 如权利要求1～6任一项所述的基于掩码建模的3D医学图像分割模型建立方法，其特征在于，所述随机掩码操作包括：
    将3D医学图像分割成大小相等且互不重叠的立方体块，随机选取一定比例的立方体块，将相应区域像素置零。
  9. 一种3D医学图像分割方法，其特征在于，包括：
    将待分割的3D医学图像输入至由权利要求1～8任一项所述的基于掩码建模的3D医学图像分割模型建立方法建立得到的3D医学图像分割模型，从所述3D医学图像分割模型的输出获取分割结果。
  10. 一种计算机可读存储介质，其特征在于，包括存储的计算机程序，所述计算机程序被处理器执行时，控制所述计算机可读存储介质所在设备执行权利要求1～8任一项所述的基于掩码建模的3D医学图像分割模型建立方法，和/或，权利要求9所述的3D医学图像分割方法。
PCT/CN2023/124922 2023-05-29 2023-10-17 基于掩码建模的3d医学图像分割模型建立方法及其应用 Ceased WO2024244261A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/689,426 US20250225776A1 (en) 2023-05-29 2023-10-17 Method for establishing 3d medical image segmentation model based on masked modeling and application thereof

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310616934.7A CN116664588B (zh) 2023-05-29 2023-05-29 基于掩码建模的3d医学图像分割模型建立方法及其应用
CN202310616934.7 2023-05-29

Publications (1)

Publication Number Publication Date
WO2024244261A1 true WO2024244261A1 (zh) 2024-12-05

Family

ID=87723582

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/124922 Ceased WO2024244261A1 (zh) 2023-05-29 2023-10-17 基于掩码建模的3d医学图像分割模型建立方法及其应用

Country Status (3)

Country Link
US (1) US20250225776A1 (zh)
CN (1) CN116664588B (zh)
WO (1) WO2024244261A1 (zh)

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116664588A (zh) * 2023-05-29 2023-08-29 华中科技大学 基于掩码建模的3d医学图像分割模型建立方法及其应用
CN119323526A (zh) * 2024-12-16 2025-01-17 厦门真景科技有限公司 一种移除穿帮物体模型的训练方法
CN119478558A (zh) * 2025-01-14 2025-02-18 中国地质大学（武汉） 一种基于sam模型的扫描电镜矿物识别方法及设备
CN119762786A (zh) * 2024-12-30 2025-04-04 北京理工大学 一种基于随机超分学习辅助和多层特征映射的腹部器官分割方法及系统
CN119809959A (zh) * 2025-03-13 2025-04-11 山东黄海智能装备有限公司 一种微生物实验菌落图像处理方法
CN119887808A (zh) * 2025-03-26 2025-04-25 华中科技大学同济医学院附属同济医院 一种类内区域动态解耦的医学图像分割方法
CN119888235A (zh) * 2025-03-24 2025-04-25 山东大学 一种数据驱动迭代学习的心血管分割方法及系统
CN119943297A (zh) * 2025-04-03 2025-05-06 磐技(上海)信息科技有限公司 基于深度学习的医疗影像数据处理方法及系统
CN119992193A (zh) * 2025-01-17 2025-05-13 电子科技大学 基于特征解耦的小样本图像分类方法、设备及存储介质
CN120088281A (zh) * 2025-05-06 2025-06-03 华中科技大学同济医学院附属协和医院 一种基于mr图像的肿瘤组织分割方法、系统及存储介质
CN120198654A (zh) * 2025-05-19 2025-06-24 西安交通大学 基于条件约束概率生成的目标轮廓不确定性误差量化方法
CN120258258A (zh) * 2025-06-05 2025-07-04 浙江大学 一种基于地理聚类和装机预测的光伏组件废弃物预测方法
CN120388033A (zh) * 2025-06-30 2025-07-29 山东大学 一种医学图像环形区域分割方法、系统、介质及设备
CN120783149A (zh) * 2025-06-30 2025-10-14 台山市富通达软包装材料科技有限公司 一种聚丙烯薄膜缺陷检测模型的训练方法、装置及聚丙烯薄膜缺陷检测方法
CN120807296A (zh) * 2025-09-16 2025-10-17 济南大学 基于知识蒸馏的医学图像超分辨率重建方法及系统

Families Citing this family (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119068180A (zh) * 2023-05-26 2024-12-03 香港理工大学 自动图像分割系统和方法
CN117351199B (zh) * 2023-09-11 2024-10-11 华中科技大学 基于框标注训练的息肉分割模型建立方法及息肉分割方法
CN117095014A (zh) * 2023-10-17 2023-11-21 四川大学 一种半监督医学图像分割方法、系统、设备、介质
CN117197472B (zh) * 2023-11-07 2024-03-08 四川农业大学 基于鼻出血内窥镜影像的高效师生半监督分割方法及装置
CN120355836A (zh) * 2023-12-11 2025-07-22 中山大学附属第一医院 一种肿瘤与血管三维模型构建方法、装置、设备和介质
CN117830324B (zh) * 2023-12-28 2024-07-12 深圳大学 一种基于多维和全局局部联合的3d医学图像分割方法
CN117577333B (zh) * 2024-01-17 2024-04-09 浙江大学 基于因果特征学习的多中心临床预后预测系统
CN118298168A (zh) * 2024-01-18 2024-07-05 华中科技大学 一种医学图像语义分割方法及系统
CN118587232B (zh) * 2024-05-17 2025-03-14 广州大学 自蒸馏与自学习的医学图像分割方法、装置及存储介质
CN118762175B (zh) * 2024-06-07 2025-12-19 天津大学 一种基于异构感知的互学习半监督3d医学图像分割方法
CN118351321B (zh) * 2024-06-18 2024-09-24 首都医科大学附属北京朝阳医院 用于纵隔肿瘤ct图像的半监督分割方法和装置
CN118587443B (zh) * 2024-08-07 2024-12-03 之江实验室 一种基于自训练和先验引导的图像分割方法和装置
CN119152208B (zh) * 2024-09-03 2025-10-17 鹏城实验室 前景分割方法、装置、设备及存储介质
CN119600034B (zh) * 2024-10-11 2025-09-30 西安电子科技大学 基于任务感知的三维医学图像重建预训练方法
CN119417853B (zh) * 2025-01-06 2025-05-02 深圳大学 基于2+1校正模式的多扰动一致性医学图像分割方法
CN119887805B (zh) * 2025-01-17 2025-09-02 湖北大学 一种基于双网络自适应伪标签生成的半监督医学图像分割方法及系统
CN119693398B (zh) * 2025-02-25 2025-05-30 杭州电子科技大学 用于三维医学图像分割的后解码器三维轴向解耦增强方法
CN120807718B (zh) * 2025-09-15 2025-11-18 东北大学 一种基于热掩膜约束的跨模态文本到红外图像生成方法
CN120876867B (zh) * 2025-09-25 2026-02-17 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) 基于动态优化调整的广义小样本分割模型训练方法及图像分割方法
CN120931647B (zh) * 2025-10-14 2026-02-06 浙江大学海南研究院 一种海上基础设施表面缺陷检测方法、系统、设备及介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114140390A (zh) * 2021-11-02 2022-03-04 广州大学 一种基于半监督语义分割的裂纹检测方法及装置
CN114283329A (zh) * 2021-11-16 2022-04-05 华能盐城大丰新能源发电有限责任公司 一种基于强变换下的半监督遥感图像语义分割方法及设备
US20230153943A1 (en) * 2021-11-16 2023-05-18 Adobe Inc. Multi-scale distillation for low-resolution detection
CN116664588A (zh) * 2023-05-29 2023-08-29 华中科技大学 基于掩码建模的3d医学图像分割模型建立方法及其应用

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11222426B2 (en) * 2020-06-02 2022-01-11 Cape Analytics, Inc. Method for property feature segmentation
CA3249084A1 (en) * 2022-04-21 2023-10-26 Bristol-Myers Squibb Company MEDICAL IMAGING ANALYSIS USING SELF-SUPERVISED LEARNING
US12354013B2 (en) * 2022-05-03 2025-07-08 Salesforce, Inc. Systems and methods for masked self-training of unsupervised image classification
CN115861164B (zh) * 2022-09-16 2025-04-08 重庆邮电大学 一种基于多领域半监督的医学图像分割方法
CN115984308A (zh) * 2023-01-10 2023-04-18 杭州电子科技大学 基于平均教师模型的半监督肺叶分割方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114140390A (zh) * 2021-11-02 2022-03-04 广州大学 一种基于半监督语义分割的裂纹检测方法及装置
CN114283329A (zh) * 2021-11-16 2022-04-05 华能盐城大丰新能源发电有限责任公司 一种基于强变换下的半监督遥感图像语义分割方法及设备
US20230153943A1 (en) * 2021-11-16 2023-05-18 Adobe Inc. Multi-scale distillation for low-resolution detection
CN116664588A (zh) * 2023-05-29 2023-08-29 华中科技大学 基于掩码建模的3d医学图像分割模型建立方法及其应用

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
YOUNGWAN LEE; JEFFREY WILLETTE; JONGHEE KIM; JUHO LEE; SUNG JU HWANG: "Exploring The Role of Mean Teachers in Self-supervised Masked Auto-Encoders", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 5 October 2022 (2022-10-05), US, XP091334714 *

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116664588A (zh) * 2023-05-29 2023-08-29 华中科技大学 基于掩码建模的3d医学图像分割模型建立方法及其应用
CN116664588B (zh) * 2023-05-29 2025-12-23 华中科技大学 基于掩码建模的3d医学图像分割模型建立方法及其应用
CN119323526A (zh) * 2024-12-16 2025-01-17 厦门真景科技有限公司 一种移除穿帮物体模型的训练方法
CN119762786A (zh) * 2024-12-30 2025-04-04 北京理工大学 一种基于随机超分学习辅助和多层特征映射的腹部器官分割方法及系统
CN119478558A (zh) * 2025-01-14 2025-02-18 中国地质大学（武汉） 一种基于sam模型的扫描电镜矿物识别方法及设备
CN119992193A (zh) * 2025-01-17 2025-05-13 电子科技大学 基于特征解耦的小样本图像分类方法、设备及存储介质
CN119809959A (zh) * 2025-03-13 2025-04-11 山东黄海智能装备有限公司 一种微生物实验菌落图像处理方法
CN119888235A (zh) * 2025-03-24 2025-04-25 山东大学 一种数据驱动迭代学习的心血管分割方法及系统
CN119887808A (zh) * 2025-03-26 2025-04-25 华中科技大学同济医学院附属同济医院 一种类内区域动态解耦的医学图像分割方法
CN119943297A (zh) * 2025-04-03 2025-05-06 磐技(上海)信息科技有限公司 基于深度学习的医疗影像数据处理方法及系统
CN120088281A (zh) * 2025-05-06 2025-06-03 华中科技大学同济医学院附属协和医院 一种基于mr图像的肿瘤组织分割方法、系统及存储介质
CN120198654A (zh) * 2025-05-19 2025-06-24 西安交通大学 基于条件约束概率生成的目标轮廓不确定性误差量化方法
CN120258258A (zh) * 2025-06-05 2025-07-04 浙江大学 一种基于地理聚类和装机预测的光伏组件废弃物预测方法
CN120388033A (zh) * 2025-06-30 2025-07-29 山东大学 一种医学图像环形区域分割方法、系统、介质及设备
CN120783149A (zh) * 2025-06-30 2025-10-14 台山市富通达软包装材料科技有限公司 一种聚丙烯薄膜缺陷检测模型的训练方法、装置及聚丙烯薄膜缺陷检测方法
CN120807296A (zh) * 2025-09-16 2025-10-17 济南大学 基于知识蒸馏的医学图像超分辨率重建方法及系统

Also Published As

Publication number Publication date
CN116664588A (zh) 2023-08-29
US20250225776A1 (en) 2025-07-10
CN116664588B (zh) 2025-12-23

Similar Documents

Publication Publication Date Title
WO2024244261A1 (zh) 基于掩码建模的3d医学图像分割模型建立方法及其应用
Guo et al. Gan-based virtual-to-real image translation for urban scene semantic segmentation
Wickramasinghe et al. Voxel2Mesh: 3D mesh model generation from volumetric data
US20240412374A1 (en) Training method and apparatus for image processing model, electronic device, computer program product, and computer storage medium
WO2024021523A1 (zh) 基于图网络的大脑皮层表面全自动分割方法及系统
WO2021203795A1 (zh) 一种基于显著性密集连接扩张卷积网络的胰腺ct自动分割方法
CN115829994B (zh) 一种肺部ct图像分割模型构建方法、装置及电子设备
CN112990359B (zh) 一种影像数据处理方法、装置、计算机及存储介质
CN111582449B (zh) 一种目标域检测网络的训练方法、装置、设备及存储介质
CN119941731B (zh) 基于大模型的肺结节分析方法、系统、设备及介质
CN116012374A (zh) 一种三维pet-ct头颈部肿瘤分割系统及方法
US20250140383A1 (en) Temporal information enhancement-based method for 3d medical image segmentation
Zhao et al. Generalizable 3D Gaussian Splatting for novel view synthesis
Xiong et al. 3D voxel reconstruction from single-view image based on cross-domain feature fusion
CN118608578A (zh) 图像配准方法、装置、电子设备、计算机可读存储介质及计算机程序产品
CN118397275A (zh) 基于质量驱动交叉学习的半监督3d影像分割方法
Guan et al. OT-StainNet: Optimal Transport Driven Semantic Matching for Weakly Paired H&E-to-IHC Stain Transfer
Fa et al. Dual-teacher ensemble models with double-copy-paste for 3d semi-supervised medical image segmentation
Yao et al. A data augmentation method for pavement crack detection based on super‐resolution and denoising diffusion probabilistic models
CN120495368A (zh) 基于语义差异建模与双分支Transformer的三维医学图像配准方法
Bai et al. Improving a segment anything model for segmenting low-quality medical images via an adapter
CN113269815A (zh) 一种基于深度学习的医学图像配准方法及终端
CN118587254A (zh) 医学图像配准方法、系统、控制装置及存储介质
Lin et al. Hyperbolic chamfer distance for point cloud completion and beyond
Wang et al. MGMFormer: Multi‐Scale Attentional Medical Image Segmentation Network for Semantic Feature Enhancement

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 18689426

Country of ref document: US

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23939232

Country of ref document: EP

Kind code of ref document: A1

WWP Wipo information: published in national office

Ref document number: 18689426

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE