CN115861250A - Self-adaptive data set semi-supervised medical image organ segmentation method and system - Google Patents
Self-adaptive data set semi-supervised medical image organ segmentation method and system Download PDFInfo
- Publication number
- CN115861250A CN115861250A CN202211607575.0A CN202211607575A CN115861250A CN 115861250 A CN115861250 A CN 115861250A CN 202211607575 A CN202211607575 A CN 202211607575A CN 115861250 A CN115861250 A CN 115861250A
- Authority
- CN
- China
- Prior art keywords
- segmentation
- medical image
- semi
- dataset
- data set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000011218 segmentation Effects 0.000 title claims abstract description 170
- 238000000034 method Methods 0.000 title claims abstract description 110
- 210000000056 organ Anatomy 0.000 title claims abstract description 60
- 238000012549 training Methods 0.000 claims abstract description 38
- 238000007781 pre-processing Methods 0.000 claims abstract description 36
- 238000002790 cross-validation Methods 0.000 claims abstract description 15
- 238000013461 design Methods 0.000 claims abstract description 14
- 238000007619 statistical method Methods 0.000 claims abstract description 12
- 238000012360 testing method Methods 0.000 claims description 46
- 230000003044 adaptive effect Effects 0.000 claims description 36
- 238000012952 Resampling Methods 0.000 claims description 20
- 238000004422 calculation algorithm Methods 0.000 claims description 12
- 230000001629 suppression Effects 0.000 claims description 8
- 238000005070 sampling Methods 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 3
- 238000002591 computed tomography Methods 0.000 description 14
- 230000008569 process Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 9
- 238000004590 computer program Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 238000003709 image segmentation Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 238000002059 diagnostic imaging Methods 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 238000002372 labelling Methods 0.000 description 3
- 210000004100 adrenal gland Anatomy 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 210000003734 kidney Anatomy 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 238000005481 NMR spectroscopy Methods 0.000 description 1
- 230000003187 abdominal effect Effects 0.000 description 1
- 210000000709 aorta Anatomy 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 210000001198 duodenum Anatomy 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 210000003238 esophagus Anatomy 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 210000000232 gallbladder Anatomy 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 210000004185 liver Anatomy 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 210000000496 pancreas Anatomy 0.000 description 1
- 231100000915 pathological change Toxicity 0.000 description 1
- 230000036285 pathological change Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 210000000952 spleen Anatomy 0.000 description 1
- 210000002784 stomach Anatomy 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 210000001631 vena cava inferior Anatomy 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a self-adaptive data set semi-supervised medical image organ segmentation method and a system, comprising the following steps: acquiring a medical image dataset comprising a tagged dataset and an untagged dataset; performing statistical analysis on the medical image data set to obtain statistical information; preprocessing the medical image dataset based on the statistical information; constructing a semi-supervised learning frame and a segmentation network according to a semi-supervised learning method and nnU-Net, wherein the semi-supervised learning frame is used for guiding nnU-Net to adaptively design a preprocessing method and a structure and a hyper-parameter of the segmentation network; training the segmentation network according to the preprocessed label data set and the preprocessed label-free data set by adopting a five-fold cross validation method to obtain a trained segmentation network; and performing organ segmentation on the medical image according to the trained segmentation network to obtain a segmentation result graph. The generalization performance is strong, the segmentation precision is high, and the medical image organ segmentation task can be realized.
Description
Technical Field
The invention relates to the technical field of medical image segmentation, in particular to a method and a system for segmenting a semi-supervised medical image organ of a self-adaptive data set.
Background
Medical image organ segmentation has important research significance and application value, and for example, in medical auxiliary diagnosis systems such as pathological changes, surgical planning and disease diagnosis, subsequent work can be carried out only after the structural contour of a corresponding organ is acquired.
Although a semi-supervised semantic segmentation related model and algorithm exist at present, the performance of the existing segmentation model is affected by the difference of attributes such as medical imaging equipment, an imaging mode, a voxel spacing and image resolution, so that the generalization performance of the existing segmentation model is poor, and the segmentation precision of a medical image is not high.
Thus, the prior art is in need of improvement and advancement.
Disclosure of Invention
The invention mainly aims to provide a method, a system, an intelligent terminal and a computer readable storage medium for semi-supervised medical image organ segmentation of a self-adaptive data set, and aims to solve the problems of poor generalization performance and low segmentation precision of medical images of the existing segmentation model.
In order to achieve the above object, the present invention provides an adaptive dataset semi-supervised medical image organ segmentation method, comprising:
acquiring a medical image dataset comprising a tagged dataset and an untagged dataset;
performing statistical analysis on the medical image data set to obtain statistical information;
preprocessing the medical image data set based on the statistical information;
constructing a semi-supervised learning frame and a segmentation network according to a semi-supervised learning method and nnU-Net, wherein the semi-supervised learning frame is used for guiding nnU-Net to adaptively design a preprocessing method and a structure and a hyper-parameter of the segmentation network;
training the segmentation network according to the preprocessed label data set and the preprocessed label-free data set by adopting a five-fold cross validation method to obtain a trained segmentation network;
and inputting the acquired medical image data set into a trained segmentation network for organ segmentation, and outputting a segmented result graph.
Optionally, the statistical information includes a pixel intensity mean, and the preprocessing the medical image data set based on the statistical information includes:
resampling each image sample in the medical image data set;
and sequentially standardizing the pixel intensity of each pixel in each image sample according to the pixel intensity mean value.
Optionally, the statistical information further includes a first median of a voxel spacing of the image samples in an XY plane and a second median in a Z-axis direction, and the resampling of each image sample in the medical image data set includes:
setting the first median value as a voxel interval of an XY plane at the time of resampling;
and if the ratio of the second median to the first median does not exceed a set threshold, setting the second median as the voxel interval in the Z direction during resampling, and otherwise, setting the lower decile digit of the second median as the voxel interval in the Z direction during resampling.
Optionally, two parallel segmentation networks are provided, and when the image samples in the label data set are input into the segmentation networks, the loss value during training includes: loss between the prediction result of the divided network and the real label, loss between the prediction result of the divided network and the prediction result of another divided network; when the image samples in the unlabeled dataset are input into the segmentation network, the loss value during training comprises: a loss between a predicted result of the one segmented network and a predicted result of another segmented network.
Optionally, the training the segmented network according to the preprocessed tag data set and the preprocessed non-tag data set by using a five-fold cross validation method includes:
sampling the preprocessed label data set by adopting a five-fold cross validation method to obtain a sample set;
and training the segmentation network by taking the sample set and the preprocessed unlabeled data set as training samples.
Optionally, the method further includes testing the trained segmented network, and the testing method includes:
acquiring test set data and preprocessing the test set data;
dividing image samples in the preprocessed test set data into blocks by adopting a sliding window with a preset step length, and inputting the blocks into a trained segmentation network;
and removing false positive areas in the prediction result of the segmentation network according to a non-maximum suppression algorithm to obtain a test result.
In order to achieve the above object, the present invention also provides an adaptive dataset semi-supervised medical image organ segmentation system, comprising:
a dataset acquisition module for acquiring a medical image dataset comprising a tagged dataset and a non-tagged dataset;
the statistical module is used for carrying out statistical analysis on the medical image data set to obtain statistical information;
a preprocessing module for preprocessing the medical image dataset based on the statistical information;
the building module is used for building a semi-supervised learning frame and a segmentation network according to a semi-supervised learning method and nnU-Net, and the semi-supervised learning frame is used for guiding nnU-Net to adaptively design a preprocessing method and the structure and the hyper-parameters of the segmentation network;
and the optimization module is used for training the segmentation network according to the preprocessed label data set and the preprocessed label-free data set by adopting a five-fold cross validation method to obtain the trained segmentation network.
Optionally, the test system further comprises a test module, configured to obtain test set data and preprocess the test set data; dividing image samples in the preprocessed test set data into blocks by adopting a sliding window with a preset step length, and inputting the blocks into a trained segmentation network; and removing false positive areas in the prediction result of the segmentation network according to a non-maximum suppression algorithm to obtain a test result.
In order to achieve the above object, the present invention further provides an intelligent terminal, which includes a memory, a processor, and a semi-supervised medical image organ segmentation program of an adaptive data set stored in the memory and executable on the processor, wherein the semi-supervised medical image organ segmentation program of the adaptive data set, when executed by the processor, implements any one of the steps of the semi-supervised medical image organ segmentation method of the adaptive data set.
In order to achieve the above object, the present invention further provides a computer readable storage medium, on which a semi-supervised medical image organ segmentation procedure of an adaptive data set is stored, which when executed by a processor implements any one of the steps of the semi-supervised medical image organ segmentation method of the adaptive data set.
According to the invention, firstly, the statistical information of the medical image data set is obtained, then the medical image data set is preprocessed based on the statistical information, a semi-supervised learning frame and a segmentation network are constructed according to a semi-supervised learning method and nnU-Net, and the segmentation network is semi-supervised trained by utilizing a small amount of preprocessed label data and a large amount of label-free data. Compared with the prior art, various data sets are self-adapted through nnU-Net, and no manual parameter adjusting process is needed; and a semi-supervised frame guides nnU-Net to adaptively design a preprocessing method, a structure of a segmentation network and a hyper-parameter. The trained segmentation network has strong generalization performance and high segmentation precision, and can realize the organ segmentation task of the medical image.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
FIG. 1 is a flow chart of an embodiment of a method for organ segmentation in semi-supervised medical images of adaptive data sets provided by the present invention;
FIG. 2 is a schematic flow chart of step S300 in the embodiment of FIG. 1;
FIG. 3 is a network architecture diagram of a semi-supervised learning framework in the embodiment of FIG. 1;
FIG. 4 is a diagram illustrating a comparison of segmentation results in the embodiment of FIG. 1;
FIG. 5 is a flow diagram of an embodiment of testing a trained segmented network;
FIG. 6 is a schematic structural diagram of an adaptive data set semi-supervised medical image organ segmentation system provided by an embodiment of the present invention;
fig. 7 is a schematic block diagram of an internal structure of an intelligent terminal according to an embodiment of the present invention.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when …" or "upon" or "in response to a determination" or "in response to a detection". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted depending on the context to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
The technical solutions in the embodiments of the present invention are clearly and completely described below with reference to the drawings of the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and it will be appreciated by those skilled in the art that the present invention may be practiced without departing from the spirit and scope of the present invention, and therefore the present invention is not limited by the specific embodiments disclosed below.
Due to the particularity of the medical image, the data volume is small, the difference between different data is large, and the data labeling cost is high. In particular, in three-dimensional medical imaging, the performance of a segmentation model is affected by differences in attributes such as imaging equipment, imaging mode, voxel pitch, image resolution and the like, and therefore, when a conventional segmentation model is migrated to a three-dimensional medical image, generalization performance is poor, segmentation accuracy is not high, and the application to organ segmentation of a medical image is difficult.
In order to solve the problems, the invention provides a data set self-adaptive automatic segmentation method of a medical image organ based on a semi-supervised learning method, and a small amount of labeled and large amount of unlabeled medical images are used as a data set. The method can adaptively design a preprocessing method, a structure of a segmentation network and related hyper-parameters aiming at a data set, does not need any manual parameter adjusting process, can automatically deploy and train a model in the tasks of a small amount of label data and a large amount of label-free data, and realizes the medical image organ segmentation task.
Exemplary method
As shown in fig. 1, an embodiment of the present invention provides a method for organ segmentation of a semi-supervised medical image of an adaptive data set, which is deployed on an electronic device such as a mobile terminal, a computer, or a server, and implements a task of organ segmentation of a three-dimensional CT image. Specifically, the segmentation method includes the following steps:
step S100: acquiring a medical image dataset comprising a tagged dataset and an untagged dataset;
specifically, the medical images include ultrasound images, CT images, nuclear magnetic resonance images, and the like, and a series of medical images may be acquired through interfaces of the respective medical imaging devices to generate a medical image data set; the medical image data set may also be retrieved from a background server. The specific acquisition mode of the medical image dataset is not limiting.
In order to reduce the workload of labeling, three deep learning methods are mainly used at present: self-supervised learning methods, semi-supervised learning, and weakly supervised learning. The self-supervision learning method is to utilize the non-label data to train the model in a supervision mode, learn basic knowledge and then carry out knowledge migration; the semi-supervised learning method directly learns from limited labeled data and a large amount of unlabeled data to obtain a high-quality segmentation result; rather than using pixel-level labeling, the weakly supervised learning approach learns image segmentation from bounding boxes, graffiti, or image-level labels. Since the performance of the weak supervised learning and the self supervised learning is not ideal enough on the medical image segmentation task, especially on the segmentation of the three-dimensional medical image, the present embodiment adopts the semi-supervised learning method to process the organ segmentation task of the three-dimensional CT image. Accordingly, the medical image dataset comprises a labeled dataset consisting of a small number of labeled CT images and an unlabeled dataset consisting of a large number of unlabeled CT images.
Step S200: performing statistical analysis on the medical image data set to obtain statistical information;
specifically, statistical analysis is performed on all tagged images and non-tagged images in the tagged data set and the non-tagged data set, the specific method of the statistical analysis is not limited, and the items of the statistical analysis may also be specifically determined according to the application scenario, for example: and carrying out statistics on image resolution, image modality, voxel interval and pixel intensity of all image samples in the medical image data set to obtain statistical information such as median of voxel interval, mean of pixel intensity, variance, median, quantile and the like. The statistical information can be used for subsequent data preprocessing, training hyper-parameters and other processes. In the statistical analysis of the present embodiment, a first median value of the voxel interval in the XY plane (mean value of all voxel intervals in the XY plane) and a second median value in the Z-axis direction (mean value of all voxel intervals in the Z-axis direction) of all image samples are also calculated and stored in the statistical information.
CT images store data in NIFTI format, which stores some additional information in addition to the data of the image, such as: distance between each pixel, origin coordinates, direction, etc. The distance between pixels is also referred to as a voxel interval. The pixel values in a CT image are HU values (Hounsfield) which are linearly related to the gray value of the pixel. The size of the HU value of a pixel is also referred to as the pixel intensity. For a three-channel image, each channel can be processed separately to obtain the pixel intensity of the pixel under each channel.
Step S300: preprocessing the medical image data set based on the statistical information;
specifically, the pretreatment process mainly comprises: it should be noted that the preprocessing does not change the format of the medical image data, and the data formats in the preprocessed labeled data set and unlabeled data set are still NIFTI.
Optionally, the pre-processing may also include other operations, such as: a clipping operation to clip all data to non-zero value regions.
In this embodiment, as shown in fig. 2, the pretreatment specifically includes the following steps:
step S310: resampling each image sample in the medical image dataset;
specifically, before resampling an image sample, a first median (a voxel interval median of an XY plane) in the statistical information is set as a voxel interval of the XY plane at the time of resampling the image sample; when the ratio of the second median (median between voxels in the Z-axis direction) to the first median does not exceed the set threshold (3 in this embodiment), the second median is set as the voxel interval in the Z-axis direction when the image sample is resampled, otherwise, the lower decile of the second median is set as the voxel interval in the Z-axis direction when the image sample is resampled (if the second median is 120, the lower decile of the second median is 12). And after determining the voxel interval during resampling, sequentially resampling each image sample in the medical image data set to adjust the voxel interval between each pixel in each image sample.
Step S320: and sequentially standardizing the pixel intensity of each pixel in each image sample according to the pixel intensity mean value.
In particular, pixel intensity normalization is also referred to as normalization. In general, the pixel intensity of the foreground pixels and the pixel intensity of the background pixels can be normalized according to the classification of the pixels. The unlabeled data set in this embodiment cannot count the pixel intensity distribution of foreground pixels due to the lack of labels. Therefore, the standardization of the tag data set and the non-tag data set is completed by uniformly adopting panoramic pixels, and the specific method comprises the following steps: taking each image sample (image sample in the non-label data set or image sample in the label data set) as an individual, subtracting the pixel intensity mean value stored in the statistical information from the pixel intensity of each pixel point, and dividing the pixel intensity mean value by the pixel intensity variance stored in the statistical information to realize the pixel intensity standardization of each image sample. For example: for the unlabeled sample A, the pixel intensity of each pixel point in the unlabeled sample A is subjected to pixel intensity mean value removal in the statistical information, and then the pixel intensity variance in the statistical information is removed, so that the pixel intensity standardization of each pixel point in the unlabeled sample A is completed.
Step S400: constructing a semi-supervised learning frame and a segmentation network according to a semi-supervised learning method and nnU-Net, wherein the semi-supervised learning frame is used for guiding nnU-Net to adaptively design a preprocessing method and a structure and a hyper-parameter of the segmentation network;
specifically, the model structure for segmenting the network is automatically generated by relying on the heuristic rules of nnU-Net (a network framework for adaptive data sets). nnU-Net is a framework for adapting any data set based on 2D U-Net,3D U-Net and U-Net Cascade, enabling automatic adjustment of all hyper-parameters without manual intervention.
Although nnU-Net is a framework for automatically deploying medical image segmentation tasks, nnU-Net can only be applied to labeled data sets, and the labels of the data sets are used for multiple times in the automatic deployment process. According to the invention, nnU-Net and the semi-supervised learning framework are combined, the semi-supervised learning framework can guide nnU-Net to carry out data analysis, hyper-parameter setting and design of a segmentation network structure aiming at a specific data set, namely, a preprocessing method, a segmentation network structure and related hyper-parameters are designed in a self-adaptive manner, so that tasks such as a design preprocessing method, a network structure and related hyper-parameters can be completed on the premise of not using a data label, and automatic deployment is completed in semi-supervised learning.
In this embodiment, a semi-supervised learning framework is constructed by using CPS (Cross Pseudo-label Supervision), and fig. 3 is a schematic diagram of the semi-supervised learning framework. The semi-supervised learning framework comprises two parallel split networks T1 and T2, the split networks T1 and T2 having the same network architecture except initialized with different weights θ 1 and θ 2. The input to the segmentation networks T1 and T2 may be a 2D image or a 3D image, and may be from a tagged dataset or from an untagged dataset, with the two segmentation networks each outputting a respective prediction of the segmentation of the input image.
The existing CPS strategy uses two networks, but both networks input the same image, and the output of each network serves as a supervision signal for the other network. The embodiment is improved on the basis of the existing CPS strategy, and the label data set and the non-label data set are respectively input into two split networks.
Step S500: training a segmentation network according to the label data set and the label-free data set by adopting a five-fold cross validation method to obtain a trained segmentation network;
specifically, the five-fold cross validation method is to divide all data sets into 5 parts, and to take a different part as a validation set each time during training, and to take the rest as a training set.
In order to make full use of the information of the unlabeled data set as much as possible in each folding experiment, in the embodiment, only the labeled data set is divided into the training set and the verification set, but the unlabeled data set is not divided, that is, all the unlabeled data sets are used in each folding training process. During training, cross entropy and Dice Loss are used as Loss functions of the segmentation network, the batch size of each training is 4 (namely 2 label data and 2 non-label data are loaded each time), the training termination condition is that 1000 rounds of circulation are carried out, and 250 times of sampling in the label data set and the non-label data set are carried out in one round. And finally, retaining the result with the best effect of the model on the verification set to obtain the trained segmentation network.
During training, when the image samples in the label data set are input into the segmentation network, the loss value during training comprises: loss between the prediction result of the divided network and the real label, loss between the prediction result of the divided network and the prediction result of another divided network; when the image samples in the unlabeled dataset are input into the segmentation network, the loss value during training comprises: a loss between a predicted result of the one segmented network and a predicted result of another segmented network. Referring to fig. 3, the pseudo tag Y1 output from the split network T1 is taken as the real tag of the split network T2, and the pseudo tag Y2 output from the split network T2 is taken as the real tag of the split network T1, to calculate the loss between the split networks. The calculation methods of the loss between the segmentation network and the real label and the loss value between the segmentation network and the segmentation network are the same, and the 1/2 cross entropy loss and the 1/2 Dice loss are accumulated.
Optionally, data enhancement may be performed on the training set during training, and the data enhancement method may be a data enhancement method commonly used in nnU-Net, such as flipping, rotation, scaling, gaussian noise, and the like.
Step S600: and inputting the acquired medical image data set into a trained segmentation network for organ segmentation, and outputting a segmented result graph.
Specifically, after the trained segmentation network is obtained, the medical image is input into the segmentation network for organ segmentation, and a prediction result of the trained segmentation network, that is, a segmentation result, can be obtained. Fig. 4 shows a comparison of the segmentation result (Baseline 3D + CPS in the figure) of the present embodiment with the segmentation results obtained by other segmentation methods.
As described above, in this embodiment, the semi-supervised learning framework guides nnU-Net to adaptively design the preprocessing method, the network structure and the related hyper-parameters thereof for the data set, and no manual parameter adjustment process is required, so that a model can be automatically deployed and trained in a task of a small amount of label data and a large amount of label-free data, and an organ segmentation task based on a CT image is realized.
The segmentation method of the present invention can be applied to medical images in data formats such as Computed Tomography (CT) images and MRI images, and can segment abdominal organs such as liver, spleen, left kidney, right kidney, pancreas, aorta, inferior vena cava, left adrenal gland, right adrenal gland, gall bladder, esophagus, duodenum, and stomach in these medical images to complete segmentation tasks. The medical image to be segmented may be 3D medical image data or may be 2D slice data. Of course, the organ that can be segmented is not limited to the above-mentioned organ, and any organ can automatically deploy training and automatically implement the segmentation task as long as there is a small amount of label data in the data set.
Moreover, although the network structure adopted in the embodiment is U-Net, the network structure may be replaced by other network structures such as FCN, deep lab, V-Net based on CNNs, or by Trans U-Net, swin U-Net based on transform, or by adding modules such as attention mechanism and residual network based on U-Net to improve the feature extraction and characterization capability.
Semi-supervised learning frameworks may rely on self-learning (self-training), co-learning (co-training), and model-based methods, among others. The semi-supervised learning framework proposed in the embodiment is a specific example algorithm in cooperative learning, and the algorithm can be replaced by other semi-supervised strategies. For example, semi-supervised learning, which is also based on collaborative learning, relies on multi-task, multi-view, multi-branch, multi-data enhancement implementations; or other self-learning based, generative model based methods may be employed.
In an embodiment, as shown in fig. 5, the method further includes testing the trained segmented network, and the specific testing step includes:
step S700: acquiring test set data and preprocessing the test set data;
step S800: dividing the preprocessed test set data into blocks by adopting a sliding window with a preset step length, and inputting the blocks into a trained segmentation network;
step S900: and removing false positive areas in the prediction result of the segmentation network according to a non-maximum suppression algorithm to obtain a test result.
Specifically, the step of acquiring the test set data is the same as step S100, and the test set data is preprocessed by the same preprocessing method as step S200. The typical CT image size is (512, 512, 400), width and height are 512 × 512, and 400 is the number of slices. Due to video/memory limitations, the entire three-dimensional CT image cannot be fed into the segmentation network, i.e., the segmentation network accepts less input than the original image, e.g., (192, 192, 160). Therefore, the original CT image needs to be segmented by the blocks with the size of (192, 192, 160) for many times, and then sent to the segmentation network to obtain the segmentation result predicted for each block, and then the blocks are fused by using the methods such as the mean value and the gaussian weight to obtain the test result.
When the original CT image is segmented, the embodiment segments the preprocessed test set data by using a sliding window with a preset step size of 0.7 (the step size of 0.7 represents that the overlap of three dimensions of HWD between each block occupies 0.7 of the whole block), then sends the segmented data into a trained segmentation network, and removes a false positive region of a prediction result of the segmentation network according to a non-maximum suppression algorithm to obtain a test result. The segmented network can be evaluated and optimized according to the test results.
Exemplary device
As shown in fig. 6, the embodiment of the present invention further provides a system for organ segmentation of a semi-supervised medical image of an adaptive data set, which corresponds to the method for organ segmentation of a semi-supervised medical image of an adaptive data set, and specifically, the segmentation system includes:
a dataset acquisition module 600 for acquiring a medical image dataset comprising a tagged dataset and an untagged dataset;
a statistical module 610, configured to perform statistical analysis on the medical image data set to obtain statistical information;
a preprocessing module 620 for preprocessing the medical image data set based on the statistical information;
a building module 630, configured to build a semi-supervised learning framework and a segmented network according to a semi-supervised learning method and nnU-Net, where the semi-supervised learning framework is configured to guide the segmented network to adaptively design a preprocessing method, a network structure, and a hyper-parameter;
and the optimization module 640 is configured to train the segmentation network according to the preprocessed label data set and the preprocessed label-free data set by using a five-fold cross validation method, so as to obtain a trained segmentation network.
Optionally, the segmentation system further includes a test module, configured to obtain test set data and preprocess the test set data; dividing the preprocessed test set data into blocks by adopting a sliding window with a preset step length, and inputting the blocks into a trained segmentation network; and removing false positive areas in the prediction result of the segmentation network according to a non-maximum suppression algorithm to obtain a test result.
In this embodiment, the system for organ segmentation of a semi-supervised medical image of an adaptive data set may refer to corresponding descriptions in the method for organ segmentation of a semi-supervised medical image of an adaptive data set, which are not described herein again.
Based on the above embodiment, the present invention further provides an intelligent terminal, and a schematic block diagram thereof may be as shown in fig. 7. The intelligent terminal comprises a processor, a memory, a network interface and a display screen which are connected through a system bus. Wherein, the processor of the intelligent terminal is used for providing calculation and control capability. The memory of the intelligent terminal comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a semi-supervised medical image organ segmentation procedure for an adaptive data set. The internal memory provides an environment for the operating system in the non-volatile storage medium and for the execution of the semi-supervised medical image organ segmentation procedure of the adaptive data set. The network interface of the intelligent terminal is used for being connected and communicated with an external terminal through a network. The adaptive dataset semi-supervised medical image organ segmentation procedure, when executed by a processor, implements the steps of any of the above adaptive dataset semi-supervised medical image organ segmentation methods. The display screen of the intelligent terminal can be a liquid crystal display screen or an electronic ink display screen.
It will be understood by those skilled in the art that the block diagram of fig. 7 is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation to the intelligent terminal to which the solution of the present invention is applied, and a specific intelligent terminal may include more or less components than those shown in the figure, or combine some components, or have different arrangements of components.
In one embodiment, a smart terminal is provided, the smart terminal comprising a memory, a processor, and a semi-supervised medical image organ segmentation procedure of an adaptive data set stored on the memory and executable on the processor, the semi-supervised medical image organ segmentation procedure of the adaptive data set when executed by the processor performing the following operational instructions:
acquiring a medical image dataset comprising a tagged dataset and an untagged dataset;
performing statistical analysis on the medical image data set to obtain statistical information;
preprocessing the medical image dataset based on the statistical information;
constructing a semi-supervised learning frame and a segmentation network according to a semi-supervised learning method and nnU-Net, wherein the semi-supervised learning frame is used for guiding nnU-Net to adaptively design a preprocessing method and a structure and a hyper-parameter of the segmentation network;
training the segmentation network according to the preprocessed label data set and the preprocessed label-free data set by adopting a five-fold cross validation method to obtain a trained segmentation network;
and inputting the acquired medical image data set into a trained segmentation network for organ segmentation, and outputting a segmented result graph.
Optionally, the statistical information includes a pixel intensity mean, and the preprocessing the medical image data set based on the statistical information includes:
resampling each image sample in the medical image dataset;
and sequentially standardizing the pixel intensity of each pixel in each image sample according to the pixel intensity mean value.
Optionally, the statistical information further includes a first median of a voxel interval of the image sample in an XY plane and a second median in a Z-axis direction, and the resampling each image sample in the medical image data set includes:
setting the first median value as a voxel interval of an XY plane at the time of resampling;
and if the ratio of the second median to the first median does not exceed a set threshold, setting the second median as the voxel interval in the Z direction during resampling, and otherwise, setting the lower decile digit of the second median as the voxel interval in the Z direction during resampling.
Optionally, two parallel segmentation networks are provided, and when the image samples in the label data set are input into the segmentation networks, the loss value during training includes: loss between the prediction result of the divided network and the real label, loss between the prediction result of the divided network and the prediction result of another divided network; when the image samples in the unlabeled dataset are input into the segmentation network, the loss value during training comprises: a loss between a predicted result of the one segmented network and a predicted result of another segmented network.
Optionally, the training the segmented network according to the preprocessed tag data set and the preprocessed non-tag data set by using a five-fold cross validation method includes:
sampling the preprocessed label data set by adopting a five-fold cross validation method to obtain a sample set;
and training the segmentation network by taking the sample set and the preprocessed unlabeled data set as training samples.
Optionally, the method further includes testing the trained segmented network, and the testing method includes:
acquiring test set data and preprocessing the test set data;
dividing image samples in the preprocessed test set data into blocks by adopting a sliding window with a preset step length, and inputting the blocks into a trained segmentation network;
and removing false positive areas in the prediction result of the segmentation network according to a non-maximum suppression algorithm to obtain a test result.
The embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a semi-supervised medical image organ segmentation program of an adaptive data set, and when executed by a processor, the semi-supervised medical image organ segmentation program of the adaptive data set implements the steps of any one of the semi-supervised medical image organ segmentation methods of an adaptive data set provided by the embodiment of the present invention.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned functions may be distributed as different functional units and modules according to needs, that is, the internal structure of the apparatus may be divided into different functional units or modules to implement all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the description of each embodiment has its own emphasis, and reference may be made to the related description of other embodiments for parts that are not described or recited in any embodiment.
Those of ordinary skill in the art would appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the above modules or units is only one logical division, and the actual implementation may be implemented by another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed.
The integrated modules/units described above, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium and can implement the steps of the embodiments of the method when the computer program is executed by a processor. The computer program includes computer program code, and the computer program code may be in a source code form, an object code form, an executable file or some intermediate form. The computer readable medium may include: any entity or device capable of carrying the above-mentioned computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, read-Only Memory (ROM), random Access Memory (RAM), electrical carrier wave signal, telecommunication signal, software distribution medium, etc. It should be noted that the contents contained in the computer-readable storage medium can be increased or decreased as required by legislation and patent practice in the jurisdiction.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art; the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein.
Claims (10)
1. A method of semi-supervised medical image organ segmentation of an adaptive dataset, the segmentation method comprising:
acquiring a medical image dataset comprising a tagged dataset and an untagged dataset;
performing statistical analysis on the medical image data set to obtain statistical information;
preprocessing the medical image dataset based on the statistical information;
constructing a semi-supervised learning frame and a segmentation network according to a semi-supervised learning method and nnU-Net, wherein the semi-supervised learning frame is used for guiding nnU-Net to adaptively design a preprocessing method and a structure and a hyper-parameter of the segmentation network;
training the segmentation network according to the preprocessed label data set and the preprocessed label-free data set by adopting a five-fold cross validation method to obtain a trained segmentation network;
and inputting the acquired medical image data set into a trained segmentation network for organ segmentation, and outputting a segmented result graph.
2. The method of adaptive dataset semi-supervised medical image organ segmentation as recited in claim 1, wherein the statistical information includes a pixel intensity mean, and wherein the preprocessing the medical image dataset based on the statistical information includes:
resampling each image sample in the medical image dataset;
and sequentially standardizing the pixel intensity of each pixel in each image sample according to the pixel intensity mean value.
3. The method of adaptive dataset semi-supervised medical image organ segmentation as recited in claim 2, wherein the statistical information further includes a first median in the XY plane and a second median in the Z-axis direction of a voxel spacing of image samples, and the resampling the respective image samples in the medical image dataset comprises:
setting the first median value as a voxel interval of an XY plane at the time of resampling;
and if the ratio of the second median to the first median does not exceed a set threshold, setting the second median as the voxel interval in the Z direction during resampling, and otherwise, setting the lower decile digit of the second median as the voxel interval in the Z direction during resampling.
4. The method of claim 1, wherein two parallel segmentation networks are provided, and wherein the training loss values for the image samples in the label dataset input to the segmentation networks comprise: loss between the prediction result of the divided network and the real label, loss between the prediction result of the divided network and the prediction result of another divided network; when the image samples in the unlabeled dataset are input into the segmentation network, the loss value during training comprises: a loss between a predicted result of the one segmented network and a predicted result of another segmented network.
5. The method for semi-supervised medical image organ segmentation of adaptive data sets according to claim 1, wherein the training of the segmentation network based on the preprocessed labeled data sets and the preprocessed unlabeled data sets using a five-fold cross-validation method comprises:
sampling the preprocessed label data set by adopting a five-fold cross validation method to obtain a sample set;
and training the segmentation network by taking the sample set and the preprocessed unlabeled data set as training samples.
6. The method of adaptive dataset semi-supervised medical image organ segmentation of claim 1 further comprising testing the trained segmentation network, the testing method comprising:
acquiring test set data and preprocessing the test set data;
dividing image samples in the preprocessed test set data into blocks by adopting a sliding window with a preset step length, and inputting the blocks into a trained segmentation network;
and removing false positive areas in the prediction result of the segmentation network according to a non-maximum suppression algorithm to obtain a test result.
7. Semi-supervised medical image organ segmentation system for adaptive datasets, the segmentation system comprising:
a dataset acquisition module for acquiring a medical image dataset comprising a tagged dataset and a non-tagged dataset;
the statistical module is used for carrying out statistical analysis on the medical image data set to obtain statistical information;
a preprocessing module for preprocessing the medical image dataset based on the statistical information;
the building module is used for building a semi-supervised learning frame and a segmentation network according to a semi-supervised learning method and nnU-Net, and the semi-supervised learning frame is used for guiding nnU-Net to adaptively design a preprocessing method and a structure and a hyper-parameter of the segmentation network;
and the optimization module is used for training the segmentation network according to the preprocessed label data set and the preprocessed label-free data set by adopting a five-fold cross validation method to obtain the trained segmentation network.
8. The adaptive dataset semi-supervised medical image organ segmentation system of claim 7, further comprising a test module for acquiring and pre-processing test set data; dividing image samples in the preprocessed test set data into blocks by adopting a sliding window with a preset step length, and inputting the blocks into a trained segmentation network; and removing false positive areas in the prediction result of the segmentation network according to a non-maximum suppression algorithm to obtain a test result.
9. An intelligent terminal, characterized in that the intelligent terminal comprises a memory, a processor and a semi-supervised medical image organ segmentation procedure of an adaptive dataset stored on the memory and executable on the processor, the semi-supervised medical image organ segmentation procedure of the adaptive dataset being executed by the processor for implementing the steps of the semi-supervised medical image organ segmentation method of the adaptive dataset as claimed in any one of claims 1 to 6.
10. Computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a semi-supervised medical image organ segmentation procedure of an adaptive dataset, which when executed by a processor implements the steps of the semi-supervised medical image organ segmentation method of an adaptive dataset as claimed in any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211607575.0A CN115861250B (en) | 2022-12-14 | 2022-12-14 | Semi-supervised medical image organ segmentation method and system for self-adaptive data set |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211607575.0A CN115861250B (en) | 2022-12-14 | 2022-12-14 | Semi-supervised medical image organ segmentation method and system for self-adaptive data set |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115861250A true CN115861250A (en) | 2023-03-28 |
| CN115861250B CN115861250B (en) | 2023-09-22 |
Family
ID=85672929
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211607575.0A Active CN115861250B (en) | 2022-12-14 | 2022-12-14 | Semi-supervised medical image organ segmentation method and system for self-adaptive data set |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115861250B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116824146A (en) * | 2023-07-05 | 2023-09-29 | 深圳技术大学 | Small sample CT image segmentation method, system, terminal and storage medium |
| CN118212490A (en) * | 2024-05-15 | 2024-06-18 | 腾讯科技（深圳）有限公司 | Training method, device, equipment and storage medium for image segmentation model |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170231484A1 (en) * | 2016-02-12 | 2017-08-17 | Canon Kabushiki Kaisha | Image processing apparatus, image processing method, and storage medium |
| EP3459461A1 (en) * | 2017-09-25 | 2019-03-27 | Koninklijke Philips N.V. | X-ray imaging reference scan |
| CN113793304A (en) * | 2021-08-23 | 2021-12-14 | 天津大学 | Intelligent segmentation method for lung cancer target area and organs at risk |
| CN114581628A (en) * | 2022-03-03 | 2022-06-03 | 北京银河方圆科技有限公司 | Cerebral cortex surface reconstruction method and readable storage medium |
| CN114612721A (en) * | 2022-03-15 | 2022-06-10 | 南京大学 | Image classification method based on multilevel adaptive feature fusion type increment learning |
| US11526994B1 (en) * | 2021-09-10 | 2022-12-13 | Neosoma, Inc. | Labeling, visualization, and volumetric quantification of high-grade brain glioma from MRI images |
-
2022
- 2022-12-14 CN CN202211607575.0A patent/CN115861250B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170231484A1 (en) * | 2016-02-12 | 2017-08-17 | Canon Kabushiki Kaisha | Image processing apparatus, image processing method, and storage medium |
| EP3459461A1 (en) * | 2017-09-25 | 2019-03-27 | Koninklijke Philips N.V. | X-ray imaging reference scan |
| CN111107787A (en) * | 2017-09-25 | 2020-05-05 | 皇家飞利浦有限公司 | X-ray imaging reference scan |
| CN113793304A (en) * | 2021-08-23 | 2021-12-14 | 天津大学 | Intelligent segmentation method for lung cancer target area and organs at risk |
| US11526994B1 (en) * | 2021-09-10 | 2022-12-13 | Neosoma, Inc. | Labeling, visualization, and volumetric quantification of high-grade brain glioma from MRI images |
| CN114581628A (en) * | 2022-03-03 | 2022-06-03 | 北京银河方圆科技有限公司 | Cerebral cortex surface reconstruction method and readable storage medium |
| CN114612721A (en) * | 2022-03-15 | 2022-06-10 | 南京大学 | Image classification method based on multilevel adaptive feature fusion type increment learning |
Non-Patent Citations (1)
| Title |
|---|
| YONGZHI HUANG 等: ""3D Cross Pseudo Supervision (3D-CPS): A semi-supervised nnU-Net architecture for abdominal organ segmentation"", 《ARXIV》, pages 1 - 13 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116824146A (en) * | 2023-07-05 | 2023-09-29 | 深圳技术大学 | Small sample CT image segmentation method, system, terminal and storage medium |
| CN116824146B (en) * | 2023-07-05 | 2024-06-07 | 深圳技术大学 | Small sample CT image segmentation method, system, terminal and storage medium |
| CN118212490A (en) * | 2024-05-15 | 2024-06-18 | 腾讯科技（深圳）有限公司 | Training method, device, equipment and storage medium for image segmentation model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115861250B (en) | 2023-09-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220005192A1 (en) | Method and system for intracerebral hemorrhage detection and segmentation based on a multi-task fully convolutional network | |
| CN111260055B (en) | Model training method based on three-dimensional image recognition, storage medium and device | |
| JP7206386B2 (en) | Image processing model training method, image processing method, network device, and storage medium | |
| US20200364570A1 (en) | Machine learning method and apparatus, program, learned model, and discrimination apparatus | |
| CN107688783B (en) | 3D image detection method and device, electronic equipment and computer readable medium | |
| US11182904B2 (en) | Systems and methods for image segmentation | |
| CN115861250B (en) | Semi-supervised medical image organ segmentation method and system for self-adaptive data set | |
| CN110223300A (en) | CT image abdominal multivisceral organ dividing method and device | |
| CN111932529A (en) | Image segmentation method, device and system | |
| CN112446892A (en) | Cell nucleus segmentation method based on attention learning | |
| CN110930378B (en) | Emphysema image processing method and system based on low data demand | |
| CN110728666A (en) | Typing method and system for chronic nasosinusitis based on digital pathological slide | |
| CN115546270A (en) | Image registration method, model training method and equipment for multi-scale feature fusion | |
| CN115661459A (en) | 2D mean teacher model using difference information | |
| CN111353987A (en) | Cell nucleus segmentation method and device | |
| CN115601330A (en) | Colonic polyp segmentation method based on multi-scale space reverse attention mechanism | |
| CN117437423A (en) | Weak supervision medical image segmentation method and device based on SAM collaborative learning and cross-layer feature aggregation enhancement | |
| CN113706562B (en) | Image segmentation method, device and system and cell segmentation method | |
| Annavarapu et al. | An adaptive watershed segmentation based medical image denoising using deep convolutional neural networks | |
| CN114972382A (en) | Brain tumor segmentation algorithm based on lightweight UNet + + network | |
| CN116091518A (en) | Mammary gland focus segmentation method and device based on dynamic contrast enhanced magnetic resonance image | |
| Mansour et al. | Kidney segmentations using cnn models | |
| Vandermeulen et al. | Local filtering and global optimization methods for 3-D magnetic-resonance angiography image enhancement | |
| CN111507950B (en) | Image segmentation method and device, electronic equipment and computer-readable storage medium | |
| CN113409324B (en) | Brain segmentation method fusing differential geometric information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |