CN116703850A - Medical image segmentation method based on field self-adaption - Google Patents
Medical image segmentation method based on field self-adaption Download PDFInfo
- Publication number
- CN116703850A CN116703850A CN202310632806.1A CN202310632806A CN116703850A CN 116703850 A CN116703850 A CN 116703850A CN 202310632806 A CN202310632806 A CN 202310632806A CN 116703850 A CN116703850 A CN 116703850A
- Authority
- CN
- China
- Prior art keywords
- image
- entropy
- domain
- features
- source domain
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000003709 image segmentation Methods 0.000 title claims abstract description 21
- 238000009826 distribution Methods 0.000 claims abstract description 34
- 230000011218 segmentation Effects 0.000 claims abstract description 23
- 238000010586 diagram Methods 0.000 claims abstract description 16
- 238000000605 extraction Methods 0.000 claims abstract description 7
- 238000012549 training Methods 0.000 claims abstract description 6
- 230000006978 adaptation Effects 0.000 claims abstract description 4
- 238000005070 sampling Methods 0.000 claims description 13
- 238000013507 mapping Methods 0.000 claims description 10
- 230000004913 activation Effects 0.000 claims description 6
- 238000005259 measurement Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 230000000694 effects Effects 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 4
- 230000008485 antagonism Effects 0.000 claims description 3
- 210000005240 left ventricle Anatomy 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 3
- 238000013528 artificial neural network Methods 0.000 claims description 2
- 230000000295 complement effect Effects 0.000 claims description 2
- 238000010606 normalization Methods 0.000 claims description 2
- 238000013508 migration Methods 0.000 abstract description 2
- 230000005012 migration Effects 0.000 abstract description 2
- 238000003745 diagnosis Methods 0.000 description 4
- 201000010099 disease Diseases 0.000 description 3
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000010191 image analysis Methods 0.000 description 2
- 238000000691 measurement method Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000003759 clinical diagnosis Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000002861 ventricular Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10088—Magnetic resonance imaging [MRI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Multimedia (AREA)
- Medical Informatics (AREA)
- Biomedical Technology (AREA)
- Databases & Information Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Image Analysis (AREA)
Abstract
A medical image segmentation method based on field adaptation comprises the following steps: and constructing a model overall architecture, and constructing an encoder network framework according to the overall network layout to convert the image characteristics from low resolution to high resolution. After the image features are extracted, noise in the image is eliminated through a re-parameterization method. The regularization method is used for measuring the data distribution of the source domain and the target domain, so that the target training network is optimized. And constructing a decoder network framework, reconstructing an image by using the labels and the re-parameterized features, and improving the extraction capability of the model on the image structure information. And inputting the three-time re-parameterized result into a divider, and predicting the division result from different scales. An entropy discriminator network framework is constructed, and entropy distribution of a source domain and a target domain is similar to indirectly minimize entropy by using a entropy diagram of a soft segmentation result obtained by a divider. The invention realizes the migration of the existing knowledge by utilizing the similarity between the marked data and the unmarked data, and realizes the field-adaptive medical image segmentation.
Description
Technical Field
The invention relates to the field of auxiliary diagnosis of artificial intelligent medical images, in particular to a medical image segmentation method based on field self-adaption.
Background
Medical image analysis is one of the important components in the field of computer vision, and is widely used for diagnosis of various diseases. The medical image is obtained through various medical imaging devices, internal tissues of the body are presented in a non-invasive mode, and a doctor can realize clinical diagnosis according to the image information. Medical image segmentation is an important step of medical image analysis, can provide support for subsequent tasks such as disease diagnosis, treatment, disease condition monitoring and the like, and the quality of segmentation directly influences the subsequent treatment effect.
With the development of artificial intelligence, a medical image segmentation method based on deep learning is gradually raised, and compared with the traditional method, the diagnosis precision can be greatly improved. However, there are still a number of problems in this area. On one hand, the medical image has a plurality of imaging modes, the organ information of different individuals is different, and the medical image has the characteristics of low contrast and unclear integral structure; on the other hand, creating a medical image dataset of a large number of samples is a difficult task, and there is a large number of irregularly shaped tissues in the medical image, which is very costly to annotate and requires a lot of time and effort. These problems present a great challenge for medical image segmentation, and existing methods have difficulty in fully solving these problems.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a medical image segmentation method based on field adaptation.
The invention is based on a UNet model, combines a field self-adaptive method based on difference and countermeasure, explicitly measures the inter-domain distance, and optimizes the prediction result by utilizing the countermeasure entropy in an output space constraint model.
A medical image segmentation method based on field self-adaption comprises the following specific steps:
step S1, using the CT and MR medical images of the heart in the MMWHS dataset as source domain data, 16 slices are sampled from the long axis view around the center of the left ventricle cavity for each 3D image, and cut to 240X 220 size.
Step S2, firstly, constructing a model overall architecture, constructing an encoder network framework according to overall network layout, performing four downsampling operations on images of a source domain and a target domain, extracting image features, performing upsampling operation, and converting the image features from low resolution to high resolution.
And S3, after the image features are extracted, mapping the features to a low-dimensional space through three times of re-parameterization, eliminating noise in the image, and generating new data similar to the input data but without noise.
And S4, measuring the data distribution of the source domain and the target domain by using a regularization method, wherein the measurement is used as a loss term and is combined with other optimized target training networks.
And S5, constructing a decoder network framework, wherein the framework is connected with the tag and the characteristic of the encoder after the parameterization, inputting the feature into a plurality of continuous convolution blocks to reconstruct an image, and improving the extraction capability of the model on the image structure information.
And S6, inputting the three-time re-parameterization result into a divider, and predicting the division result from different scales.
And S7, constructing an entropy discriminator network frame, wherein the model adopts a PatchGan architecture, and inputs a entropy diagram of a soft segmentation result obtained by the segmenter, so that the entropy discriminator pair Ji Shang is constructed to map the distribution of a source domain and a target domain, and the entropy distribution of the source domain and the target domain is similar to indirectly minimize entropy.
The invention provides a medical image segmentation method based on field adaptation, which utilizes the similarity between marked data and unmarked data to migrate the existing knowledge, realizes knowledge migration in different fields, reserves the transferable common characteristics of a source field and a target field through an explicit measurement method and entropy minimization, and reduces the inter-domain distribution difference. In addition, the weighted self-information distribution of the source domain and the target domain is adjusted based on the antagonism entropy minimization by utilizing a discriminator loss optimization model, so that the prediction entropy of the target domain is indirectly minimized, and clearer semantic segmentation output and finer object edges are generated. The method forces the soft segmentation of the source domain and the target domain to be similar, and promotes the segmentation network to generate similar prediction for the source domain and the target domain.
The invention has the advantages that: the source domain and target domain data have strong similarity in semantic layout. While the explicit distance metric approach optimizes the model by minimizing the source domain and target domain feature loss, ignoring this similarity, and generating an uncertain prediction for the target domain. In order to inhibit the uncertainty prediction, the invention further uses an entropy-driven countermeasure learning model, and by forcing entropy mapping similarity of a source domain and a target domain, the inter-domain difference is reduced, and entropy minimization can prevent the model from being over-fitted, so that generalization performance and robustness are improved.
Drawings
FIG. 1 is a model overall architecture of the present invention
FIG. 2 is an encoder network architecture of the present invention
FIG. 3 is a re-parameterized module of the present invention
FIG. 4 is a block diagram of an entropy discriminator of the invention
Fig. 5 is a flow chart of the method of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments, and all other embodiments obtained by those skilled in the art without making any inventive effort based on the embodiments of the present invention are within the scope of protection of the present invention.
A medical image segmentation method based on field self-adaption comprises the following specific steps:
step S1, using the CT and MR medical images of the heart in the MMWHS dataset as source domain data, 16 slices are sampled from the long axis view around the center of the left ventricle cavity for each 3D image, and cut to 240X 220 size.
The detailed description of the data set processing in the above step S1 is as follows:
in the experiment, the image is cut into 240×220 size, normalized and input into the network. The CT image and the MR image are respectively used as a source domain, and the segmentation effect and the segmentation performance in a target domain are tested.
Step S2, firstly, constructing a model overall architecture, constructing an encoder network framework according to overall network layout, performing four downsampling operations on images of a source domain and a target domain, extracting image features, performing upsampling operation, and converting the image features from low resolution to high resolution.
The detailed description of the model overall architecture and the encoder network framework in the above step S2 is as follows:
firstly, a model overall architecture is constructed, wherein the model mainly comprises three modules, namely a generator module, a data distribution measurement module and an entropy discriminator module, and the generator module comprises an encoder, a decoder and a divider. An encoder network framework is constructed according to an overall network layout.
And S21, performing four downsampling operations on the images of the source domain and the target domain, extracting image features, performing upsampling operation, converting the image features from low resolution to high resolution, performing four downsampling operations on the images by the encoder in a U-shaped structure, and extracting the image features.
And S22, after the image features are extracted, four times of up-sampling are used for converting low resolution into high resolution image features, and context information of the images is obtained through jump connection, so that semantic differences of the features of different layers are reduced. Each time of up-sampling can acquire image information of different levels, a feature image obtained by the second time of up-sampling is one fourth of an original image, and the feature image contains global information of the image; the feature map obtained by the third up-sampling is one half of the original image, contains the local information of the image, and can analyze the local structure and texture details in the image through the feature map; the third upsampling restores the image to the original image size, where the feature map contains smaller structural and detail information in the image. The encoder outputs a feature map obtained by three layers of up-sampling, so that the divider and the decoder acquire multi-level image features, and the constraint on the model is enhanced.
And S3, after the image features are extracted, mapping the features to a low-dimensional space through three times of re-parameterization, eliminating noise in the image, and generating new data similar to the input data but without noise.
The detailed description of the re-parameterization in the above step S3 is as follows:
and S31, after the image features are extracted, calculating the mean value and variance of the input features by using a neural network, and generating data similar to the original features.
In step S32, in order to prevent the situation that the mean value and the variance are directly used to sample and overfit, noise is extracted from the standard normal distribution in the sampling process, and the noise is multiplied by the variance and added to generate samples with diversity, so that the stability of the model can be ensured.
And S4, measuring the data distribution of the source domain and the target domain by using a regularization method, wherein the measurement is used as a loss term and is combined with other optimized target training networks.
The explicit measurement method in step S4 is described in detail as follows:
potential features of the source domain and the target domain are obtained through the encoder, and z is respectively s And z t . Will z s And z t Is expressed as a probability density function ofAnd->，θ s And theta t For the parameters to be learned, the posterior probability is expressed as +.>And->. Obtaining an approximation of the posterior distribution by parameterized modeling, i.e. +.>And->. Due to domain differences between the source domain and the target domain, the extracted features z s And z t Is a different distribution, thus +.>And->Inequality by calculating->And->The distance between them constrains them to the same distribution. M samples are independently and randomly sampled from a source domain and a target domain respectively, a regularization distance is calculated, and a calculation formula is shown in a formula (1):
z s representing a potential feature of the device,represents the i-th sample in the source domain, +.>Representing the j-th sample in the target domain. Since the variables of the potential space follow normal distribution, the kernel definition is as shown in equation (2):
is->The first element of (a) is a first element of (b). Sigma (sigma) 2 Representing variance and u representing mean.
And S5, constructing a decoder network framework, wherein the framework is connected with the tag and the characteristic of the encoder after the parameterization, inputting the feature into a plurality of continuous convolution blocks to reconstruct an image, and improving the extraction capability of the model on the image structure information.
The details of the establishment of the decoder network framework in the above step S5 are as follows:
the decoder is connected with the tag and the characteristic of the encoder after the parameterization, and the characteristic is input into a plurality of continuous convolution blocks to reconstruct an image, so that the extraction capability of the model on the image structure information is improved. Each block consists of a convolution layer, an instance normalization layer, and an activation layer. The source domain and the target domain have the same generator and all comprise three parts of an encoder, a decoder and a divider, and the encoders share weights. The difference is that the source domain reconstructs the image using the re-parameterized features and segmentation labels, while the target domain does not have labels, and the re-parameterized features and the predicted segmentation results are used to reconstruct the image.
And S6, inputting the three-time re-parameterization result into a divider, and predicting the division result from different scales.
The details of the divider prediction and division result in the above step S6 are as follows:
and after the feature is subjected to the re-parameterization to remove noise, inputting the re-parameterization result into a divider, and predicting the division result from different scales. The three segmentation results respectively comprise integral structure information and texture detail information of the target region, are complementary, and are fused with prediction results of different scales by using a single convolution layer to serve as final segmentation results.
And S7, constructing an entropy discriminator network frame, wherein the model adopts a PatchGan architecture, and inputs a entropy diagram of a soft segmentation result obtained by the segmenter, so that the entropy discriminator pair Ji Shang is constructed to map the distribution of a source domain and a target domain, and the entropy distribution of the source domain and the target domain is similar to indirectly minimize entropy.
The above-mentioned step S7 is described in detail as follows:
step S71, a discriminatory loss optimization model is utilized based on antagonism entropy minimization, weighted self-information distribution of a source domain and a target domain is adjusted, prediction entropy of the target domain is minimized indirectly, and clearer semantic segmentation output and finer object edges are generated. Knowing the input source domain image x s Pixel-level probability prediction of (2)Entropy mapping is obtained by means of shannon entropy calculation, as shown in a formula (3):
where n is the number of images and c is the number of channels.
Step S72, introducing an countermeasure training frame to construct an entropy discriminator D E Mapping E (x) to Ji Shang s ) And E (x) t ) The entropy distribution of the source domain and the target domain is made similar to indirectly minimize entropy. The purpose of the entropy discriminator is to determine whether the entropy diagram is from the source domain or the target domain. The framework uses a GAN network to match entropy distributions of source domain data and target domain data. The arbiter uses the PatchGan architecture to distinguish between 8×8 patches. Contains 5 convolution layers, and the feature mapping number of each layer is [64,128,256,512,1 ]]The convolution kernel size is 4 x 4, the first layer convolution step size is 1. The middle convolution layer step size is 2, which contains a Relu activation function, and the parameter is 0.2. The last layer uses a sigmoid activation function. The input of the discriminator is a entropy diagram of the source domain and the target domain, the output is 0 or 1 as the domain classification result, 0 represents the entropy diagram from the source domain, and 1 represents the entropy diagram from the target domain.
The invention relates to a medical image segmentation method based on field self-adaption. The following describes specific operation steps and provides an effect diagram to demonstrate the effectiveness of the method.
The medical images of the CT and MR of the heart in the MMWHS dataset were used as source domain data, respectively, for each 3D image 16 slices were sampled from the long axis view around the center of the left ventricular cavity, cut to 240 x 220 size.
In the experimental process, firstly, a model overall architecture is constructed, an encoder network framework is constructed according to overall network layout, four downsampling operations are performed on images of a source domain and a target domain, image features are extracted, upsampling operation is performed, and the image features are converted from low resolution to high resolution. The overall architecture of the model is shown in fig. 1, and the encoder network architecture is shown in fig. 2.
After extracting the image features, the features are mapped to a low-dimensional space through three times of re-parameterization, noise in the image is eliminated, and new data similar to the input data but without noise is generated. The regularization method is then used to measure the data distribution of the source and target domains, which is used as a penalty term in combination with other optimization objectives to train the network. The re-parameterization block of the encoder is shown in fig. 3.
And secondly, constructing a decoder network framework, a divider network framework and an entropy discriminator network framework, wherein the decoder network framework is connected with the features of the label and the encoder after the parameterization, inputting the features into a plurality of continuous convolution blocks to reconstruct images, and improving the extraction capability of the model on the image structure information. The divider network framework predicts the division result from different scales through the result of three times of re-parameterization. The entropy discriminator network framework adopts a PatchGan architecture, and a entropy diagram of a soft segmentation result obtained by the segmenter is input, so that the distribution of the source domain and the target domain mapped by the entropy discriminator pair Ji Shang is constructed, and the entropy distribution of the source domain and the target domain is similar to indirectly minimize entropy. The entropy discriminator network framework is shown in fig. 4.
Claims (8)
1. A medical image segmentation method based on field adaptation comprises the following steps:
step S1, using CT and MR medical images of the heart in MMWHS data set as source domain data respectively, sampling 16 slices from a long axis view around the center of the left ventricle cavity for each 3D image, and cutting into 240X 220 size;
step S2, firstly, constructing a model overall architecture, constructing an encoder network framework according to overall network layout, performing four downsampling operations on images of a source domain and a target domain, extracting image features, performing upsampling operation, and converting the image features from low resolution to high resolution;
s3, after the image features are extracted, mapping the features to a low-dimensional space through three times of re-parameterization, eliminating noise in the image, and generating new data similar to input data but without noise;
s4, measuring the data distribution of the source domain and the target domain by using a regularization method, wherein the measurement is used as a loss term and is combined with other optimized target training networks;
s5, constructing a decoder network frame, wherein the frame is connected with the tag and the characteristic of the encoder after the parameterization, inputting the feature into a plurality of continuous convolution blocks to reconstruct an image, and improving the extraction capability of the model on the image structure information;
and S6, inputting the three-time re-parameterization result into a divider, and predicting the division result from different scales.
And S7, constructing an entropy discriminator network frame, wherein the model adopts a PatchGan architecture, and inputs a entropy diagram of a soft segmentation result obtained by the segmenter, so that the entropy discriminator pair Ji Shang is constructed to map the distribution of a source domain and a target domain, and the entropy distribution of the source domain and the target domain is similar to indirectly minimize entropy.
2. A field-adaptive based medical image segmentation method according to claim 1, wherein: the step S1 specifically comprises the following steps:
the image is cut to a size of 240×220, normalized and input to the network. The CT image and the MR image are respectively used as a source domain, and the segmentation effect and the segmentation performance in a target domain are tested.
3. A field-adaptive based medical image segmentation method according to claim 1, wherein: and S2, constructing a model overall architecture, wherein the model mainly comprises three modules, namely a generator module, a data distribution measurement module and an entropy discriminator module, and the generator module comprises an encoder, a decoder and a divider. According to the overall network layout, constructing an encoder network framework, which comprises the following specific steps:
step S21, performing four downsampling operations on images of a source domain and a target domain, extracting image features, performing upsampling operation, converting the image features from low resolution to high resolution, performing four downsampling operations on the images by an encoder in a U-shaped structure, and extracting the image features;
s22, after extracting image features, four times of up-sampling are used for converting low resolution into high resolution image features, context information of the images is obtained through jump connection, and semantic differences of the features of different layers are reduced; each time of up-sampling can acquire image information of different levels, a feature image obtained by the second time of up-sampling is one fourth of an original image, and the feature image contains global information of the image; the feature map obtained by the third up-sampling is one half of the original image, contains the local information of the image, and can analyze the local structure and texture details in the image through the feature map; the image is restored to the original image size through the third upsampling, and the characteristic diagram contains smaller structure and detail information in the image; the encoder outputs a feature map obtained by three layers of up-sampling, so that the divider and the decoder acquire multi-level image features, and the constraint on the model is enhanced.
4. A field-adaptive based medical image segmentation method according to claim 1, wherein: the step S3 specifically comprises the following steps:
s31, after extracting image features, calculating the mean value and variance of input features by using a neural network, and generating data similar to the original features;
in step S32, in order to prevent the situation that the mean value and the variance are directly used to sample and overfit, noise is extracted from the standard normal distribution in the sampling process, and the noise is multiplied by the variance and added to generate samples with diversity, so that the stability of the model can be ensured.
5. A field-adaptive based medical image segmentation method according to claim 1, wherein: the step S4 specifically comprises the following steps:
potential features of the source domain and the target domain are obtained through the encoder, and z is respectively s And z t The method comprises the steps of carrying out a first treatment on the surface of the Will z s And z t Is expressed as a probability density function ofAnd->θ s And theta t For the parameters to be learned, the posterior probability is expressed as +.>And->Obtaining an approximation of the posterior distribution by parameterized modeling, i.e. +.>And->Due to domain differences between the source domain and the target domain, the extracted features z s And z t Is a different distribution, thus +.>And->Inequality by calculating->Andthe distance between them constrains them to the same distribution; m samples are independently and randomly sampled from a source domain and a target domain respectively, a regularization distance is calculated, and a calculation formula is shown in a formula (1):
z s representing a potential feature of the device,represents the i-th sample in the source domain, +.>Represents the jth sample in the target domain; since the variables of the potential space follow normal distribution, the kernel definition is as shown in equation (2):
is->The first element of (a) is a first element of (b). Sigma (sigma) 2 Representing variance and u representing mean.
6. A field-adaptive based medical image segmentation method according to claim 1, wherein: the step S5 specifically comprises the following steps:
the decoder is connected with the tag and the characteristic of the encoder after the parameterization, and the characteristic is input into a plurality of continuous convolution blocks to reconstruct images, so that the extraction capacity of the model on the image structure information is improved; each block consists of a convolution layer, an instance normalization layer and an activation layer; the source domain and the target domain have the same generator and all comprise three parts of an encoder, a decoder and a divider, wherein the encoder shares weight; the difference is that the source domain reconstructs the image using the re-parameterized features and segmentation labels, while the target domain does not have labels, and the re-parameterized features and the predicted segmentation results are used to reconstruct the image.
7. A field-adaptive based medical image segmentation method according to claim 1, wherein: the step S6 specifically comprises the following steps:
after the characteristic is subjected to the re-parameterization to remove noise, inputting a re-parameterization result into a divider, and predicting the division result from different scales; the three segmentation results respectively comprise integral structure information and texture detail information of the target region, are complementary, and are fused with prediction results of different scales by using a single convolution layer to serve as final segmentation results.
8. A field-adaptive based medical image segmentation method according to claim 1, wherein: the step S7 specifically comprises the following steps:
step S71, adjusting weighted self-information distribution of a source domain and a target domain based on antagonism entropy minimization by utilizing a discriminator loss optimization model, indirectly minimizing prediction entropy of the target domain, and generating clearer semantic segmentation output and finer object edges; knowing the input source domain image x s Pixel-level probability prediction of (2)Entropy mapping is obtained by means of shannon entropy calculation, as shown in a formula (3):
wherein n is the number of images and c is the number of channels;
step S72, introducing an countermeasure training frame to construct an entropy discriminator D E Mapping E (x) to Ji Shang s ) And E (x) t ) To indirectly minimize entropy by making the entropy distributions of the source domain and the target domain similar; the purpose of the entropy discriminator is to determine whether the entropy diagram is from a source domain or a target domain; the framework uses a GAN network to match entropy distributions of source domain data and target domain data; the discriminator adopts a PatchGan architecture to distinguish 8 multiplied by 8 patch; contains 5 convolution layers, and the feature mapping number of each layer is [64,128,256,512,1 ]]The convolution kernel size is 4×4, and the first layer convolution step length is 1; the step length of each middle convolution layer is 2, and the step length comprises a Relu activation function, and the parameter is 0.2; the last layer uses a sigmoid activation function; the input of the discriminator is a entropy diagram of the source domain and the target domain, the output is 0 or 1 as the domain classification result, 0 represents the entropy diagram from the source domain, and 1 represents the entropy diagram from the target domain.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310632806.1A CN116703850A (en) | 2023-05-30 | 2023-05-30 | Medical image segmentation method based on field self-adaption |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310632806.1A CN116703850A (en) | 2023-05-30 | 2023-05-30 | Medical image segmentation method based on field self-adaption |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116703850A true CN116703850A (en) | 2023-09-05 |
Family
ID=87823209
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310632806.1A Pending CN116703850A (en) | 2023-05-30 | 2023-05-30 | Medical image segmentation method based on field self-adaption |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116703850A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117786823A (en) * | 2024-02-26 | 2024-03-29 | 陕西天润科技股份有限公司 | Light weight processing method based on building monomer model |
-
2023
- 2023-05-30 CN CN202310632806.1A patent/CN116703850A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117786823A (en) * | 2024-02-26 | 2024-03-29 | 陕西天润科技股份有限公司 | Light weight processing method based on building monomer model |
| CN117786823B (en) * | 2024-02-26 | 2024-05-03 | 陕西天润科技股份有限公司 | Light weight processing method based on building monomer model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | Single slice based detection for Alzheimer’s disease via wavelet entropy and multilayer perceptron trained by biogeography-based optimization | |
| CN107886514B (en) | Mammary gland molybdenum target image lump semantic segmentation method based on depth residual error network | |
| JP2022540634A (en) | 3D Point Cloud Object Detection and Instance Segmentation Based on Deep Learning | |
| CN110739070A (en) | brain disease diagnosis method based on 3D convolutional neural network | |
| CN109389585B (en) | Brain tissue extraction method based on full convolution neural network | |
| JP2023550844A (en) | Liver CT automatic segmentation method based on deep shape learning | |
| CN112365980B (en) | Brain tumor multi-target auxiliary diagnosis and prospective treatment evolution visualization method and system | |
| CN114926746A (en) | SAR image change detection method based on multi-scale differential feature attention mechanism | |
| CN113496495B (en) | Medical image segmentation model building method capable of realizing missing input and segmentation method | |
| CN110859624A (en) | Brain age deep learning prediction system based on structural magnetic resonance image | |
| CN115511767B (en) | Self-supervised learning multi-modal image fusion method and application thereof | |
| CN116703850A (en) | Medical image segmentation method based on field self-adaption | |
| CN115375711A (en) | Image segmentation method of global context attention network based on multi-scale fusion | |
| CN115496720A (en) | Gastrointestinal cancer pathological image segmentation method based on ViT mechanism model and related equipment | |
| CN113781461A (en) | Intelligent patient monitoring and sequencing method | |
| CN114743037A (en) | Deep medical image clustering method based on multi-scale structure learning | |
| CN117274599A (en) | Brain magnetic resonance segmentation method and system based on combined double-task self-encoder | |
| CN113870327B (en) | Medical image registration method based on prediction multi-level deformation field | |
| CN112990359B (en) | Image data processing method, device, computer and storage medium | |
| CN112908465A (en) | Ultrasonic key frame automatic identification method based on anomaly detection and semi-supervision | |
| CN111755131A (en) | COVID-19 early screening and severity degree evaluation method and system based on attention guidance | |
| Goutham et al. | Brain tumor classification using Efficientnet-B0 model | |
| CN116091412A (en) | Method for segmenting tumor from PET/CT image | |
| CN116309640A (en) | Image automatic segmentation method based on multi-level multi-attention MLMA-UNet network | |
| Liu et al. | Segmentation for multimodal brain tumor images using dual-tree complex wavelet transform and deep reinforcement learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |