CN114240966B - Self-supervision learning method for 3D medical image segmentation training feature extractor - Google Patents
Self-supervision learning method for 3D medical image segmentation training feature extractor Download PDFInfo
- Publication number
- CN114240966B CN114240966B CN202111523320.1A CN202111523320A CN114240966B CN 114240966 B CN114240966 B CN 114240966B CN 202111523320 A CN202111523320 A CN 202111523320A CN 114240966 B CN114240966 B CN 114240966B
- Authority
- CN
- China
- Prior art keywords
- image
- augmentation
- path module
- module
- image blocks
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012549 training Methods 0.000 title claims abstract description 34
- 238000003709 image segmentation Methods 0.000 title claims abstract description 32
- 238000000034 method Methods 0.000 title claims abstract description 19
- 238000013434 data augmentation Methods 0.000 claims abstract description 26
- 230000003416 augmentation Effects 0.000 claims description 33
- 230000006870 function Effects 0.000 claims description 13
- 230000003190 augmentative effect Effects 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 5
- 230000011218 segmentation Effects 0.000 claims description 5
- 230000003321 amplification Effects 0.000 claims description 4
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 4
- 230000000694 effects Effects 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000000605 extraction Methods 0.000 abstract description 5
- 238000013459 approach Methods 0.000 description 2
- 238000012512 characterization method Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a self-supervision learning method for a 3D medical image segmentation training feature extractor, which comprises three parts of data augmentation, an online path and a target path; giving an input image, firstly randomly extracting two image blocks from the input image, wherein the two image blocks are overlapped in a certain proportion; then, respectively carrying out different data augmentation on the two image blocks, and then respectively inputting the two image blocks into an online path and a target path; since the data augmentation performed by the two image blocks is known, the relative position correspondence of the two image blocks at the pixel level in the feature space is easy to obtain; each pair of matched pixels in the two image blocks represents the same local area in the original image, and the model judges that the corresponding local characteristics in the two image blocks are consistent; through the constraint of local consistency, the model can learn the feature extraction capability of finer granularity in the 3D medical image data, and can preserve the spatial structural relationship inside the image.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a self-supervision learning method.
Background
In recent years, a method based on a deep convolutional neural network has achieved great success in a 3D image segmentation task. In deep learning, in order to ensure that a model obtained by training has higher accuracy and generalization, a large amount of annotation data is generally required to be provided. However, in the medical image field, the segmentation and labeling of the 3D image are time-consuming and labor-consuming, and the amount of available labels is very limited. Unlabeled 3D medical image data is relatively more readily available and the data is more voluminous. Self-supervision learning is one of the very hot research directions in the field of artificial intelligence in recent years, and by using large-scale unmarked data, the deep learning model can automatically learn the characteristic characterization capability from massive data without any expert annotation under the self-defined supervision task. For example, paper Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning published in conference 34th Conference on Neural Information Processing Systems is a self-supervised learning method for keeping information obtained by a model consistent even when an input image is transformed differently, and feature extractors obtained by training The method show very excellent performance in downstream tasks such as image segmentation, classification and detection.
However, current self-supervised learning approaches are basically designed based on global consistency constraints of images, i.e. the model is required to consider the information obtained from two different perspectives of the same image to be consistent. The consistency on the whole image level has global property, so that the spatial structure relation inside the image in the model is destroyed to a certain extent, and the downstream intensive prediction task such as 3D medical image segmentation is not facilitated.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a self-supervision learning method aiming at a 3D medical image segmentation training feature extractor, which comprises three parts of data augmentation, an online path and a target path; giving an input image, firstly randomly extracting two image blocks from the input image, wherein the two image blocks are overlapped in a certain proportion; then, respectively carrying out different data augmentation on the two image blocks, and then respectively inputting the two image blocks into an online path and a target path; since the data augmentation performed by the two image blocks is known, the relative position correspondence of the two image blocks at the pixel level in the feature space is easy to obtain; each pair of matched pixels in the two image blocks represents the same local area in the original image, and the model judges that the corresponding local characteristics in the two image blocks are consistent; through the constraint of local consistency, the model can learn the feature extraction capability of finer granularity in the 3D medical image data, and can preserve the spatial structural relationship inside the image.
The technical scheme adopted by the invention for solving the technical problems comprises the following steps:
step 1: constructing a data augmentation model;
step 1-1: definition data augmentation module
Data augmentation moduleIncluding 6 augmentation modes: (1) overturning: randomly flipped with 50% probability in 3 axes of the 3D image; (2) clipping and scaling: cutting the image blocks randomly, interpolating the cut image blocks, and recovering the size to be the size before cutting; (3) Gaussian noise is added with 10% probability, variance from [0, a]Randomly selecting seeds; (4) Gaussian blur with 20% probability, sigma of gaussian kernel is from [ b,1]Randomly selecting seeds; (5) adjusting brightness and contrast with 50% probability; (6) The voxel-by-voxel gamma transformation is performed with 50% probability as follows:
wherein i is new Representing the voxel values after gamma conversion, i old Representing voxel values before gamma conversion; lambda is from [0.7,1.5 ]]Randomly sampled values within the range; normalizing and adjusting the transformed voxel value to [0,1 ]]；
Step 1-2: given a label-free 3D medical image training sample x, extracting two image blocks containing overlapping areas from the x; slave data augmentation moduleTwo different combinations of augmentation modes are randomly selected, two different augmentation modes are respectively carried out on the two image blocks extracted in x, and the image blocks after augmentation are respectively expressed as x 1 And x 2 The method comprises the steps of carrying out a first treatment on the surface of the Note that if the (2) th augmentation mode is selected, the overlapping area of the two image blocks is completely reserved during clipping;
step 2: constructing an online path module and a target path module;
the online path module and the target path module are parallel and respectively comprise an encoder, a mapper and a space position regulator in sequence;
encoders of the online path module and the target path module both adopt 3D ResNet50 network structures for extracting the amplified image block x 1 And x 2 Is a feature of the image;
the mappers of the online path module and the target path module are all of a convolution layer-batch normalization layer-activation function layer-convolution layer structure and are used for mapping the output of each encoder to a feature space; the outputs of the online path module and the target path module mappers are f respectively 1 And f 2 ；
The spatial position adjusters of the online path module and the target path module firstly respectively and sequentially align two features, and the specific operation is as follows:
(1) Mirror image adjustment: for the (1) th augmentation mode, for f 1 And f 2 Respectively turn τ 1 And τ 2 To the inverse operation of (a) to make f after inverse amplification 1 And f 2 I.e.And->Consistent with the position in the original image:
wherein,and->Refers to turning tau 1 And τ 2 Is the reverse of the above;
if the mode of augmentation to the input image block in step 1-2 is not the (1) th mode of augmentation, then:
(2) ROI adjustment: for the (2) th augmentation mode, first, from the known augmented location informationAnd->Extracting features of the overlapping region, and then adjusting the overlapping region to be respectively matched with f by bilinear interpolation 1 And f 2 Is of the same size, the resulting aligned ROI is characterized as +.>And->Wherein F represents flip, CS represents clipping and scaling;
if the mode of augmentation to the input image block in step 1-2 is not the (2) th mode of augmentation, then:
the online path module is a mapper after the two features are aligned; the target path module has no mapper;
step 3: pre-training a self-supervision learning model by utilizing local consistency loss;
the self-supervision learning model comprises a data augmentation model, an online path module and a target path module;
designing a loss function based on local consistency constraints when x 1 ,x 2 When the online path and the target path are respectively input:
wherein Y is θ A mapper representing the on-line path,is L in the channel dimension of the feature 2 Normalizing, wherein N represents the input quantity, and H, W and D represent the height, width and depth of the feature respectively;
when x is 1 ,x 2 The input target path and the on-line path respectively also meet the local consistency constraint (1), namely L local (x 2 ,x 1 )；
Thus, the overall satisfied local consistency constraint is:
L total (x 1 ,x 2 )＝L local (x 1 ,x 2 )+L local (x 2 ,x 1 ) (2)
pre-training the self-supervision learning model by adopting the loss function of the step (2) until the pre-training is completed;
step 4: constructing a 3D medical image segmentation network;
the 3D medical image segmentation network comprises three parts in sequence: a 3d resnet50 structured encoder, a spatial pyramid pooling module and a decoder; wherein three jump connections are also included between the encoder and decoder;
the encoder with the 3D ResNet50 structure is a pre-trained encoder in a self-supervision learning model online path; the space pyramid pooling module comprises 3 separable convolution layers, and the expansion rates are 2,4 and 8 respectively; the decoder comprises 4 deconvolution blocks and one convolution layer for segmentation;
step 5: verifying the effect of self-supervision pre-training on 3D medical image segmentation;
augmenting a 3D medical image segmentation dataset;
the optimizer used in the network training is a random gradient descent optimizer, and the loss function is a combination of the Dice similarity coefficient loss and the cross entropy loss;
and 3D medical image segmentation network training is completed, and 3D medical image segmentation is realized.
Preferably, a=0.1 and b=0.5.
The beneficial effects of the invention are as follows:
the existing self-supervision learning method is basically designed based on global consistency constraint of images, so that the feature extractor obtained through training can easily ignore the spatial structure relationship in the images and only pay attention to global image features, and is not suitable for downstream tasks of dense prediction such as 3D medical image segmentation. The invention provides a self-supervision learning method based on local consistency constraint, which can lead a model to pay attention to the spatial structure relation in an image and learn the feature extraction capacity with finer granularity, so that the feature extractor obtained by training is more suitable for downstream 3D medical image segmentation tasks.
Drawings
FIG. 1 is a schematic block diagram of the method of the present invention.
Detailed Description
The invention will be further described with reference to the drawings and examples.
The existing self-supervision learning task is basically designed based on global consistency constraint of images, which ensures consistency on the whole image level, but damages the spatial structure relationship inside the images in the model to a certain extent, so that the learned feature characterization does not completely fit the dense prediction task such as 3D medical image segmentation.
In order to better learn the spatial structure relationship in the image, the invention provides a self-supervision learning method based on local consistency constraint. The method comprises three parts of data augmentation, online path and target path. Given an input image, two image blocks are first randomly extracted from the input image, and in order to ensure the existence of a later constraint relationship, a certain proportion of overlapping of the two image blocks is required. And then respectively carrying out different data augmentation, such as rotation, scaling, mirror symmetry and the like, on the two image blocks, and then respectively inputting the two image blocks subjected to the data augmentation into an online path and a target path. Because different data augmentation modes are used, two image blocks do not have one-to-one corresponding spatial position relationship, and therefore the characteristics obtained by the two paths are difficult to directly match at the pixel level. However, since the data augmentation by two image blocks is known, the relative positional correspondence of the two image blocks at the pixel level in the feature space is readily obtained. Each pair of matched pixels in the two image blocks characterizes the same local region in the original image, so the model is required to consider that the corresponding local features in the two image blocks should remain consistent. Through the constraint of local consistency, the model can automatically learn the feature extraction capability of finer granularity in the 3D medical image data, and can preserve the spatial structural relationship inside the image.
Given a label-free training sample x, firstly extracting two image blocks with a certain overlapping area from x, and inputting the image blocks into a data amplification moduleThe two image blocks are amplified in two different amplifying modes, and the amplified image blocks are respectively expressed as x 1 And x 2 Then respectively X 1 And x 2 The input to the online path module and the target path module are used for feature extraction and feature alignment. The features extracted by the online path module and the target path module are respectively f 1 And f 2 To represent. Since the way of augmentation is known, it is easy to apply to f 1 And f 2 Alignment at the pixel level is performed. The present model introduces local consistency loss so that the aligned features become consistent in the optimization. After training, only an encoder (namely a feature extractor) of the online path module is needed to build a segmentation model of the 3D medical image segmentation task.
A self-supervised learning method for a 3D medical image segmentation training feature extractor, comprising the steps of:
step 1: constructing a data augmentation model;
step 1-1: definition data augmentation module
Data augmentation moduleIncluding 6 augmentation modes: (1) overturning: randomly flipped with 50% probability in 3 axes of the 3D image; (2) clipping and scaling: cutting the image blocks randomly, interpolating the cut image blocks, and recovering the size to be the size before cutting; (3) Gaussian noise is added with 10% probability, variance from [0,0.1 ]]Randomly selecting seeds; (4) Gaussian blur is performed with 20% probability, and σ of the Gaussian kernel is from [0.5,1]Randomly selecting seeds; (5) adjusting brightness and contrast with 50% probability; (6) The voxel-by-voxel gamma transformation is performed with 50% probability as follows:
wherein i is new Representing the voxel values after gamma conversion, i old Representing voxel values before gamma conversion; lambda is from [0.7,1.5 ]]Randomly sampled values within the range; normalizing and adjusting the transformed voxel value to [0,1 ]]；
Step 1-2: given a label-free 3D medical image training sample x, extracting two image blocks containing overlapping areas from the x; slave data augmentation moduleTwo different combinations of augmentation modes are randomly selected, two different augmentation modes are respectively carried out on the two image blocks extracted in x, and the image blocks after augmentation are respectively expressed as x 1 And x 2 The method comprises the steps of carrying out a first treatment on the surface of the Note that if the (2) th augmentation mode is selected, the overlapping area of the two image blocks is completely reserved during clipping;
step 2: constructing an online path module and a target path module;
the online path module and the target path module are parallel and respectively comprise an encoder, a mapper and a space position regulator in sequence;
encoders of the online path module and the target path module both adopt 3D ResNet50 network structures for extracting the amplified image block x 1 And x 2 Is a feature of the image;
the mappers of the online path module and the target path module are all of a convolution layer-batch normalization layer-activation function layer-convolution layer structure and are used for mapping the output of each encoder to a feature space; the outputs of the online path module and the target path module mappers are f respectively 1 And f 2 ；
The spatial position adjusters of the online path module and the target path module firstly respectively and sequentially align two features, and the specific operation is as follows:
(1) Mirror image adjustment: for the (1) th augmentation mode, for f 1 And f 2 Respectively turn τ 1 And τ 2 To the inverse operation of (a) to make f after inverse amplification 1 And f 2 I.e.And->Consistent with the position in the original image:
wherein,and->Refers to turning tau 1 And τ 2 Is the reverse of the above;
if the mode of augmentation to the input image block in step 1-2 is not the (1) th mode of augmentation, then:
(2) ROI adjustment: for the (2) th augmentation mode, first, from the known augmented location informationAnd->Extracting features of the overlapping region, and then adjusting the overlapping region to be respectively matched with f by bilinear interpolation 1 And f 2 Is of the same size, the resulting aligned ROI is characterized as +.>And->Wherein F represents flip, CS represents clipping and scaling;
if the mode of augmentation to the input image block in step 1-2 is not the (2) th mode of augmentation, then:
the online path module is a mapper after the two features are aligned; the target path module has no mapper;
step 3: pre-training a self-supervision learning model by utilizing local consistency loss;
the self-supervision learning model comprises a data augmentation model, an online path module and a target path module;
designing a loss function based on local consistency constraints when x 1 ,x 2 When the online path and the target path are respectively input:
wherein Y is θ Representing an online pathThe mapping means may be a function of the mapping means,is L in the channel dimension of the feature 2 Normalizing, wherein N represents the input quantity, and H, W and D represent the height, width and depth of the feature respectively;
when x is 1 ,x 2 The input target path and the on-line path respectively also meet the local consistency constraint (1), namely L local (x 2 ,x 1 )；
Thus, the overall satisfied local consistency constraint is:
L total (x 1 ,x 2 )＝L local (x 1 ,x 2 )+L local (x 2 ,x 1 ) (2)
pre-training the self-supervision learning model by adopting the loss function of the step (2) until the pre-training is completed;
step 4: constructing a 3D medical image segmentation network;
the 3D medical image segmentation network comprises three parts in sequence: a 3d resnet50 structured encoder, a spatial pyramid pooling module and a decoder; wherein three jump connections are also included between the encoder and decoder;
the encoder with the 3D ResNet50 structure is a pre-trained encoder in a self-supervision learning model online path; the space pyramid pooling module comprises 3 separable convolution layers, and the expansion rates are 2,4 and 8 respectively; the decoder comprises 4 deconvolution blocks and one convolution layer for segmentation;
step 5: verifying the effect of self-supervision pre-training on 3D medical image segmentation;
augmenting a 3D medical image segmentation dataset;
the optimizer used in the network training is a random gradient descent optimizer, and the loss function is a combination of the Dice similarity coefficient loss and the cross entropy loss;
and 3D medical image segmentation network training is completed, and 3D medical image segmentation is realized.
Claims (2)
1. A self-supervised learning method for a 3D medical image segmentation training feature extractor, comprising the steps of:
step 1: constructing a data augmentation model;
step 1-1: definition data augmentation module
Data augmentation moduleIncluding 6 augmentation modes: (1) overturning: randomly flipped with 50% probability in 3 axes of the 3D image; (2) clipping and scaling: cutting the image blocks randomly, interpolating the cut image blocks, and recovering the size to be the size before cutting; (3) Gaussian noise is added with 10% probability, variance from [0, a]Randomly selecting seeds; (4) Gaussian blur with 20% probability, sigma of gaussian kernel is from [ b,1]Randomly selecting seeds; (5) adjusting brightness and contrast with 50% probability; (6) The voxel-by-voxel gamma transformation is performed with 50% probability as follows:
wherein i is new Representing the voxel values after gamma conversion, i oid Representing voxel values before gamma conversion; lambda is from [0.7,1.5 ]]Randomly sampled values within the range; normalizing and adjusting the transformed voxel value to [0,1 ]]；
Step 1-2: given a label-free 3D medical image training sample x, extracting two image blocks containing overlapping areas from the x; slave data augmentation moduleTwo different combinations of augmentation modes are selected randomly, and two different augmentation modes are respectively carried out on the two image blocks extracted in the x, so that the image blocks are augmentedThe image blocks after are respectively denoted as x 1 And x 2 The method comprises the steps of carrying out a first treatment on the surface of the Note that if the (2) th augmentation mode is selected, the overlapping area of the two image blocks is completely reserved during clipping;
step 2: constructing an online path module and a target path module;
the online path module and the target path module are parallel and respectively comprise an encoder, a mapper and a space position regulator in sequence;
encoders of the online path module and the target path module both adopt 3D ResNet50 network structures for extracting the amplified image block x 1 And x 2 Is a feature of the image;
the mappers of the online path module and the target path module are all of a convolution layer-batch normalization layer-activation function layer-convolution layer structure and are used for mapping the output of each encoder to a feature space; the outputs of the online path module and the target path module mappers are f respectively 1 And f 2 ；
The spatial position adjusters of the online path module and the target path module firstly respectively and sequentially align two features, and the specific operation is as follows:
(1) Mirror image adjustment: for the (1) th augmentation mode, for f 1 And f 2 Respectively turn τ 1 And τ 2 To the inverse operation of (a) to make f after inverse amplification 1 And f 2 I.e.And->Consistent with the position in the original image:
wherein,and->Refers to turning tau 1 And τ 2 Is the reverse of the above;
if the mode of augmentation to the input image block in step 1-2 is not the (1) th mode of augmentation, then:
(2) ROI adjustment: for the (2) th augmentation mode, first, from the L-known augmented location informationAnd->Extracting features of the overlapping region, and then adjusting the overlapping region to be respectively matched with f by bilinear interpolation 1 And f 2 Is of the same size, the resulting aligned ROI is characterized as +.>And->Wherein F represents flip, CS represents clipping and scaling;
if the mode of augmentation to the input image block in step 1-2 is not the (2) th mode of augmentation, then:
the online path module is a mapper after the two features are aligned; the target path module has no mapper;
step 3: pre-training a self-supervision learning model by utilizing local consistency loss;
the self-supervision learning model comprises a data augmentation model, an online path module and a target path module;
designing a loss function based on local consistency constraints when x 1 ，x 2 When the online path and the target path are respectively input:
wherein Y is θ A mapper representing the on-line path,is L in the channel dimension of the feature 2 Normalizing, wherein N represents the input quantity, and H, W and D represent the height, width and depth of the feature respectively;
when x is 1 ，x 2 The input target path and the on-line path respectively also meet the local consistency constraint (1), namely L local (x 2 ，x 1 )；
Thus, the overall satisfied local consistency constraint is:
L total (x 1 ，x 2 )＝L local (x 1 ，x 2 )+L local (x 2 ，x 1 ) (2)
pre-training the self-supervision learning model by adopting the loss function of the step (2) until the pre-training is completed;
step 4: constructing a 3D medical image segmentation network;
the 3D medical image segmentation network comprises three parts in sequence: a 3d resnet50 structured encoder, a spatial pyramid pooling module and a decoder; wherein three jump connections are also included between the encoder and decoder;
the encoder with the 3D ResNet50 structure is a pre-trained encoder in a self-supervision learning model online path; the space pyramid pooling module comprises 3 separable convolution layers, and the expansion rates are 2,4 and 8 respectively; the decoder comprises 4 deconvolution blocks and one convolution layer for segmentation;
step 5: verifying the effect of self-supervision pre-training on 3D medical image segmentation;
augmenting a 3D medical image segmentation dataset;
the optimizer used in the network training is a random gradient descent optimizer, and the loss function is a combination of the Dice similarity coefficient loss and the cross entropy loss;
and 3D medical image segmentation network training is completed, and 3D medical image segmentation is realized.
2. A self-supervised learning method for a 3D medical image segmentation training feature extractor as defined in claim 1, wherein a = 0.1 and b = 0.5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111523320.1A CN114240966B (en) | 2021-12-13 | 2021-12-13 | Self-supervision learning method for 3D medical image segmentation training feature extractor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111523320.1A CN114240966B (en) | 2021-12-13 | 2021-12-13 | Self-supervision learning method for 3D medical image segmentation training feature extractor |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114240966A CN114240966A (en) | 2022-03-25 |
| CN114240966B true CN114240966B (en) | 2024-03-15 |
Family
ID=80755575
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111523320.1A Active CN114240966B (en) | 2021-12-13 | 2021-12-13 | Self-supervision learning method for 3D medical image segmentation training feature extractor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114240966B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2020103905A4 (en) * | 2020-12-04 | 2021-02-11 | Chongqing Normal University | Unsupervised cross-domain self-adaptive medical image segmentation method based on deep adversarial learning |
| CN112686898A (en) * | 2021-03-15 | 2021-04-20 | 四川大学 | Automatic radiotherapy target area segmentation method based on self-supervision learning |
| CN113011427A (en) * | 2021-03-17 | 2021-06-22 | 中南大学 | Remote sensing image semantic segmentation method based on self-supervision contrast learning |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11354778B2 (en) * | 2020-04-13 | 2022-06-07 | Google Llc | Systems and methods for contrastive learning of visual representations |
-
2021
- 2021-12-13 CN CN202111523320.1A patent/CN114240966B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2020103905A4 (en) * | 2020-12-04 | 2021-02-11 | Chongqing Normal University | Unsupervised cross-domain self-adaptive medical image segmentation method based on deep adversarial learning |
| CN112686898A (en) * | 2021-03-15 | 2021-04-20 | 四川大学 | Automatic radiotherapy target area segmentation method based on self-supervision learning |
| CN113011427A (en) * | 2021-03-17 | 2021-06-22 | 中南大学 | Remote sensing image semantic segmentation method based on self-supervision contrast learning |
Non-Patent Citations (2)
| Title |
|---|
| 基于自监督学习的番茄植株图像深度估计方法;周云成;许童羽;邓寒冰;苗腾;吴琼;;农业工程学报(24);全文 * |
| 面向乳腺超声图像分割的混合监督双通道反馈U-Net;贡荣麟;施俊;王骏;;中国图象图形学报(10);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114240966A (en) | 2022-03-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Jiang et al. | Edge-enhanced GAN for remote sensing image superresolution | |
| Guo et al. | Scene-driven multitask parallel attention network for building extraction in high-resolution remote sensing images | |
| CN110210551B (en) | Visual target tracking method based on adaptive subject sensitivity | |
| CN111986099B (en) | Tillage monitoring method and system based on convolutional neural network with residual error correction fused | |
| Zhang et al. | Remote sensing image spatiotemporal fusion using a generative adversarial network | |
| CN110706157B (en) | Face super-resolution reconstruction method for generating confrontation network based on identity prior | |
| CN110276354B (en) | High-resolution streetscape picture semantic segmentation training and real-time segmentation method | |
| CN112488210A (en) | Three-dimensional point cloud automatic classification method based on graph convolution neural network | |
| CN114202672A (en) | Small target detection method based on attention mechanism | |
| Guo et al. | Super-resolution integrated building semantic segmentation for multi-source remote sensing imagery | |
| CN113902915A (en) | Semantic segmentation method and system based on low-illumination complex road scene | |
| CN113780149A (en) | Method for efficiently extracting building target of remote sensing image based on attention mechanism | |
| CN111640116B (en) | Aerial photography graph building segmentation method and device based on deep convolutional residual error network | |
| CN114519819B (en) | Remote sensing image target detection method based on global context awareness | |
| CN110633640A (en) | Method for identifying complex scene by optimizing PointNet | |
| Li et al. | An aerial image segmentation approach based on enhanced multi-scale convolutional neural network | |
| CN110751271B (en) | Image traceability feature characterization method based on deep neural network | |
| Liu et al. | Image-free single-pixel segmentation | |
| CN116503602A (en) | Unstructured environment three-dimensional point cloud semantic segmentation method based on multi-level edge enhancement | |
| CN116596966A (en) | Segmentation and tracking method based on attention and feature fusion | |
| Neupane et al. | Building footprint segmentation using transfer learning: a case study of the city of melbourne | |
| Zhao et al. | Squnet: An high-performance network for crater detection with dem data | |
| CN112785629A (en) | Aurora motion characterization method based on unsupervised deep optical flow network | |
| Sulaiman et al. | Building Precision: Efficient Encoder-Decoder Networks for Remote Sensing based on Aerial RGB and LiDAR data | |
| CN114240966B (en) | Self-supervision learning method for 3D medical image segmentation training feature extractor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |