CN112308833A - One-shot brain image segmentation method based on circular consistent correlation - Google Patents

One-shot brain image segmentation method based on circular consistent correlation Download PDF

Info

Publication number
CN112308833A
CN112308833A CN202011182378.XA CN202011182378A CN112308833A CN 112308833 A CN112308833 A CN 112308833A CN 202011182378 A CN202011182378 A CN 202011182378A CN 112308833 A CN112308833 A CN 112308833A
Authority
CN
China
Prior art keywords
image
generator
module
mapping
network model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011182378.XA
Other languages
Chinese (zh)
Other versions
CN112308833B (en
Inventor
王连生
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xiamen University
Original Assignee
Xiamen University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xiamen University filed Critical Xiamen University
Priority to CN202011182378.XA priority Critical patent/CN112308833B/en
Publication of CN112308833A publication Critical patent/CN112308833A/en
Application granted granted Critical
Publication of CN112308833B publication Critical patent/CN112308833B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30016—Brain

Abstract

The invention discloses a one-shot brain image segmentation method based on circular consistent correlation, which comprises the following steps of: s1, obtaining and classifying the brain anatomical structure image to obtain an unlabeled image y and an atlas x, wherein the atlas x has an label xs(ii) a S2, constructing an LT-NET network model, wherein the LT-NET network model comprises a generator GFGenerator GBAnd two discriminators D; s3, input the image set x and the unlabelled image y into the generator GFTo obtain a forward mapping Δ pF(ii) a S4, mapping the forward direction to delta pFApplied to the atlas x and the Label x, respectivelysObtaining a reconstructed image in accordance with the supervision loss
Figure DDA0002750540510000011
And labeling
Figure DDA0002750540510000012
S5, reconstructing the image
Figure DDA0002750540510000013
And atlas x input generator GBGet backward mapping Δ pB(ii) a S6, mapping the backward direction to delta pBAre applied to the reconstructed images respectively
Figure DDA0002750540510000014
And labeling
Figure DDA0002750540510000015
Obtaining a reconstructed image in cooperation with the supervision loss
Figure DDA0002750540510000016
And labeling
Figure DDA0002750540510000017
The method adopts the LT-NET network model to be constructed, the efficiency of image segmentation is effectively improved by matching with supervision loss, and the unidirectional correlation learning performance is improved.

Description

One-shot brain image segmentation method based on circular consistent correlation
Technical Field
The invention relates to the technical field of medical image segmentation, in particular to a one-shot brain image segmentation method based on circular consistent correlation.
Background
Common methods for brain anatomy segmentation are segmentation by traditional machine learning, which relies on manually extracted features with limited feature representation and generalization capabilities, and Convolutional Neural Network (CNN) learning was developed because it is completely data-driven and can automatically retrieve hierarchical features using self-learned advanced features, eliminating the limitations of manual features in traditional machine learning methods, with sufficient labeled data, convolutional neural network has a better effect in a fully supervised segmentation task, using segmentation algorithms with forward and backward correlations, i.e. improving the segmentation network to learn the forward mapping of atlas x to unlabeled image y, through warp manipulation to reconstructed images
Figure BDA0002750540490000011
Subsequent learning of reconstructed images
Figure BDA0002750540490000012
The backward mapping to the atlas x can play a positive role in the forward correlation learning and the final segmentation result, but the learning mode has the defects of difficult efficiency meeting the requirement and low performance of the unidirectional correlation learning.
Disclosure of Invention
The invention aims to provide a one-shot brain image segmentation method based on circular consistent correlation, which effectively improves the efficiency of image segmentation and improves the learning performance of unidirectional correlation.
In order to achieve the purpose, the invention adopts the following technical scheme:
a one-shot brain image segmentation method based on circular consistent correlation comprises the following steps:
s1, obtaining and classifying the brain anatomical structure image to obtain a labeled image and an unlabeled image y, and dividing the labeled image into an atlas x with a label xs；
S2, constructing an LT-NET network model, wherein the LT-NET network model comprises a generator GFGenerator GBAnd two discriminators D, generators GFAnd generator GBBoth comprise twin encoders and decoders;
s3, input the image set x and the unlabelled image y into the generator GFBy means of a generator GFIs processed by the twin encoder and decoder to obtain the forward mapping deltapF；
S4, mapping the forward direction to delta pFApplied to the atlas x and the Label x, respectivelysBy means of a discriminator D and a generator GFPerforming cyclic countermeasure in coordination with supervision loss and obtaining reconstructed images through warp operation
Figure BDA0002750540490000021
And labeling
Figure BDA0002750540490000022
S5, reconstructing the image
Figure BDA0002750540490000023
And atlas x input generator GBBy means of a generator GBProcessed by the twin encoder and decoder to obtain a backward mapping Δ pB；
S6, mapping the backward direction to delta pBAre applied to the reconstructed images respectively
Figure BDA0002750540490000024
And labeling
Figure BDA0002750540490000025
By means of a discriminator D and a generator GBPerforming cyclic countermeasure in coordination with supervision loss and obtaining reconstructed images through warp operation
Figure BDA0002750540490000026
And labeling
Figure BDA0002750540490000027
Further, the surveillance loss includes an image consistency loss
Figure BDA0002750540490000028
Transformation consistency Ltran_cyc(ΔpF，ΔpB)Anatomically consistent
Figure BDA0002750540490000029
And anatomical difference consistency
Figure BDA00027505404900000210
Figure BDA00027505404900000211
The formulas are respectively expressed as:
Figure BDA00027505404900000212
Figure BDA00027505404900000213
Figure BDA00027505404900000214
Figure BDA00027505404900000215
wherein the content of the first and second substances,
Figure BDA00027505404900000216
represented as for supervised reconstruction of images
Figure BDA00027505404900000217
Loss of image consistency, L, consistent with atlas xtran_cyc(ΔpF，ΔpB)Denoted as Δ p for supervised forward mappingFAnd backward mapping Δ pBIs equal to (x), where ρ (x) is (x)2+ε2)γRepresenting a generalized charbonier penalty, t represents a voxel point, e is 0.001, y is 0.45,
Figure BDA00027505404900000218
and
Figure BDA00027505404900000219
respectively denoted as x for supervision of the annotationsAnd label
Figure BDA00027505404900000220
And labeling
Figure BDA00027505404900000221
Anatomical consistency and anatomical variance consistency.
Further, the supervisory loss further includes antagonistic loss
Figure BDA00027505404900000222
Loss of similarity
Figure BDA00027505404900000223
And a smoothing loss Lsmooth(ΔpF，ΔpB)The formulas are respectively expressed as:
Figure BDA0002750540490000031
Figure BDA0002750540490000032
Lsmooth(ΔpF,ΔpB)＝Lsmooth(ΔpF)+Lsmooth(ΔpB)
Figure BDA0002750540490000033
Figure BDA0002750540490000034
wherein f isy(t) and
Figure BDA0002750540490000035
respectively representing the unmarked image y and the reconstructed image
Figure BDA0002750540490000036
Local average intensity:
Figure BDA0002750540490000037
ti denotes a volume around t of l3Coordinates within the range, l ═ 3; te Ω represents all the position spaces in Δ p, Lsmooth (ΔpF，ΔpB)Expressed using the spatial gradient between adjacent pixels in the x, y, z direction
Figure BDA0002750540490000038
Further, the total supervision loss of the LT-NET network model is as follows:
Figure BDA0002750540490000039
wherein λ is1,、λ2、λ3Is a measure of the weight lost, λ1＝1，λ2＝3，λ3＝10。
Further, the generator GFAnd generator GBThe system also comprises a double attention module which extracts the atlas x and the unmarked image y/reconstructed image by the split flow of the twin encoder
Figure BDA00027505404900000310
Inputting the related features into a double-attention module, learning the spatial information and channel information of the related features by the double-attention module, and transmitting the spatial information and channel information to a decoder, and decoding by the decoder to obtain forward mapping delta pFBackward mapping Δ pB。
Furthermore, the double-attention module comprises a space attention module and a channel module, the space attention module captures space information in a space dimension, the channel module captures channel information in a channel dimension, and the space information and the channel information are added to obtain a new feature map and are transmitted to the decoder.
Furthermore, the twin encoder is provided with 5 encoding sub-modules which are respectively a first encoding sub-module, a second encoding sub-module, a third encoding sub-module, a fourth encoding sub-module and a fifth encoding sub-module, and the two encoding sub-modules form 1 processing stream; the twin encoder has 2 processing streams and is simultaneously connected with 1 fifth encoding submodule;
the decoder is provided with 5 decoding sub-modules which are respectively a first decoding sub-module, a second decoding sub-module, a third decoding sub-module, a fourth decoding sub-module and a fifth decoding sub-module; the first decoding submodule is connected with the double-attention module, the second decoding submodule receives the first decoding submodule and is respectively in long connection with the fourth coding submodule of 2 processing streams, the third decoding submodule receives the second decoding submodule and is respectively in long connection with the third coding submodule of 2 processing streams, the fourth decoding submodule receives the third decoding submodule and is respectively in long connection with the second coding submodule of 2 processing streams, the fifth decoding submodule receives the fourth decoding submodule, and the fifth decoding submodule outputs a forward mapping delta p from an image set x to an unmarked image yFReconstruction of images
Figure BDA0002750540490000041
Backward mapping Δ p to atlas xB。
Further, the coding sub-module is composed of ResNet-34 stacked by basic residual modules.
Further, the arbiter D adopts a PatchGAN arbiter.
After adopting the technical scheme, compared with the background technology, the invention has the following advantages:
1. the invention constructs LT-NET network model and generates GFGenerator GBRespectively confronted with a discriminator D, the LT-NET network model has supervision loss, and the supervision loss is matched with a generator GFGenerator GBLet the generator GFGenerator GBRespectively obtain forward mapping delta p with highest accuracyFAnd backward mapping Δ pBForward mapping of Δ pFAnd backward mapping Δ pBThe reconstructed images can be obtained by performing warp operation respectively
Figure BDA0002750540490000042
Labeling
Figure BDA0002750540490000043
Reconstructing an image
Figure BDA0002750540490000044
And labeling
Figure BDA0002750540490000045
The LT-NET network model effectively improves the image segmentation efficiency, improves the unidirectional correlation learning performance, and obtains reconstructed images and labels with higher accuracy through learning.
2. The discriminator D selects the PatchGAN discriminator, the PatchGAN discriminator can better discriminate the local part of the image, each patch is judged whether the image is true or false by dividing the image into a plurality of patches, and finally the judgment of the image level is obtained, so that the accuracy and the performance are superior to those of a common discriminator.
3. The present invention supervises losses including loss of image consistency
Figure BDA0002750540490000046
Transformation consistency Ltran_cyc(ΔpF，ΔpB)Anatomically consistent
Figure BDA0002750540490000047
Anatomical disparity consistency
Figure BDA0002750540490000048
Loss of antagonism
Figure BDA0002750540490000049
Loss of similarity
Figure BDA00027505404900000410
And a smoothing loss Lsmooth(ΔpF，ΔpB)The LT-NET network model is supervised in different spaces through different supervision losses, so that the LT-NET network model effectively improves the image segmentation efficiency and improves the unidirectional correlation learning performance.
Drawings
FIG. 1 is a schematic flow chart of the present invention;
FIG. 2 is a schematic diagram of the LT-NET network model structure of the present invention;
FIG. 3 is a schematic diagram of the operation of a twin encoder and decoder according to the present invention;
FIG. 4 is a graph showing the segmentation and comparison of the LT-NET network model and the MABMIS network model according to the present invention;
FIG. 5 is a graph showing a segmentation comparison of the LT-NET network model and the PICSL-MALF network model according to the present invention;
FIG. 6 is a graph showing the segmentation and comparison of the LT-NET network model, the VoxelMorph network model and the DataAug network model according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In the present invention, it should be noted that the terms "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. are all based on the orientation or positional relationship shown in the drawings, and are only for convenience of describing the present invention and simplifying the description, but do not indicate or imply that the apparatus or element of the present invention must have a specific orientation, and thus, should not be construed as limiting the present invention.
Examples
As shown in fig. 1 to fig. 3, the present invention discloses a one-shot brain image segmentation method based on circular consistent correlation, which includes the following steps:
s1, obtaining and classifying the brain anatomical structure image to obtain a labeled image and an unlabeled image y, and dividing the labeled image into an atlas x with a label xs。
S2, constructing an LT-NET network model, wherein the LT-NET network model comprises a generator GFGenerator GBAnd two discriminators D, generators GFAnd generator GBBoth include a twin encoder and decoder.
S3, input the image set x and the unlabelled image y into the generator GFBy means of a generator GFIs processed by the twin encoder and decoder to obtain the forward mapping deltapF。
S4, mapping the forward direction to delta pFApplied to the atlas x and the Label x, respectivelysBy means of a discriminator D and a generator GFPerforming cyclic countermeasure in coordination with supervision loss and obtaining reconstructed images through warp operation
Figure BDA0002750540490000061
And labeling
Figure BDA0002750540490000062
S5, reconstructing the image
Figure BDA0002750540490000063
And atlas x input generator GBBy means of a generator GBProcessed by the twin encoder and decoder to obtain a backward mapping Δ pB。
S6, mapping the backward direction to delta pBAre applied to the reconstructed images respectively
Figure BDA0002750540490000064
And labeling
Figure BDA0002750540490000065
By means of a discriminator D and a generator GBPerforming cyclic countermeasure in coordination with supervision loss and obtaining reconstructed images through warp operation
Figure BDA0002750540490000066
And labeling
Figure BDA0002750540490000067
The LT-NET network model of the embodiment is constructed based on a GAN network, and the network is added with a cycle structure and a antagonism thought; so that the generated reconstructed image
Figure BDA0002750540490000068
The image is reconstructed according to the unmarked image y
Figure BDA0002750540490000069
Consistent with atlas x, label x of atlas xsAnd labels
Figure BDA00027505404900000610
And (4) the same.
Monitoring loss including for constraining atlas x and reconstructing image
Figure BDA00027505404900000611
Loss of image consistency
Figure BDA00027505404900000612
The forward and backward correlations are inverse functions of each other, such that the forward mapping Δ p is appliedFThe mapping of the backward mapping can be returned to the original statetran_cyc(ΔpF，ΔpB)Segmentation labeling for constraining an atlas x true in anatomical spacexsAnd reconstructed annotations
Figure BDA00027505404900000613
Of consistent anatomical consistency
Figure BDA00027505404900000614
And anatomical difference consistency
Figure BDA00027505404900000615
The formulas are respectively expressed as:
Figure BDA00027505404900000616
Figure BDA00027505404900000617
Figure BDA00027505404900000618
Figure BDA00027505404900000619
wherein the content of the first and second substances,
Figure BDA00027505404900000620
represented as for supervised reconstruction of images
Figure BDA00027505404900000621
Loss of image consistency, L, consistent with atlas xtran_cyc(ΔpF，ΔpB)Denoted as Δ p for supervised forward mappingFAnd backward mapping Δ pBIs equal to (x), where ρ (x) is (x)2+ε2)γDenotes a generalized charbonier penalty term, t denotes voxel points, te Ω denotes all position spaces in Δ p, e ═ 0.001, γ ═ 0.45,
Figure BDA0002750540490000071
and
Figure BDA0002750540490000072
Figure BDA0002750540490000073
respectively denoted as x for supervision of the annotationsAnd label
Figure BDA0002750540490000074
And labeling
Figure BDA0002750540490000075
Anatomical consistency and anatomical variance consistency.
The loss of supervision also includes the loss of antagonism
Figure BDA0002750540490000076
Loss of similarity
Figure BDA0002750540490000077
And a smoothing loss Lsmooth(ΔpF，ΔpB)The formulas are respectively expressed as:
Figure BDA0002750540490000078
Figure BDA0002750540490000079
Lsmooth(ΔpF,ΔpB)＝Lsmooth(ΔpF)+Lsmooth(ΔpB)
Figure BDA00027505404900000710
Figure BDA00027505404900000711
wherein f isy(t) and
Figure BDA00027505404900000712
respectively representing the unmarked image y and the reconstructed image
Figure BDA00027505404900000713
Local average intensity:
Figure BDA00027505404900000714
ti denotes a volume around t of l3Coordinates within the range, l ═ 3; te Ω represents all the position spaces in Δ p, Lsmooth (ΔpF，ΔpB)Expressed using the spatial gradient between adjacent pixels in the x, y, z direction
Figure BDA00027505404900000715
The total supervision loss of the LT-NET network model is as follows:
Figure BDA00027505404900000716
wherein λ is1,、λ2、λ3Is a measure of the weight lost, λ1＝1，λ2＝3，λ3＝10。
The purpose of this embodiment is to learn a label x that can be used to label an atlas xsThe method is transferred to the relevance mapping of an unlabeled image y, and compared with the occlusion problem in the optical flow estimation task, the occlusion-non-occlusion symmetric consistency exists in the forward optical flow estimation and the backward optical flow estimation, and the embodiment provides an anatomical difference consistency loss to simply standardize the quality of a composite segmentation graph; this loss is addressed based on: the structural differences between the atlas x and the unlabeled image y in the forward and backward paths should follow a periodic consistency in the anatomical space.
Generator GFAnd generator GBThe system also comprises a double attention module which extracts the atlas x and the unmarked image y/reconstructed image by the split flow of the twin encoder
Figure BDA0002750540490000081
Inputting the related features into a double-attention module, learning the spatial information and channel information of the related features by the double-attention module, and transmitting the spatial information and channel information to a decoder, and decoding by the decoder to obtain forward mapping delta pFBackward mapping Δ pB。
The double-attention module comprises a space attention module and a channel module, the space attention module captures space information in a space dimension, the channel module captures channel information in a channel dimension, and the space information and the channel information are added to obtain a new feature map and are transmitted to the decoder.
The twin encoder is provided with 5 encoding sub-modules which are respectively a first encoding sub-module, a second encoding sub-module, a third encoding sub-module, a fourth encoding sub-module and a fifth encoding sub-module and form 1 processing stream; the twin encoder has 2 processing streams and is simultaneously connected with 1 fifth encoding submodule.
The decoder is provided with 5 decoding sub-modules which are respectively a first decoding sub-module, a second decoding sub-module, a third decoding sub-module, a fourth decoding sub-module and a fifth decoding sub-module; the first decoding submodule is connected with the double-attention module, the second decoding submodule receives the first decoding submodule and is respectively in long connection with the fourth coding submodule of 2 processing streams, the third decoding submodule receives the second decoding submodule and is respectively in long connection with the third coding submodule of 2 processing streams, the fourth decoding submodule receives the third decoding submodule and is respectively in long connection with the second coding submodule of 2 processing streams, the fifth decoding submodule receives the fourth decoding submodule, and the fifth decoding submodule outputs a forward mapping delta p from an image set x to an unmarked image yFReconstruction of images
Figure BDA0002750540490000082
Backward mapping Δ p to atlas xB
The coding sub-module is composed of ResNet-34 stacked by basic residual modules.
The discriminator D adopts a PatchGAN discriminator.
Evaluation of experiments
As shown in fig. 4-6, brain anatomy images evaluated in this experiment were collected from The Child and Adolescent neurodevelopmental program (CANDI), The Child and Adolescent NeuroDevelopment Initiative, of The university of massachusetts, disclosing a series of brain structure images as images of experimental examples and mrbrain 18 data published by MICCAI2018 race.
The evaluation index of the experimental evaluation adopts a Dice similarity coefficient to evaluate the segmentation accuracy of the model, and the accuracy is used for measuring the similarity between the manual labeling and the prediction result:
Figure BDA0002750540490000091
wherein y issRepresenting a manual annotation of the test data,
Figure BDA0002750540490000092
the experimental evaluation takes the average Dice coefficient and the standard deviation of Dice as an evaluation standard, reflects the discrete degree of the prediction result of the measured data and is defined as:
Figure BDA0002750540490000093
where n denotes the number of test data, diceiThe Dice value representing the ith test datum,
Figure BDA0002750540490000094
the average Dice of all test data is shown, and the smaller the standard deviation is, the more stable the performance of the model is.
The effectiveness of the supervision loss was verified by ablative experiments with the results shown in table 1 below:
Figure BDA0002750540490000095
TABLE 1 comparison of ablation results with supervised losses
The supervision loss of the LT-NET network model introduces the supervision of other spaces, so that the network performance can be further improved; by utilizing the forward correlation mapping, the segmentation graph of the atlas x can be synthesized into the segmentation graph of the unlabeled image y, and conversely, the synthesized segmentation graph can be restored into the segmentation graph of the atlas x by utilizing the backward correlation mapping; supervision loss of the LT-NET network model ensures the integrity and internal consistency of an anatomical structure, and effectively improves network performance.
Comparing an LT-NET network model with a MABMIS network model and a PICSL-MALF network model, wherein the MABMIS network model is composed of a tree-based group-by-group registration method and a group-by-group iterative segmentation method, and the PICSL-MALF network model provides a multi-atlas joint label fusion technology and a correction learning technology to solve the problems of the traditional voting-based multi-atlas label fusion strategy; in this experimental example, 2 to 5 atlas sets are used to verify the effects of the mabsis network model and the PICSL _ MALF network model, and the results are compared with the LT-NET network model trained in a one-shot mode, and the comparison results are shown in the following table 2:
Figure BDA0002750540490000101
TABLE 2 LT-NET network model and MABMIS, PICSL _ MALF network model comparison table
The LT-NET network model is far superior to MABMIS network model and PICSL-MALF network model which use 5 images only by using 1 image set, the MABMIS network model and the PICSL-MALF network model are abandoned due to overlong time consumption, and the MABMIS is in
Figure BDA0002750540490000102
The environment of (2) requires about 14 minutes to segment a case, PICSL _ MALF requires 3 minutes in a single Tesla P40 GPU environment, while LT-NET network model requires only 4 seconds in a single Tesla P40 GPU environment.
The segmentation results of the LT-NET network model and the MABMIS network model are visualized, the last four columns in the graph represent the segmentation results of the MABMIS network model when the atlas is 2-5, and it can be seen from the graph that the segmentation effect of the MABMIS network model is better and better along with the increase of the atlas, but compared with the LT-NET network model, the segmentation effect of the MABMIS network model is poorer, and serious multi-score and omission conditions exist.
The segmentation results of the LT-NET network model and the PICSL-MALF network model are visualized, the last four columns in the graph represent the segmentation results of the PICSL-MALF network model when the graph sets are 2-5, the edge segmentation of the PICSL-MALF network model is superior to that of the MABMIS network model, and the condition of excessive segmentation still exists; in conclusion, the LT-NET network model based on one-shot learns more accurate correlation in the brain anatomical structure segmentation task, so that an accurate segmentation result can be obtained.
The LT-NET network model was compared to the voxelmorphh network model and the DataAug network model, and the comparative segmentation results are shown in table 3 below:
Figure BDA0002750540490000111
TABLE 3 comparison of LT-NET network model with VoxelMorph and DataAug network models
Using only one labeled datum, the LT-NET network model achieved an average Dice of 82.3% performance, 6.3% and 1.9% higher than the VoxelMorph network model and the DataAug network model, respectively. In addition, compared with the DataAug network model, the LT-Net network model uses an end-to-end training mode, only one network needs to be trained, and the DataAug network model needs to train a plurality of networks, so that the process is complex; in summary, the LT-NET network model has simplicity and effectiveness in brain anatomy segmentation tasks.
The segmentation results of the LT-NET network model, the VoxelMorph network model and the DataAug network model are visualized, and the images show that the LT-NET network model can better segment detail and edge information.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (9)

1. A one-shot brain image segmentation method based on circular consistent correlation is characterized by comprising the following steps:
s1, obtaining and classifying the brain anatomical structure image to obtain a labeled image and an unlabeled image y, and dividing the labeled image into an atlas x with a label xs；
S2, constructing an LT-NET network model, wherein the LT-NET network model comprises a generator GFGenerator GBAnd two discriminators D, generators GFAnd generator GBBoth comprise twin encoders and decoders;
s3, input the image set x and the unlabelled image y into the generator GFBy means of a generator GFIs processed by the twin encoder and decoder to obtain the forward mapping deltapF；
S4, mapping the forward direction to delta pFApplied to the atlas x and the Label x, respectivelysBy means of a discriminator D and a generator GFPerforming cyclic countermeasure in coordination with supervision loss and obtaining reconstructed images through warp operation
Figure FDA0002750540480000011
And labeling
Figure FDA0002750540480000012
S5, reconstructing the image
Figure FDA0002750540480000013
And atlas x input generator GBBy means of a generator GBProcessed by the twin encoder and decoder to obtain a backward mapping Δ pB；
S6, mapping the backward direction to delta pBAre applied to the reconstructed images respectively
Figure FDA0002750540480000014
And labeling
Figure FDA0002750540480000015
By means of a discriminator D and a generator GBPerforming cyclic countermeasure in coordination with supervision loss and obtaining reconstructed images through warp operation
Figure FDA0002750540480000016
And labeling
Figure FDA0002750540480000017
2. The one-shot brain image segmentation method based on circular consistent correlation as claimed in claim 1, wherein: the supervised loss includes an image consistency loss
Figure FDA0002750540480000018
Transformation consistency Ltran_cyc(ΔpF，ΔpB)Anatomically consistent
Figure FDA0002750540480000019
And anatomical difference consistency
Figure FDA00027505404800000110
The formulas are respectively expressed as:
Figure FDA00027505404800000111
Figure FDA00027505404800000112
Figure FDA00027505404800000113
Figure FDA00027505404800000114
wherein the content of the first and second substances,
Figure FDA0002750540480000021
represented as for supervised reconstruction of images
Figure FDA0002750540480000022
Loss of image consistency, L, consistent with atlas xtran_cyc(ΔpF，ΔpB)Denoted as Δ p for supervised forward mappingFAnd backward mapping Δ pBIs equal to (x), where ρ (x) is (x)2+ε2)γRepresenting a generalized charbonier penalty, t represents a voxel point, e is 0.001, y is 0.45,
Figure FDA0002750540480000023
and
Figure FDA0002750540480000024
respectively denoted as x for supervision of the annotationsAnd label
Figure FDA0002750540480000025
And labeling
Figure FDA0002750540480000026
Anatomical consistency and anatomical variance consistency.
3. The one-shot brain image segmentation method based on circular consistent correlation as claimed in claim 2, wherein: the supervision loss also includes antagonism loss
Figure FDA0002750540480000027
Loss of similarity
Figure FDA0002750540480000028
And a smoothing loss Lsmooth(ΔpF，ΔpB)The formulas are respectively expressed as:
Figure FDA0002750540480000029
Figure FDA00027505404800000210
Lsmooth(ΔpF,ΔpB)＝Lsmooth(ΔpF)+Lsmooth(ΔpB)
Figure FDA00027505404800000211
Figure FDA00027505404800000212
wherein f isy(t) and fy(t) respectively representing the unmarked image y and the reconstructed image
Figure FDA00027505404800000213
Local average intensity:
Figure FDA00027505404800000214
ti denotes a volume around t of l3Coordinates within the range, l ═ 3; te Ω represents all the position spaces in Δ p, Lsmooth (ΔpF，ΔpB)Expressed as | | (Δ p (t)) using a spatial gradient between adjacent pixels in the x, y, z directions2。
4. A one-shot brain image segmentation method based on circular consistent correlation as claimed in claims 2-3, characterized in that: the total supervision loss of the LT-NET network model is as follows:
Figure FDA00027505404800000215
wherein λ is1,、λ2、λ3Is a measure of the weight lost, λ1＝1，λ2＝3，λ3＝10。
5. The one-shot brain image segmentation method based on circular consistent correlation as claimed in claim 1, wherein: the generator GFAnd generator GBThe system also comprises a double attention module, extracts the relevant characteristics of the image set x and the unmarked image y/reconstructed image y and the image set x by the split flow of the twin encoder, inputs the relevant characteristics into the double attention module, the double attention module learns the spatial information and the channel information of the relevant characteristics and transmits the spatial information and the channel information to a decoder, and the decoder decodes the spatial information and the channel information to obtain the forward mapping delta pFBackward mapping Δ pB。
6. The one-shot brain image segmentation method based on circular consistent correlation as claimed in claim 5, wherein: the double-attention module comprises a space attention module and a channel module, the space attention module captures space information in a space dimension, the channel module captures channel information in a channel dimension, and the space information and the channel information are added to obtain a new characteristic diagram and are transmitted to the decoder.
7. The one-shot brain image segmentation method based on circular consistent correlation as claimed in claim 5, wherein: the twin encoder is provided with 5 encoding sub-modules which are respectively a first encoding sub-module, a second encoding sub-module, a third encoding sub-module, a fourth encoding sub-module and a fifth encoding sub-module and form 1 processing stream; the twin encoder has 2 processing streams and is simultaneously connected with 1 fifth encoding submodule;
the decoder is provided with 5 decoding submodelsThe blocks are respectively a first decoding submodule, a second decoding submodule, a third decoding submodule, a fourth decoding submodule and a fifth decoding submodule; the first decoding submodule is connected with the double-attention module, the second decoding submodule receives the first decoding submodule and is respectively in long connection with the fourth coding submodule of 2 processing streams, the third decoding submodule receives the second decoding submodule and is respectively in long connection with the third coding submodule of 2 processing streams, the fourth decoding submodule receives the third decoding submodule and is respectively in long connection with the second coding submodule of 2 processing streams, the fifth decoding submodule receives the fourth decoding submodule, and the fifth decoding submodule outputs a forward mapping delta p from an image set x to an unmarked image yFBackward mapping Δ p of reconstructed image y to atlas xB。
8. The one-shot brain image segmentation method based on circular consistent correlation as claimed in claim 7, wherein: the coding sub-module is composed of ResNet-34 stacked by basic residual modules.
9. The one-shot brain image segmentation method based on circular consistent correlation as claimed in claim 1, wherein: the discriminator D adopts a PatchGAN discriminator.
CN202011182378.XA 2020-10-29 2020-10-29 One-shot brain image segmentation method based on circular consistent correlation Active CN112308833B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011182378.XA CN112308833B (en) 2020-10-29 2020-10-29 One-shot brain image segmentation method based on circular consistent correlation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011182378.XA CN112308833B (en) 2020-10-29 2020-10-29 One-shot brain image segmentation method based on circular consistent correlation

Publications (2)

Publication Number Publication Date
CN112308833A true CN112308833A (en) 2021-02-02
CN112308833B CN112308833B (en) 2022-09-13

Family

ID=74331740

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011182378.XA Active CN112308833B (en) 2020-10-29 2020-10-29 One-shot brain image segmentation method based on circular consistent correlation

Country Status (1)

Country Link
CN (1) CN112308833B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108062753A (en) * 2017-12-29 2018-05-22 重庆理工大学 The adaptive brain tumor semantic segmentation method in unsupervised domain based on depth confrontation study
CN109559332A (en) * 2018-10-31 2019-04-02 浙江工业大学 A kind of sight tracing of the two-way LSTM and Itracker of combination
CN110288609A (en) * 2019-05-30 2019-09-27 南京师范大学 A kind of multi-modal whole-heartedly dirty image partition method of attention mechanism guidance
CN111047594A (en) * 2019-11-06 2020-04-21 安徽医科大学 Tumor MRI weak supervised learning analysis modeling method and model thereof
CN111402259A (en) * 2020-03-23 2020-07-10 杭州健培科技有限公司 Brain tumor segmentation method based on multi-level structure relation learning network

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108062753A (en) * 2017-12-29 2018-05-22 重庆理工大学 The adaptive brain tumor semantic segmentation method in unsupervised domain based on depth confrontation study
CN109559332A (en) * 2018-10-31 2019-04-02 浙江工业大学 A kind of sight tracing of the two-way LSTM and Itracker of combination
CN110288609A (en) * 2019-05-30 2019-09-27 南京师范大学 A kind of multi-modal whole-heartedly dirty image partition method of attention mechanism guidance
CN111047594A (en) * 2019-11-06 2020-04-21 安徽医科大学 Tumor MRI weak supervised learning analysis modeling method and model thereof
CN111402259A (en) * 2020-03-23 2020-07-10 杭州健培科技有限公司 Brain tumor segmentation method based on multi-level structure relation learning network

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
GUHA BALAKRISHNAN ET AL.: "VoxelMorph: A Learning Framework for Deformable Medical Image Registration", 《IEEE TRANSACTIONS ON MEDICAL IMAGING》 *
SHUXIN WANG ET AL.: "Conquering Data Variations in Resolution: A Slice-Aware Multi-Branch Decoder Network", 《IEEE TRANSACTIONS ON MEDICAL IMAGING》 *
SHUXIN WANG ET AL.: "LT-Net: Label Transfer by Learning Reversible Voxel-Wise Correspondence for One-Shot Medical Image Segmentation", 《2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)》 *
杜振龙 等: "基于改进CycleGAN的图像风格迁移", 《光学精密工程》 *
贡荣麟等: "面向乳腺超声图像分割的混合监督双通道反馈U-Net", 《中国图象图形学报》 *

Also Published As

Publication number Publication date
CN112308833B (en) 2022-09-13

Similar Documents

Publication Publication Date Title
Yang et al. Unsupervised learning of geometry with edge-aware depth-normal consistency
Jiang et al. Disentangled human body embedding based on deep hierarchical neural network
CN111814238A (en) BIM real-time imaging method for breeding house based on artificial intelligence and mixed cloud reasoning
CN111062326A (en) Self-supervision human body 3D posture estimation network training method based on geometric drive
CN106649886A (en) Method for searching for images by utilizing depth monitoring hash of triple label
CN112598053A (en) Active significance target detection method based on semi-supervised learning
CN112489092A (en) Fine-grained industrial motion mode classification method, storage medium, equipment and device
CN111914618A (en) Three-dimensional human body posture estimation method based on countermeasure type relative depth constraint network
CN112308833B (en) One-shot brain image segmentation method based on circular consistent correlation
Zhou et al. Modal evaluation network via knowledge distillation for no-service rail surface defect detection
CN116977714A (en) Image classification method, apparatus, device, storage medium, and program product
CN113192186B (en) 3D human body posture estimation model establishing method based on single-frame image and application thereof
Zhang et al. Multi-modal attention guided real-time lane detection
Guo Analysis of artificial intelligence technology and its application in improving the effectiveness of physical education teaching
CN110738099A (en) low-resolution pedestrian re-identification method based on self-adaptive double-branch network
CN112348786B (en) One-shot brain image segmentation method based on bidirectional correlation
CN113255701B (en) Small sample learning method and system based on absolute-relative learning framework
CN113961737A (en) Search model training method and device, search method and device, electronic device and storage medium
Xia et al. GCENet: Global contextual exploration network for RGB-D salient object detection
Ding et al. Improving the generalization of network based relative pose regression: dimension reduction as a regularizer
CN117649422B (en) Training method of multi-modal image segmentation model and multi-modal image segmentation method
CN112308834A (en) One-shot brain image segmentation method based on forward correlation
Paudel et al. Landscape image season transfer using Generative Adversarial Networks
Li An Application Comparison of GAN-based Image Translation Methods
CN116311365A (en) Human skeleton generation method oriented to behavior recognition

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant