AU2020103905A4

AU2020103905A4 - Unsupervised cross-domain self-adaptive medical image segmentation method based on deep adversarial learning

Info

Publication number: AU2020103905A4
Application number: AU2020103905A
Authority: AU
Inventors: Shaoguo Cui; Yan Wei
Original assignee: Chongqing Normal University
Current assignee: Chongqing Normal University
Priority date: 2020-12-04
Filing date: 2020-12-04
Publication date: 2021-02-11
Anticipated expiration: 2028-12-04

Abstract

The invention provides an unsupervised cross-domain self-adaptive medical image segmentation method based on deep adversarial learning, which comprises the following steps: constructing deep encoder-decoder fully convolutional network segmentation model, constructing a domain discriminator network model, segmentation system pre-training and parameter optimization, constructing the target domain MRI automatic semantic segmentation system to form an MRI semantic segmentation image. In this application, the deep encoder decoder fully convolutional neural network is adopted to model segmentation system, and the high-level semantic features and low-level detail features are jointly utilized to predict pixel label; and the domain discriminaor network is used to guide the segmentation model to learn domain invariant features and strong generalized segmentation functions through adversarial learning, so as to minimize the data distribution difference between the source domain and the target domain indirectly, so that the learned segmentation system has the same segmentation accuracy in the target domain as in the source domain, which improves the cross domain generalization performance of the MRI automatic semantic segmentation method, and realizes the unsupervised cross domain adaptive MRI accurate segmentation. -1/4 MRI segmentation image MRI images and Training MRI brain tumor automatic segmentation segmentation labels of -system with the data set of the source domain Label predictor the source domain MR I images and Feature extractor for the source domain (fixed domain labels of the _ parameters) source domain Training domain discriminator adversarial training MRI images and Feature extractor for domain labels of the Training feature extractor for the target domain the target domain target domain images ofthe trget domain Figure 1

Description

-1/4

MRI segmentation image

MRI images and Training MRI brain tumor automatic segmentation segmentation labels of -system with the data set of the source domain Label predictor the source domain

MR I images and Feature extractor for the source domain (fixed domain labels of the _ parameters) source domain

Training domain discriminator

adversarial training

MRI images and Feature extractor for domain labels of the Training feature extractor for the target domain the target domain target domain

images ofthe

domain trget

Figure 1

Unsupervised cross-domain self-adaptive medical image segmentation method

based on deep adversarial learning

TECHNICAL FIELD

The invention relates to the technical field of medical image analysis, in

particular to an unsupervised cross-domain adaptive multimodal MRI semantic

segmentation method based on deep adversarial learning.

BACKGROUND TECHNOLOGY

With the progress of science and technology, medical imaging technology has

made great progress. MRI uses magnetic resonance to obtain electromagnetic signals

from human body and reconstruct human body information. MRI quantitative analysis

has been widely used to analyze the characteristics of many brain diseases. In order to

quantify tissue shrinkage and segmentation, corresponding brain tissue measurements

are necessary. Correspondingly, in order to quantify the changes in brain structure, it

is necessary to segment the MRI acquired at different time points. In addition, the

detection and precise location of abnormal tissues and the surrounding normal

structure are critical for diagnosis.

SUMMARY

Aiming at the problems of slow segmentation speed , limited segmentation

accuracy and significant degradation of segmentation performance in the case of

domain offset in the prior MRI technology, this invention provides an unsupervised

domain self-adaptive semantic segmentation method based on deep adversarial

learning, in which the deep encoder-decode fully convolutional neural network is

adopted to model segmentation system, and the high-level semantic features and low-level detail features are jointly utilized to predict pixel label; and the domain discriminaor network is used to guide the segmentation model to learn domain invariant features and strong generalized segmentation functions through confrontation learning, so as to minimize the data distribution difference between the source domain and the target domain indirectly, so that the learned segmentation system has the same segmentation accuracy in the target domain as in the source domain, which improves the cross domain generalization performance of the MRI brain tutor automatic semantic segmentation method and realizes the unsupervised cross domain adaptive MRI accurate segmentation, and solves the segmentation system learning problem in the target domain without labeled data samples.

In order to solve the above technical problems, the invention adopts the following technical scheme:

An unsupervised cross-domain self-adaptive medical image semantic segmentation method based on deep adversarial learning, which is characterized by comprising the following steps:

Si.Constructing a deep encoder-decoder fully convolutional network segmentation system model;

S2. Constructing a domain discriminator network model;

S3. Pre-training and parameter optimization of the medical image segmentation system;

S4. Constructing an automatic MRI semantic segmentation system for the target domain to form an MRI semantic segmentation image;

Preferably, the deep encoder-decoder fully convolutional network segmentation system model comprises a feature extractor and a label predictor, wherein the feature extractor comprises a feature encoder and a feature decoder, the feature extractor performances layer by layer extraction from low-level detail features to high-level semantic features on the multimodal MRI images, and the label predictor predicts pixel categories of medical images by utilizing fusion features, and outputs a predicted probability distribution map of the medical image pixel categories.

Preferably, the feature extractor performances layer by layer extraction from

low-level detail features to high-level semantic features on the input MRI images

through convolution and maximum pooling operations, and the feature decoder

integrates high-level semantic features and low-level detail features layer by layer

through deconvolution, feature migration and convolution operations.

Preferably, a domain discriminator network model is constructed to predict

whether the feature of the medical image is from a source domain image or a target

domain image.

Preferably, the pre-training and parameter optimization of the medical image

segmentation system comprise: performing supervised pre-training on the deep

encoder-decoder fully convolutional network segmentation model by using source

domain label data to generate a medical image pre-segmentation system.

Preferably, the pre-segmentation system of medical image specifically

comprises:

S5.1. Initializing the medical image segmentation model parameters by Xavier

method;

S5.2. Performing pre-training on the source domain segmentation system model

by two-fold cross validation;

S5.3. Performing bi-linear interpolation twice up-sampling on the four-mode

MRI images of the same section, take it as a four-channel input network, and generate

the pixel label prediction probability distribution map through the network forward

calculation;

S5.4 Taking the minimum value of the standard monitoring loss function as the

optimization objective, and solve the network parameters by the stochastic gradient

descent algorithm and the back propagation algorithm to obtain the medical image pre-segmentationsystem.

Preferably, the process of constructing the target domain MRI automatic

semantic segmentation system is as follows:

S6.1. Constructing a final MRI automatic semantic segmentation system of the

target domain with the source domain label predictor and the target domain feature

extractor;

S6.2. Performing bi-linear interpolation twice up-sampling on the four-mode

MRI images of the same section, and take it as the target domain MRI brain tumor

automatic semantic segmentation system network;

S6.3. Taking the subscript of the component of the maximum probability of each

medical image pixel as a medical image pixel category label to form a final MRI

semantic segmentation image.

The invention discloses the following technical effects:

1. The fully convolution network is used for segmentation without extracting

image blocks, and the whole image is input for end-to-end training, which makes the

training rather simply;

2. The whole image segmentation can be completed by one forward calculation,

which can improve the speed of brain segmentation;

3. The high-level semantic features are combined with the low-level detail

features to predict the pixel categories, so that the pixel label prediction accuracy can

be improved; the deconvolution operation further improves the dimension of the

feature image, so that the final segmentation image has the same resolution as the

input image does;

4. Making use of the deep domain discriminator network loss to indirectly

measure the distribution deviation of the data features and minimize the distribution

difference by an adversarial learning method, thereby avoiding complex measurement

of data distribution difference and minimization solution in the high-dimensional feature space;

5. Under the condition that there's no labeled data in the target domain, the target

domain segmentation system with the same segmentation precision as the source

domain can be generated only by using the source domain labeled data to train the

model, which solves unsupervised learning problem of the segmentation system in the

context of the target domain unmarked data, and improves the cross-domain

generalization performance of the segmentation system is improved.

BRIEF DESCRIPTION OF THE FIGURES

In order to explain the embodiments of the present invention or the technical

solutions in the prior art more clearly, the following will briefly introduce the

drawings needed in the embodiments. Obviously, the drawings in the following

description are only some of the present invention embodiments. .For the ordinary

technical personnel in this art, other drawings can be obtained on the basis of these

drawings without creative labour.

Figure 1 is the process diagram of the invention;

Figure 2 is the structural diagram of the MRI brain tumor semantic segmentation

network model provided by the invention;

Figure 3 is the structural diagram of the domain discriminator network model

provided by the invention;

Figure 4 is the process diagram of the MRI brain tumor semantic segmentation

system training method provided by the invention (the solid line represents network

parameter optimization, and the dotted line represents fixed network parameter).

DESCRIPTION OF THE IMPLEMENTATION

In order to make it easy to understand the technical means, creative features, and

achievement purpose and efficiency of the invention, the invention will be further

explained with reference to specific drawings.

Please refer to Figures 1 to 4, the invention provides an unsupervised

cross-domain self-adaptive medical image segmentation method based on deep

adversarial learning, which comprises the following steps:

Si. Constructing a deep encoder-decoder fully convolutional network

segmentation model:

S1.1. The deep encoder-decoder fully convolutional network segmentation

system comprises a feature extractor and a label predictor, wherein the feature

extractor comprises a feature encoder and a feature decoder; the feature encoder is

suitable for performing image feature extraction layer by layer on input four-mode

MRI images -FLAIR, TI, Tic and T2 through convolution and max-pooling

operations, expanding the receptive field, decreasing the resolution and completing

the extraction layer by layer from low-level detail features to high-level semantic

features. The feature decoder integrates the high-level semantic features and the

low-level detail features layer by layer through deconvolution, feature migration and

convolution operations, which increases the resolution. The label predictor is suitable

for predicting pixel categories by utilizing fusion features, and outputting a predicted

probability distribution map of the pixel categories. Specifically, the deep

encoder-decoder fully convolutional network input is 4 channels. The size of each

channel is 480*480, and the 4 channels represent respectively FLAIR, TI, Tic and T2

four-mode MRI images. The final output is 5 channels. The size of each channel is

240*240, and the 5 channels represent respectively five categories: normal tissues,

edema areas, non-enhanced tumor areas, enhanced tumor areas and necrosis areas;

Si.2. The feature encoder comprises the first to fifth feature encoder layers and

the first to fourth max-pooling layers. The first max-pooling layer is positioned

behind the first feature encoder layer, the second max-pooling layer is positioned

behind the second feature encoder layer, the third max-pooling layer is positioned

behind the third feature encoder layer, the fourth max-pooling layer is positioned

behind the fourth feature encoder layer, namely, one max-pooling layer is arranged

behind each feature encoder layer; the feature decoder comprises the first to third deconvolution layers and the first to third feature decoder layers, the first deconvolution layer is positioned behind the fifth feature encoder layer and before the first feature decoder layer, the second deconvolution layer is positioned before the second feature decoder layer and behind the first feature decoder layer, the third deconvolution layer is positioned before the third feature decoder layer and behind the second feature decoder layer. Each feature encoder layer and feature decoder layer is a layer group composed of two convolutional layers. In order to ensure that the size of the feature image is not changed in the convolution process, Padding=1 is set in the convolution process, that is, the periphery of the image is filled with 0 in the convolution process; the label predictor comprises the first to third label prediction layers and a Softmax probability conversion layer, wherein the first to third label prediction layers and the Softmax probability conversion layer are sequentially positioned behind the third feature decoder layer.

As a specific example, the detailed structure of the deep encoder-decoder fully convolutional network segmentation system model is shown in the following table 1:

Table 1 MRI brain tumor segmentation system model parameter table (Padding=1) Low level Kernal Kernel No. Layer name stride feature Input Output Function size number migration

Convolutional Feature 3x3 1 64 - 4x480x480 480x480x64 layer 11-+ReLU encoder Convolutional Feature 2 3x3 1 64 - 480x480x64 480x480x64 layer 1-2+ReLU encoder Max-pooling Feature 3 2x2 2 - - 480x480x64 240x240x64 layer encoder Convolutional 240x240x12 Feature 4 3x3 1 128 - 240x240x64 layer 21+ReLU 8 encoder Convolutional 240x240x12 Feature 3x3 1 128 - 240x240x128 layer 2-2+ReLU 8 encoder The max-pooling 120x120x12 Feature 6 2x2 2 - - 240x240x128 layer 2 8 encoder Convolutional 120x120x25 Feature 7 3x3 1 256 - 120x120x128 layer 31+ReLU 6 encoder Convolutional 120x120x25 Feature 8 3x3 1 256 - 120x120x256 layer 3-2+ReLU 6 encoder

120x120x256 60x60x256 Feature 9 The max-pooling 2x2 2 layer3 encoder Convolutional Feature 3x3 1 512 60x60x256 60x60x512 layer 4_1+ReLU encoder Convolutional Feature 11 3x3 1 512 60x60x512 60x60x512 layer 4_2+ReLU encoder Themax-pooling Feature 12 2x2 2 - 60x60x512 30x30x512 layer4 encoder Convolutional Feature 13 3x3 1 1024 30x30x512 30x30x1024 layer 5_1+ReLU encoder Convolutional Feature 14 3x3 1 1024 30x30x1024 30x30x1024 layer 5_2+ReLU encoder Feature Deconvolution 2x2 2 512 30x30x1024 60x60x512 decoder Convolutional Convolutional Feature 16 3x3 1 512 layer 60x60x1024 60x60x512 layer 6 1+ReLU decoder 4_2+ReLU Convolutional Feature 17 3x3 1 512 60x60x512 60x60x512 layer 6_2+ReLU decoder 120x120x25 Feature 18 Deconvolution 2x2 2 256 60x60x512 6 decoder Convolutional Convolutional 120x120x25 Feature 19 3x3 1 256 layer 120x120x512 layer 7 1+ReLU 6 decoder 3_2+ReLU Convolutional 120x120x25 Feature 3x3 1 256 120x120x256 layer 7_2+ReLU 6 decoder 240x240x12 Feature 21 Deconvolution 2x2 2 128 120x120x256 8 decoder Convolutional Convolutional 240x240x12 Feature 22 3x3 1 128 layer 240x240x256 layer 8 1+ReLU 8 decoder 2_2+ReLU Convolutional 240x240x12 Feature 23 3x3 1 128 240x240x128 layer 8_2+ReLU 8 decoder Convolutional 240x240x51 Label 24 1xi 1 512 240x240x128 layer 9_1+ReLU 2 estimation Convolutional 240x240x10 Label 1xi 1 128 240x240x512 layer 9_2+ReLU 24 estimation Convolution 240x240x102 Label 26 layer 1xi 1 5 - 240x240x5 4 estimation 9_3+softmax

As can be seen from Table 1, in the step S12, the numbers of convolution kernels

of the first to fifth feature encoder layers are 64, 128, 256, 512 and 1024 in sequence, the numbers of convolution kernels of the first to third feature decoder layers are 512,

256 and 128 in sequence, the size of the convolution kernels of the feature encoder

layer and the feature decoder layer is 3*3, the step size is 1, the numbers of

convolution kernels of the first to third label prediction layers are 512, 128 and 5 in

sequence, the size of the convolution kernels of the first to third label prediction

layers is 1*1, the step size is 1, the size of pooling kernels of each max-pooling layer

is 2*2, the step size is 2, the size of the convolution kernels of each deconvolution

layer is 2*2, the step size of each deconvolution layer is 2. Wherein, the max-pooling

layer is used for carrying out twice down-sampling to remove redundant features and

enlarge the receptive field; the feature decoder specifically carries out twice

up-sampling through an deconvolution layer to reduce the number of output channels

to half of the original number, and carries out cascade connection of an deconvolution

result in the first feature decoder layer and a low-level detail feature image with the

same resolution transferred by the second convolution layer of the fourth feature

encoder layer 4_2+ReLU; cascade connection of an deconvolution result in the

second feature decoder layer and a low-level detail feature image with the same

resolution transferred by the second convolution layer of the third feature encoder

layer 3_2+ReLU; and cascade connection of an deconvolution result in the third

feature decoder layer and a low-level detail feature image with the same resolution

transferred by the second convolution layer of the second feature encoder layer

2_2+ReLU, completing the layer-by-layer fusion of high-level semantic features and

low-level detail features.

S2. Constructing a domain discriminator network model:

S2.1. The domain discriminator network is suitable for inputting the output

features of the second convolution layer of the third feature decoder layer 8_2+ReLU

to predict whether the input features come from a source domain image or a target

domain image;

S2.2. The domain discriminator network comprises a convolution layer, a first

fully connected layer 1, a second fully connected layer 2 and a third fully connected layer 3, which are sequentially arranged, and the detailed structure is shown in the following table 2:

Table 2 Domain discriminator network model parameter table Kernel Input No Layer name stride Kernel number Output size

Convolutional layer 1 lx1 1 1 8 2+ReLU 240x240x1 (240x240x128)

Fully Connected 57600 (=240x240) 2 2048 layer 1+ReLU Fully Connected 2048 3 1024 layer 2+ReLU Fully Connected 1024 4 2 layer 3+Softmax

It can be concluded from table 2, that in the step S22, the size of convolution

kernels of the convolution layer is 1*1, the step size is 1, and the number of

convolution kernels is 1; the two-dimensional neurons output by the convolution layer

are expanded in rows to form one-dimensional array neurons, which are used as the

input of the first fully connected layer 1, and the corresponding node numbers of the

output ends of the first fully connected layer 1, the second fully connected layer 2 and

the third fully connected layer 3 are respectively 2048, 1024 and 2.

As a specific embodiment, in order to facilitate the extraction of features for the

source domain image and the target domain image, the feature decoder together with

the feature extractor are referred to as the feature extractor in this application The

source domain and the target domain respectively construct the feature extractors with

the same structure (but the model parameters are different, which can be learned by

training data later) to extract features from the source domain image and the target

domain image respectively; and then the features extracted from the two domain

images (the output result of the convolution layer 8_2+ReLU) are input to the domain

discriminator to predict whether the two domain images come from the source domain

or the target domain. As a preferred embodiment, the category label of the source

domain is 0, and the category label of the target domain is 1. Of course, the technicians in the art can also set the category labels of the source domain and the target domain in other ways as long as the domain discriminator can effectively predict.

S3. Pre-training and parameter optimization of the segmentation model: performing supervised pre-training on the deep encoder-decoder fully convolutional network segmentation model established in the step Si by using the label data of the source domain to generate a pre-segmentation system, which specifically comprises the following steps:

S3.1. Initializing network parameters, and initialize the parameters of the convolution kernels and the deconvolution kernels by Xavier method;

S3.2. Dividing the source domain training data and the labels into training set and validation set in a 1:1 ratio, and performing pre-training on the source domain segmentation system model by adopting two-fold cross validation; for example, the inventor obtained a total of 274 pieces of four-mode MRI image data with segmentation labels, wherein the image size is 240*240, each mode comprises 155 slices, and the total number of the data samples is 274*155=42470. Perform data expansion processing on these images and corresponding labels: using the data expansion technology of horizontal inversion, vertical inversion, reduction and rotation of 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees and 315 degrees to increase training data samples to be 10 times of the original data samples, so that the sample numbers of the expanded training set and the expanded validation set are respectively 212350;

S3.3. Performing bi-linear interpolation twice up-sampling on FLAIR, TI, Tic and T2 four-mode MRI images of the same brain section, and the size of the samples changes into 480*480, take them as the four-channel input network to generate a pixel label prediction probability distribution image through network forward calculation. As a specific embodiment, the forward calculation comprises:

Convolution operation: the output feature image Zi corresponding to any convolution kernel in the network is calculated by the following formula: k Z,=f(bW+ W,@X,)

Wherein f represents a nonlinear excitation function, bi represents an offset

term corresponding to the No.i convolution kernel, r represents an input channel

index number, k represents an input channel number, ff, represents No. r

channel weight matrix of No.i convolution kernel, 0 represents a convolution

operation, X, represents an image of No. r input channel;

Batch normalization and nonlinear excitation: perform normalization with the

mean value of 0 and the variance of1 on each convolution-obtained feature image Z,

and perform nonlinear conversion on each normalized value by using a rectifying

linear unit ReLU as a nonlinear excitation function, wherein the rectifying linear unit

ReLU is defined as follows:

f(x)= max(0, x)

Wherein f(x) represents a rectifying linear unit function, max represents a

maximum value, X is an input value;

Probability value conversion: convert the label prediction score output by the

network into a probability distribution by Softmax function, which is defined as

follows:

en Yf =softmax(Oy)= e

1=1

Wherein Oi represents a predicted score of a pixel at the final output of the

segmentation network on Category, Y represents the probability that the input pixel belongs to Category ], C represents the number of categories, which is 5.

S3.4. Adopting a standard supervision loss function as an optimization target,

and the loss function is defined as follows:

K L,,(0,O)=-E ,y,)-(x Eiy' I logC(M'(x';O );')

Wherein L,(,,O) is a source domain segmentation loss function, q, is a

source domain feature extractor network parameter, Oc is a label predictor network

parameter, X, is a source domain image set, Y is a source domain segmentation

label set, x, is a sample, y, is a label corresponding to the sample,

(x,,y,) (X ,Y)represents that the sample and the corresponding segmentation label

(x,,yj)obey source domain data distribution (X,,Y), M,(.) is a source domain

feature mapping function, namely a feature extractor for the source domain, C(.) is

a pixel prediction function, K=5 represents the number of the pixel categories,

Ijy represents that when js , then I =1, otherwise I=0 , log represents a

logarithmic value, and E represents a mathematical expectation;

S35. Taking the minimum value of L,(O,,O,) as the optimization goal. Solve

the network parameter 0, and 0c by random gradient descent algorithm and

reverse propagation algorithm to obtain an MRI brain tumor pre-segmentation system.

As an embodiment, the number of samples used in each iteration during random

gradient descent iteration, namely the batch size (BatchSize) is 32, the initial learning

rate is set as 1e3 , the learning rate is gradually reduced tole by linear attenuation

technology; the momentum factor is set as 0.9 without using a dropout technology.

S4. Adversarial training and parameter optimization of the target domain feature

extractor: establish a model of feature extractor for the target domain according to the establishing method of feature extractor in step Si, and generate feature extractor for the target domain through adversarial training, which specifically comprises the following steps:

S41. Initializing the feature extractor for the target domain by the feature extractor parameters of the pre-segmentation system generated in step S3, and initializing the domain discriminator network parameters by Xavier method;

S42. Obtaining 424700 target domain training samples with the same number of the source domain training samples, setting domain category labels of source domain image and target domain image, performing twice up-sampling on source domain images and target domain images by bi-linear interpolation, the size of samples are 480*480, inputing extracted features of the source domain and the target domain feature extractors sequentially, and then inputing the extraction features into the domain discriminator built in S2 to predict whether the extracted features come from the source domain or the target domain, output the domain category labels;

S43. Training the domain discriminator and the feature extractor for the target domain alternately in a adversarial mode. On one hand, optimizing the domain discriminator so that the domain category labels can be accurately predicted. On the other hand, optimizing the feature extractor for the target domain so that the extracted features can not be distinguished from the extracted features of the source domain, that is, minimizing the distribution difference between the source domain data and the

target domain data to obtain the optimal network model parameter 0, of the feature

extractor for the target domain; in S43, the parameters of the feature extractor for the source domain are fixed, which can't be trained.

As a specific embodiment, the optimization domain discriminator in S43 specifically includes: optimizing the domain discriminator network with the following objective function:

P P Lady, (0d)= -E(-X)id, I1ogD(M,'(x ,;q,);Od) E(,-Xx)ZI'[i-d, ]log(1- D(7Vf (x,;0,); d))

Wherein L,1 D (0) represents the loss of domain discriminator, x, ~ X,

represents that x, obeys the source domain distribution X, , X, represents that

x, obeys the target domain distribution X,, X is a training sample of the source

domain discriminator, X, is a training sample of the target domain discriminator, O,

is a network parameter of the feature extractor for the source domain, 0, is a

network parameter of the feature extractor for the target domain, 0 d is a network

parameter of the domain discriminator, Ms(.) is the source domain feature mapping

function, namely the feature extractor for the source domain, M,(.) is the target

domain feature mapping function, namely the feature extractor for the target domain,

D(.) is the domain category prediction function, P is the domain category number,

ds and d, are domain labels, ]I=( represents that when id=a , then I=1

otherwiseI=0; [i=d,] represents that when id,, then I =1, otherwise I=0. log

represents a logarithmic value, and E represents a mathematical expectation;

Taking the minimum value ofLV,(0,) as the optimization objective, that is to

seek the value of 0,, which maximize the loss of the target domain category made by

the domain discriminator. At the moment, the domain discriminator cannot distinguish whether input data comes from the target domain or not; and update the

network parameter 0, by random gradient descent algorithm and reverse propagation

algorithm according to the fixed parameter Od . As an embodiment, the number of

samples used per iteration in the random gradient descent iteration, namely the batch size (BatchSize) is 128, wherein, the number of samples of the source domain image and the target domain image are both 64, and the initial learning rate is set as from le-2 to le-5 .

S5. Automatic semantic segmentation of the target domain MRI brain tumor:

S51. Constituting the final MRI brain tumor automatic semantic segmentation

system of the target domain with the label predictor learned in the source domain in

S3 and the feature extractor for the target domain learned in confrontation in S4;

S52. Carrying out twice up-sampling on FLAIR, TI, Tic and T2 four-mode MRI

images of the same brain section by bi-linear interpolation, wherein the size of the

four-mode MRI images are changed into 480*480, taking it as an MRI brain tumor

automatic semantic segmentation system network of the target domain formed by the

four-channel input S51;

S53. Adopting the same network forward calculation as S33 to generate a pixel

category prediction probability distribution image: extracting image features of the

target domain by using a target domain feature mapping function M,(x,;0,)

, predicting pixel categories according to the extracted image features of the target

domain by using a label prediction function C(M,;0,), generating five pixel label

prediction probability distribution images with the size of 240*240, and respectively

represent the probability of each pixel on five categories;

S54. Taking the subscript of the component of the maximum probability of each

pixel as a pixel category label to form a final MRI brain tumor semantic segmentation

image.

Compared with the existing techniques, the unsupervised cross-domain

self-adaptive medical image segmentation method based on deep adversarial learning

provided by the invention has the following advantages:

1. The segmentation of the whole image can be completed by one forward

calculation, which can improve the speed of brain tumor segmentation;

2. The high-level semantic features are combined with the low-level detail

be improved; the deconvolution operation further improves the dimension of the

feature image, so that the final segmentation image has the same resolution as the input image;

3. Making use of the network loss of the deep domain discriminator to indirectly

difference by an adversarial learning method, thereby avoiding complex data

distribution difference measurement and minimization solution in a high-dimensional

feature space;

4. Under the condition that there's no labeled data in the target domain, the target

domain segmentation system with the same segmentation precision as the source

domain can be generated only by using the labeled data to train the model, which

solves unsupervised learning problem of the segmentation system in the context of the

target domain unmarked data, and improves the cross-domain generalization

performance of the segmentation system is improved.

Finally, it is to be noted that the above-mentioned embodiments illustrate rather

than limit the invention. Although the invention has been described in detail with

reference to preferred embodiments, it is supposed be understood by the common

technicians in this art that modifications and equivalents may be made thereto, but

without departing from the spirit and scope of the invention, all of these modifications

and equivalents shall be included within the scope of the claims of the invention.

Claims

1. An unsupervised cross-domain adaptive medical image segmentation method based on deep adversarial learning, which is characterized by comprising:

Si. Constructing a deep encoder-decoder fully convolutional network

segmentation model;

S2. Constructing a domain discriminator network model;

S3. Carrying out pre-training and parameter optimization of the segmentation model;

S4. Adversarial training and parameter optimization of the target domain feature extractor;

2.According to claim 1, the unsupervised cross-domain adaptive medical image

segmentation method based on deep adversarial learning is characterized in that the deep encoder-decoder fully convolutional network segmentation system model

comprises a feature extractor and a label predictor; the feature extractor comprises a

feature encoder and a feature decoder; the feature extractor performances layer by layer extraction from low-level detail features to high-level semantic features on the

multimodal MRI images, and the label predictor predicts pixel categories of medical

images by utilizing fusion features, and outputs a predicted probability distribution map of the medical image pixel categories.

3. According to claim 2, the unsupervised cross-domain adaptive medical image

segmentation method based on deep adversarial learning is characterized in that the feature encoder performances layer by layer extraction from low-level detail features

to high-level semantic features on the input MRI images through convolution and

maximum pooling operations, and the feature decoder integrates high-level semantic

features and low-level detail features layer by layer through deconvolution, feature migration and convolution operations.

4. According to claim 1, the unsupervised cross-domain adaptive medical image segmentation method based on deep adversarial learning is characterized in that the domain discriminator network model is constructed for predicting whether a feature of a medical image is from a source domain image or a target domain image.

5. According to claim 1, the unsupervised cross-domain adaptive medical image

segmentation method based on deep adversarial learning is characterized in that the

pre-training and parameter optimization of the medical image segmentation system comprise: performing supervised pre-training on the deep encoder-decoder fully

convolutional network segmentation model by using label data of the source domain

to generate a medical image pre-segmentation system.

6. According to claim 5, the unsupervised cross-domain adaptive medical image

pre-segmentation system of medical image specifically comprises:

S5.1. Initializing parameters of the segmentation model of medical image by

Xavier method;

S5.2. Performing pre-training on the segmentation system model of the source

domain by two-fold cross validation;

S5.3. Performing twice up-sampling on the section of the same part of the

four-mode MRI images by bi-linear interpolation, take it as a four-channel input network to generate a pixel label prediction probability distribution image through

forward network calculation;

S5.4. Taking the minimum value of the standard monitoring loss function as the optimization objective, and solve the network parameters by the random gradient

descent algorithm and the reverse propagation algorithm to obtain the

pre-segmentation system of medical image.

7. According to claim 1, the unsupervised cross-domain adaptive medical image

segmentation method based on deep adversarial learning is characterized in that the process of constructing the MRI automatic semantic segmentation system of the target domain is as follows:

S6.1. Constituting the final MRI automatic semantic segmentation system of the target domain with the label predictor for the source domain and the feature extractor

for the target domain;

S6.2. Carrying out twice up-sampling on the section of the same part of MRI images by bi-linear interpolation, and take it as an MRI brain tumor automatic

semantic segmentation system network of the target domain;

S6.3. Taking the subscript of the maximum probability component of each pixel as a pixel category label to form a final MRI semantic segmentation image.

-1/4- MRI segmentation image

MRI images and Training MRI brain tumor automatic segmentation segmentation labels of system with the data set of the source domain Label predictor the source domain 2020103905

MRI images and Feature extractor for the source domain (fixed domain labels of the parameters) source domain

Training domain discriminator

adversarial training

MRI images of the target domain

Figure 1

Input MRI images -2/4-

Figure 2 Output brain tumor segmentation image Convolution

Max-pooling Deconvolution Feature Migration

-3/4- Deformation

Fully connected layer 1

Convolution layer 8_2 Convolution layer 2020103905

Fully connected layer 3

Fully connected layer 2

Figure 3

-4/4- 04 Dec 2020

MRI four-mode image and 源域MRI四模态图像及分割 segmentation labels of the source domain 标签

Feature extractor Brain tumor

Label predictor Pre-training of the for the source segmentation 源域特征提取 domain image 源域分割系统预训练 segmentation system of the 脑肿瘤分割图 2020103905

source domain 器

MRI four-mode Image of the source domain 源域MRI四模态图像

Feature extractor for the 源域特征提取 source domain 器 Adversarial training between the

Domain discriminator domain discriminator and the Domain 域鉴别器与目标域特 feature extractor of the target category label domain 征提取器对抗训练 MRI four-mode image of the 域分类标签 target domain 目标域MRI四模态图像 Feature extractor for the target 目标域特征提 domain 取器

MRI four-mode image of the target domain 目标域MRI四模态图像 MRI brain tumor segmentation system of the target domain Feature extractor Brain tumor 目标域MRI脑肿瘤分 Label Predictor

for 目标域特征提the target segmentation 割系统 domain 脑肿瘤分割图 image 取器

Figure 4