CN114677511A - Lung nodule segmentation method combining residual ECA channel attention UNet with TRW-S - Google Patents

Abstract

A lung nodule segmentation method combining residual ECA channel attention UNet with TRW-S comprises the following steps: step 1: reading lung CT images, and preprocessing the images; step 2: cutting out a lung nodule picture and generating a corresponding mask; and step 3: dividing the acquired lung nodule picture and the corresponding mask into a data set to obtain a training set, a verification set and a test set; and 4, step 4: constructing a residual error ECA channel attention UNet deep learning network; and 5: inputting the obtained training set and verification set into the constructed deep learning network for training; step 6: inputting the obtained test set into a trained network to obtain a prediction graph; and 7: inputting the prediction graph into a sequence tree weighted information transfer algorithm (TRW-S) based on a Markov random field to carry out edge smoothing; and 8: and outputting the predicted image.

Description

Lung nodule segmentation method combining residual ECA channel attention UNet with TRW-S

Technical Field

The invention belongs to the technical field of image segmentation, and particularly relates to a lung nodule segmentation method combining a residual attention UNet network and a sequence tree weighted information propagation (TRW-S) algorithm.

Background

The lung cancer is the cancer with the highest incidence rate in the world nowadays, accounts for 11.6% of all cancers, and the lung cancer is the first to occur in 2019 men living in China, and smoking is the most important reason for the lung cancer. The malignant nodules in the lung are a precursor generated by lung tumors, and are characterized in that the high-density white shadows or ground vitreous false shadows of the lung with the diameter smaller than 3cm are shown on CT images, and the lesion segmentation of the pulmonary nodules by using a segmentation algorithm can assist doctors in judging the disease condition. Further classification of lesion malignancy and malignancy may also be made by dividing the lesion into segments. The early discovery, identification, tracking and removal of malignant lung nodule lesions is an important means for preventing lung cancer. The lung nodules are screened manually, a large amount of time and energy of imaging doctors are usually consumed for distinguishing, diagnosis can be performed by the doctors with the aid of a machine algorithm, and the working intensity of the doctors is reduced.

Image segmentation by using an end-to-end deep learning network becomes the mainstream of image semantic segmentation, pulmonary nodules can be classified in a pixel level manner through training based on Unet and an improved full convolution neural network thereof, and finally, a binary image is output to mark the pulmonary nodule region and the edge thereof. The full convolution network combines high-dimensional position information and low-dimensional edge information to complete the task of semantic segmentation through a decoding-coding mode. But the disadvantages are that: due to the influence of the edge volume effect, the problem of edge blurring usually exists when the Lung nodule CT image is segmented by using the Unet and the improved network thereof, and the lung nodule is difficult to segment completely when some lesions are small or the difference between lesion tissues and surrounding tissues is small. By introducing the processed public data set and comparing the public data set with the conventional general medical image segmentation method, the method can find that a better lung nodule segmentation effect cannot be obtained by adopting UNet and an improved method thereof; the fundamental reasons are that: in the network training process, the fluctuation is large, the connection between channels cannot be well obtained, the final result edge delimitation is fuzzy, and even some misjudgments exist in a lesion area; the object of the present invention is to solve the technical problem.

Disclosure of Invention

The invention aims to solve the technical problems of fuzzy edge delimitation and inconsistent segmentation inside a lesion region in the segmentation of a pulmonary nodule by using a UNet and the existing UNet improved network thereof.

A lung nodule segmentation method combining residual ECA channel attention UNet with TRW-S comprises the following steps:

step 1: reading lung CT images, and preprocessing the images;

step 2: cutting out a lung nodule picture and generating a corresponding mask;

and 3, step 3: dividing the acquired lung nodule picture and the corresponding mask into a data set to obtain a training set, a verification set and a test set;

and 4, step 4: constructing a residual error ECA channel attention UNet deep learning network;

and 5: inputting the obtained training set and verification set into the constructed deep learning network for training;

step 6: inputting the obtained training set into a trained neural network to obtain a prediction graph;

and 7: inputting the prediction graph into a sequence tree weighted information transfer algorithm (TRW-S) based on a Markov random field to carry out edge smoothing;

and 8: and outputting the predicted image.

In step 1, reading a lung CT image in a specified format in a target data set; and adjusting the window position and the window width of the CT image.

In step 2, a part of the CT image having a lung nodule is cut out, and a mask image corresponding to the lung nodule is generated.

In step 4, the constructed residual ECA channel attention UNet deep learning network specifically includes:

input layer → convolution module → first residual ECA channel attention module (composed of two convolutions of 3 × 3, combination of BN layer and one ECA channel attention module combined with residual connection) → first max pooling layer → second residual ECA channel attention module → second max pooling layer → third residual ECA channel attention module → third max pooling layer → fourth residual ECA channel attention module → fourth max pooling layer → void convolution pyramid module → first double upsampling, channel fusion of the feature map formed by first double upsampling and the feature map formed by fourth residual ECA channel attention module → first ECA channel attention module (composed of two convolutions of 3 × 3, BN layer and one ECA channel attention module) → second double upsampling, channel fusion of the feature map formed by second double upsampling and the feature map formed by third residual ECA channel attention module → first ECA channel attention module → combination of the feature map formed by second double upsampling and the third residual ECA channel attention module → combination of the feature map Two ECA channel attention modules → three times of upsampling, the feature map formed by the three times of upsampling and the feature map formed by the second residual ECA channel attention module are subjected to channel fusion → the third ECA channel attention module → the fourth times of upsampling, the feature map formed by the fourth times of upsampling and the feature map formed by the first residual ECA channel attention module are subjected to channel fusion → the fourth ECA channel attention module → a convolution module → a sigmoid function → a prediction map.

In step 5, during training of the neural network, a loss function combining binary cross entropy and confidence is used, Nadam is used as an optimization function, and cross-over ratio is used as an evaluation index of the model.

In step 7, a Markov random field is constructed for the prediction graph, and a binary item is smoothed, so that the edge continuity of the segmentation effect and the consistency of the interior of the region are improved, and the method specifically comprises the following steps:

7-1) constructing a Markov random field in a gray level prediction image output by a deep learning network, taking each pixel of the image as a node of the Markov random field, and using the gray value of the pixel as the weight of a univariate item of the node;

7-2) constructing a sequence tree weighted information transfer algorithm (TRW-S) based on a Markov random field to carry out binary item smoothing on the segmentation edge, wherein the weight of the unitary item is the pixel value size of the segmentation result generated by the segmentation network.

Compared with the prior art, the invention has the following technical effects:

1) in the prior art, after a lung nodule is segmented by using a full convolution neural network, a hard threshold method is usually adopted to directly carry out binarization on a segmentation result, so that the influence among pixels is not considered; according to the invention, a sequence tree weighted information transfer algorithm based on a Markov random field is constructed in an image output by a deep learning network, the boundary of a pulmonary nodule is better determined through information transfer among pixels, and meanwhile, gray areas (namely areas with inaccurate segmentation) in some lesion areas can be adjusted by depending on the influence of surrounding lesion pixels on the gray areas, so that the purposes of high consistency in the segmentation areas and reasonable and clear segmentation boundaries are finally achieved;

2) the invention improves the backbone network architecture of UNet, uses a mode of combining a residual error network with ECA channel attention in the backbone network, and uses ASPP to improve the network;

3) because the residual error structure is added in the decoder of the network, the influence on the segmentation result is small, therefore, the invention adds the residual error structure in the encoding part of the UNet main network, and directly transmits the input information to the output by using the residual error structure, thereby reducing the information loss; the problem of gradient explosion is reduced for the network, so that the constructed network model can be converged more quickly and better;

4) an ECA channel attention mechanism is added into the network constructed by the method, so that the model can better pay attention to the information of the important channel, and the training effect of the model is improved; the ASPP cavity convolution pyramid is used in the model, so that the model can better acquire multi-scale information, the detection effect on small lesions is improved, and finally batch standardization is inserted in the model to prevent overfitting of the model.

Drawings

The invention is further illustrated by the following examples in conjunction with the accompanying drawings:

FIG. 1 is a flow chart of the present invention;

FIG. 2 is a block diagram of a neural network of the present invention;

FIG. 3 is a graph showing the results of the experiment according to the present invention.

Detailed Description

As shown in fig. 1 to 2, a lung nodule segmentation method includes the following steps:

1. reading lung CT image in dcm format in LIDC-IDRI data set

2. Adjusting the window level of the CT image to 1600 and the window width to 450

3. Reading an XML (extensive Makeup language) labeling file, cutting out a part with lung nodules in the CT image through labeling information, reading the XML labeling file with the size of 64 x 64, and generating a mask image corresponding to the lung nodules through edge segmentation information;

4. dividing the picture intercepted in the step 3 into data sets

5. Construction of residual ECA channel attention UNet deep learning network

6. Inputting the training set and the verification set divided in the step five into the network constructed in the step 6 for training

7. Inputting the test set into the trained network in the step 7 to obtain a prediction graph

8. Inputting the output graph in the step 7 into a sequence tree weighted information transfer algorithm (TRW-S) based on a Markov random field for edge smoothing

9. Outputting a predicted image;

in step 1, LIDC-IDRI data sets were collected from the national cancer institute, for a total of 1018 study cases, each case varied from about 200 and 400 CT images, and were labeled by four physicians with four professionals, each CT image was 512 x 512, and 3mm-33mm lung nodule slices containing complete outlines of nodes were selected for segmentation.

In step 3, the part containing the nodes is cut into pictures with the size of 64 x 64 by taking the nodes as the center according to the position information marked by the XML file in the data set, corresponding boundary pictures are generated according to the coordinate values of the node outline information, and the boundaries are filled up through an algorithm to obtain corresponding mask pictures. Finally, 10454 lung nodule pictures and corresponding masks are captured.

In step 4, a ratio of 8:1:1 is used for data set division, wherein the training set comprises 8363 pairs of pictures, the verification set comprises 1045 pairs of pictures, and the test set comprises 1046 pairs of pictures.

A residual ECA channel attention UNet full convolution neural network is constructed in step 5:

the lung nodule image with the input size of 64 × 64 is convolved by 1 × 1 to obtain a, the A enters a residual ECA channel attention module, specifically, the A is convolved by two 3 × 3, the BN layer and the ECA channel attention module obtain B, the A is convolved by 1 × 1 and is added with the B element by element to obtain B1, the B2 is obtained through the maximum pooling layer, the size of the feature map becomes 32 × 32, and the number of channels becomes 128. B2 passes through a residual ECA channel attention module to obtain C1, C1 passes through a maximum pooling layer to obtain C2, the size of the feature map is changed to 16 × 16, and the number of channels is changed to 256. C2 gets D1 through a residual ECA channel attention module, D2 is got through a layer of maximum pooling layer by D1, the size of the feature map becomes 8 x 8, and the number of channels becomes 512. D2 obtains E1 through a residual ECA channel attention module, E2 is obtained through E1 passing through a maximum pooling layer, the size of a feature map is changed to 4 x 4, the number of channels is changed to 1024, and E2 obtains multi-scale information through a hollow pyramid convolution module to obtain F. F is subjected to two-time upsampling to obtain E-1, the E-1 and E1 are subjected to channel fusion and then subjected to two-layer 3 x 3 convolution, and a BN layer and an ECA channel pay attention to obtain E-2. E-2 is sampled twice to obtain D-1, the D-1 and D1 are subjected to channel fusion and then subjected to two-layer 3 x 3 convolution, and the BN layer and ECA channel attention are obtained to obtain D-2. D-2 is up sampled twice to obtain C-1, the C-1 and C1 are subjected to channel fusion and then subjected to two-layer 3 x 3 convolution, and the BN layer and ECA channel attention are obtained to obtain C-2. B-1 is obtained by performing double-time upsampling on C-2, and B-2 is obtained by performing 3 x 3 convolution on B-1 and B1 after channel fusion, and attention is paid to a BN layer and an ECA channel. B-2 obtains A-1 as a prediction result after 1 x 1 convolution and sigmoid function.

In step 6, a loss function combining binary cross entropy and confidence coefficient is used, Nadam is used as an optimization function, a cross-over ratio is used as an evaluation index of the model, the batch _ size is set to be 32, the learning rate is 1e-4, and the effect is optimal after 200 rounds of training. And adjusting the model hyper-parameters through the verification set precision.

In step 8, a Markov random field is constructed for the output graph, and a binary term is smoothed, so that the edge continuity of the segmentation effect and the consistency of the interior of the region are improved, and the method specifically comprises the following steps:

8-1) constructing a Markov random field in a gray level prediction graph output by the network constructed by the invention, taking each pixel of the image as a node of the Markov random field, and using the gray value of the pixel as the weight of a univariate item of the node;

8-2) constructing a sequence tree weighted information transfer algorithm (TRW-S) based on a Markov random field to carry out binary item smoothing on the segmentation edge, wherein the weight of the unitary item is the pixel value of the segmentation result generated by the segmentation network.

Claims (6)

1. A residual ECA channel attention UNet combined TRW-S lung nodule segmentation method, characterized in that it comprises the following steps:

step 1: reading lung CT images, and preprocessing the images;

step 2: cutting out a lung nodule picture and generating a corresponding mask;

and 3, step 3: dividing the acquired lung nodule picture and the corresponding mask into a data set to obtain a training set, a verification set and a test set;

and 4, step 4: constructing a residual error ECA channel attention UNet deep learning network;

and 5: inputting the obtained training set and verification set into the constructed deep learning network for training;

step 6: inputting the obtained test set into a trained network to obtain a prediction graph;

and 7: inputting the prediction graph into a sequence tree weighted information transfer algorithm (TRW-S) based on a Markov random field to carry out edge smoothing;

and 8: and outputting the predicted image.

2. The method according to claim 1, wherein in step 1, a lung CT image in a format specified in the target data set is read; and adjusting the window position and the window width of the CT image.

3. The method according to claim 1, wherein in step 2, a portion of the CT image having a lung nodule is cut out, and a mask image corresponding to the lung nodule is generated.

4. The method according to claim 1, wherein in step 4, the constructed residual ECA channel attention UNet deep learning network is specifically:

input layer → convolution module → first residual ECA channel attention module → first max pooling layer → second residual ECA channel attention module → second max pooling layer → third residual ECA channel attention module → third max pooling layer → fourth residual ECA channel attention module → fourth max pooling layer → cavity convolution pyramid module → first double upsampling;

performing channel fusion on a feature map formed by the first two-time upsampling and a feature map formed by a fourth residual ECA channel attention module → a first ECA channel attention module → a second two-time upsampling;

performing channel fusion on the feature map formed by the second two-time upsampling and the feature map formed by the third residual ECA channel attention module → performing second ECA channel attention module → performing third two-time upsampling;

performing channel fusion on the feature map formed by the third two-time upsampling and the feature map formed by the second residual ECA channel attention module → the third ECA channel attention module → the fourth two-time upsampling;

and performing channel fusion on the feature map formed by the fourth two-time upsampling and the feature map formed by the first residual ECA channel attention module → a fourth ECA channel attention module → a convolution module → a sigmoid function → a prediction map.

5. The method according to claim 1, wherein in step 5, in training of the neural network, a loss function combining binary cross entropy and confidence is used, Nadam is used as an optimization function, and cross-over is used as an evaluation index of the model.

6. The method according to claim 1, wherein a markov random field is constructed for the prediction graph in step 7, and the binary term is smoothed to improve the edge continuity of the segmentation effect and the intra-region consistency, and specifically comprises the following steps:

7-1) constructing a Markov random field in a gray level prediction image output by a deep learning network, taking each pixel of the image as a node of the Markov random field, and using the gray value of the pixel as the weight of a univariate item of the node;

7-2) constructing a sequence tree weighted information transfer algorithm (TRW-S) based on the Markov random field to carry out binary item smoothing on the segmentation edge, wherein the weight of the unitary item is the pixel value of the segmentation result generated by the segmentation network.

Images