CN113223021B - A UNet-based method for lung X-ray image segmentation - Google Patents

Abstract

The invention discloses a lung X-ray image segmentation method based on UNet, which comprises the following steps: preprocessing a lung X-ray image in a Kaggle data set to obtain a training set; constructing an improved UNet framework by using a method of transfer learning and adding residual blocks; taking the training sample obtained in the step 1 and the lung X-ray segmentation image in the Kaggle data set as input data of the improved UNet, and training an improved UNet segmentation model; introducing the lung X-ray image test data to be segmented into a trained improved UNet model to obtain a segmented result; and performing post-processing on the segmented result to obtain a final segmented result. Experimental results show that the method can effectively solve the segmentation problem of the lung X-ray image, obtains a segmentation method with higher accuracy than a UNet-based lung X-ray image segmentation method, and can be further applied to segmentation of other medical X-ray images.

Description

A UNet-based method for lung X-ray image segmentation

technical field

The invention belongs to the field of image segmentation, relates to the segmentation of lung X-ray images, and can be used in the fields of medical image segmentation, lung disease identification and the like.

Background technique

In recent years, with the continuous improvement of human and computer performance, deep learning has gradually become a common image processing method. Deep learning can be applied to the field of medical image segmentation and recognition. Compared with the experience judgment of doctors, it has higher recognition accuracy and has become a hotspot of current research. Pneumonia is a relatively common lung disease, frequently occurring in children, and lung image segmentation is a major step in pneumonia identification technology. The effect of segmentation directly affects the accuracy of subsequent identification.

In practical applications, the contrast between the lung parenchyma and the background in the lung X-ray image is not high, and the noise generated during the shooting process of the instrument will affect the segmentation and recognition of the lung X-ray image. Although there are many deep learning networks that can solve the problems of image segmentation, recognition and classification, and have achieved remarkable results, there are few methods applied to the field of lung X-ray image segmentation. Small, all increase the difficulty of lung segmentation and lung disease identification. In the field of medical image segmentation, the UNet segmentation model is a commonly used segmentation method. The connection method of adding skip connections improves the accuracy of model segmentation. In addition, UNet is very tolerant to the size of the input image and is a good image segmentation method.

SUMMARY OF THE INVENTION

The invention provides a UNet-based lung X-ray image segmentation method. On the basis of retaining the UNet U-shaped structure, the method of using migration learning, changing the convolution method and adding parallel residual blocks can improve the lung X-ray image. Segmentation accuracy.

For solving the above-mentioned technical problems, the technical scheme of the present invention is as follows:

A UNet-based lung X-ray image segmentation method, comprising the following steps:

S1, preprocessing lung X-ray images in the Kaggle dataset to obtain a training set;

S2, using transfer learning, changing the convolution method and adding residual blocks to build an improved UNet framework;

S3, using the training sample obtained in step S1 and the lung X-ray segmentation image in the Kaggle dataset as the input data of the improved UNet, and training the improved UNet segmentation model;

S4, introducing the lung X-ray image test data to be segmented into the trained improved UNet model to obtain segmented images;

S5, post-processing the segmented result to obtain a final segmented result.

Preferably, in step S1, the original lung X-ray image is preprocessed by using image edge filling, Gaussian filtering, contrast-limiting regional histogram equalization (CLAHE) and data enhancement, and the preprocessing method is specifically:

A1, according to the length and width of the image, expand the original lung X-ray image and the corresponding label image into a square image. Specifically:

When the length and width of the original lung X-ray image are different, the width and length of the image are made the same by adding pixels with a pixel value of 0, so as to obtain the edge-filled lung X-ray image and its corresponding label image;

A2: Perform Gaussian filtering on the original lung image processed in A1 to obtain a denoised lung X-ray image, where the Gaussian kernel is a Gaussian template matrix with a standard deviation of 1 and a size of 3×3;

A3, perform CLAHE on the image processed in A2 to obtain a high-contrast lung X-ray image, in which the number of small areas divided by the image is 8 × 8, and the trimming limit is 2 of the cumulative sum of the pixel values of all pixels in the image %;

A4, the data enhancement of the lung X-ray image processed in A3 and the label image processed in A1 is as follows:

最后将N1、N2、N3和N4共同组成的数据集作为训练神经网络的数据集。The lung X-ray images processed in A3 are corresponding to the label images processed in A1 to form a dataset N1, and the images and labels in N1 are flipped up and down to obtain a dataset N2, and the angle values are angle1 and angle2. Two rotations of to obtain datasets N3 and N4, where

Finally, the dataset composed of N1, N2, N3 and N4 is used as the dataset for training the neural network.

Preferably, in step S2, the method of using transfer learning, changing the convolution mode and adding residual blocks to construct the improved UNet framework is specifically:

B1, using the transfer learning method to improve UNet is as follows:

Based on the UNet segmentation model, the encoding part of the MobileNet network model is transferred to obtain the Mobile_UNet model. The coding part in this model is 5 layers: the first layer includes 2 convolution blocks convblock1 and convblock2, where convblock1 consists of 32 standard convolution kernels with stride 2 and size 3×3, and convblock2 consists of 64 strides It consists of a depthwise separable convolution kernel of size 1 and size 3×3; the second layer includes 2 convolution blocks convblock3 and convblock4, of which convblock3 consists of 128 depthwise separable volumes with stride 2 and size 3×3 Convolution kernels, convblock4 consists of 128 depthwise separable convolution kernels with stride 1 and size 3×3; the third layer includes 2 convolution blocks convblock5 and convblock6, of which convblock5 consists of 256 strides 2, It consists of depthwise separable convolution kernels of size 3×3, convblock6 consists of 256 depthwise separable convolution kernels with stride 1 and size 3×3; the fourth layer includes 6 convolution blocks convblock7-convblock12, Among them, convblock7 consists of 512 depthwise separable convolution kernels with stride 2 and size 3×3, and the remaining 5 convolution blocks consist of 512 depthwise separable convolutions with stride 1 and size 3×3 Kernel composition; the fifth layer includes 2 convolution blocks convblock13-convblock14, of which convblock13 consists of 1024 depthwise separable convolution kernels with stride 2 and size 3×3, and convblock14 consists of 1024 stride 1 and size It consists of 3×3 depthwise separable convolution kernels. The decoding part in the Mobile_UNet model consists of 5 layers: the first four layers are composed of an upsampling module, a standard convolution module and a depthwise separable convolution module. The number of kernels in each module of each layer is (512, 512, 512 ), (256, 256, 256), (128, 128, 128), (64, 64, 64), and the kernel size is 3 × 3; the fifth layer includes a convolution block with 2;

B2, using the method of changing the convolution method to improve UNet is as follows:

Replace the second convolution block of each layer in the decoding part of Mobile_UNet obtained in B1 with a depthwise separable convolution block;

B3, improving UNet by adding residual blocks is as follows:

Add two parallel residual blocks to Mobile_UNet obtained in B1, the first residual block consists of a 3×3 depthwise separable convolutional block, and the second residual block consists of two serial dilation rates respectively consists of 2 and 3 empty convolution blocks.

Preferably, in step S5, post-processing is performed on the result of the segmentation in S4, and the final segmentation result obtained is specifically:

For the post-processing of the segmented images in S4, remove the part of the connected area smaller than 1/5 of the area identified as the lung parenchyma area, obtain the final lung segmentation image and calculate the segmentation accuracy of the lung X-ray image.

Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

The segmentation accuracy of lung X-ray images was improved by using transfer learning, changing the convolution method, and adding parallel residual blocks.

Description of drawings

1 is a flowchart of a UNet-based lung X-ray image segmentation method provided by an embodiment;

Fig. 2 is the lung original X-ray image and label image provided by the embodiment;

Fig. 3 is the preprocessed lung original X-ray image and label image provided by the embodiment;

Fig. 4 is the improved UNet segmentation model that the embodiment provides;

Fig. 5 is the image after the improved UNet segmentation provided by the embodiment;

FIG. 6 is a segmented image after post-processing provided by the embodiment.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, and do not limit the protection scope of the present invention.

The embodiments of the present invention will be further described below with reference to the accompanying drawings.

FIG. 1 is a flowchart of a UNet-based lung X-ray image segmentation method provided by an embodiment. As shown in FIG. 1 , the UNet-based lung X-ray image segmentation method provided in this embodiment includes the following steps:

S1, preprocessing lung X-ray images in the Kaggle dataset to obtain a training set;

S2, using transfer learning, changing the convolution method and adding residual blocks to build an improved UNet framework;

S3, using the training sample obtained in step S1 and the lung X-ray segmentation image in the Kaggle dataset as the input data of the improved UNet, and training the improved UNet segmentation model;

S4, introducing the lung X-ray image test data to be segmented into the trained improved UNet model to obtain a segmented image;

S5, post-processing the segmented result to obtain a final segmented result.

FIG. 2 is a lung X-ray image and a label image in the Kaggle data set provided by the embodiment, and FIG. 3 is a preprocessed original lung X-ray image and label image provided by the embodiment. As shown in FIG. 3 , the preprocessing method of the lung X-ray image provided by this embodiment includes four parts: image edge filling, Gaussian filtering, CLAHE and data enhancement. Specifically:

A1, according to the length and width of the image, expand the original lung X-ray image and the corresponding label image into a square image. Specifically:

When the length and width of the original lung X-ray image are different, the width and length of the image are made the same by adding pixels with a pixel value of 0, so as to obtain the edge-filled lung X-ray image and its corresponding label image;

A2: Perform Gaussian filtering on the original lung image processed in A1 to obtain a denoised lung X-ray image, where the Gaussian kernel is a Gaussian template matrix with a standard deviation of 1 and a size of 3×3;

A3, perform CLAHE on the image processed in A2 to obtain a high-contrast lung X-ray image, in which the number of small areas divided by the image is 8 × 8, and the trimming limit is 2 of the cumulative sum of the pixel values of all pixels in the image %;

A4, the data enhancement of the lung X-ray image processed in A3 and the label image processed in A1 is as follows:

最后将N1、N2、N3和N4共同组成的数据集作为训练神经网络的数据集。The lung X-ray images processed in A3 are corresponding to the label images processed in A1 to form a dataset N1, and the images and labels in N1 are flipped up and down to obtain a dataset N2, and the angle values are angle1 and angle2. Two rotations of to obtain datasets N3 and N4, where

Finally, the dataset composed of N1, N2, N3 and N4 is used as the dataset for training the neural network.

FIG. 4 is an improved UNet segmentation model provided by the embodiment. As shown in Figure 4, using transfer learning, changing the convolution method and adding residual blocks to construct an improved UNet framework is as follows:

B1, using the transfer learning method to improve UNet is as follows:

Based on the UNet segmentation model, the encoding part of the MobileNet network model is transferred to obtain the Mobile_UNet model. The coding part in this model is 5 layers: the first layer includes 2 convolution blocks convblock1 and convblock2, where convblock1 consists of 32 standard convolution kernels with stride 2 and size 3×3, and convblock2 consists of 64 strides It consists of a depthwise separable convolution kernel of size 1 and size 3×3; the second layer includes 2 convolution blocks convblock3 and convblock4, of which convblock3 consists of 128 depthwise separable volumes with stride 2 and size 3×3 Convolution kernels, convblock4 consists of 128 depthwise separable convolution kernels with stride 1 and size 3×3; the third layer includes 2 convolution blocks convblock5 and convblock6, of which convblock5 consists of 256 strides 2, It consists of depthwise separable convolution kernels of size 3×3, convblock6 consists of 256 depthwise separable convolution kernels with stride 1 and size 3×3; the fourth layer includes 6 convolution blocks convblock7-convblock12, Among them, convblock7 consists of 512 depthwise separable convolution kernels with stride 2 and size 3×3, and the remaining 5 convolution blocks consist of 512 depthwise separable convolutions with stride 1 and size 3×3 Kernel composition; the fifth layer includes 2 convolution blocks convblock13-convblock14, of which convblock13 consists of 1024 depthwise separable convolution kernels with stride 2 and size 3×3, and convblock14 consists of 1024 stride 1 and size It consists of 3×3 depthwise separable convolution kernels. The decoding part in the Mobile_UNet model consists of 5 layers: the first four layers are composed of an upsampling module, a standard convolution module and a depthwise separable convolution module. The number of kernels in each module of each layer is (512, 512, 512 ), (256, 256, 256), (128, 128, 128), (64, 64, 64), and the kernel size is 3 × 3; the fifth layer includes a convolution block with 2;

B2, use the method of changing the convolution method to improve UNet, and replace the second convolution block of each layer in the decoding part of Mobile_UNet obtained in B1 with a depthwise separable convolution block;

B3, improving UNet by adding residual blocks is as follows:

Add two parallel residual blocks to Mobile_UNet obtained in B1, the first residual block consists of a 3×3 depthwise separable convolutional block, and the second residual block consists of two serial dilation rates respectively consists of 2 and 3 empty convolution blocks.

FIG. 5 is an image segmented by the improved UNet provided by the embodiment. As shown in Figure 5, the output image has the same size as the original image, only black and white in the image, and white is the identified lung parenchyma area. FIG. 6 is a segmented image after post-processing provided by the embodiment. As shown in Figure 6, the result of the improved UNet segmentation is post-processed to remove the part of the connected area smaller than 1/5 of the area identified as the lung parenchyma area to obtain the final lung segmentation image and calculate the segmentation of the lung X-ray image Accuracy.

The effect of the present invention can be verified by the following experiments:

1. Experimental Conditions

The GPU is NVIDIA GeForce 930M, and the experimental environment is configured on the WINDOWS 10 system, as shown in Table 1.

Table 1 Experimental environment settings

surroundings Version Anaconda 4.7.12 Python 3.7.4 Tensorflow-GPU 1.13.2 Cuda 10.0 Cudnn 7.4.1.5 VSCode 1.56.2

2. Experiment content

Segment the same test data with improved UNet and UNet respectively, and input the processed image into the trained segmentation model with an image size of 192*192*3. The experimental results show that the improved UNet segmentation accuracy has a certain improvement, from the UNet 97.98% segmentation accuracy to 98.06%.

The above-mentioned embodiments of the present invention are only examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those of ordinary skill in the art, changes or modifications in other different forms can also be made on the basis of the above description. There is no need and cannot be exhaustive of all implementations here. Any modifications, equivalent replacements and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the claims of the present invention.

Claims (2)

1. A lung X-ray image segmentation method based on UNet is characterized by comprising the following steps:

s1, preprocessing the lung X-ray image in the Kaggle data set to obtain a training set;

s2, constructing an improved UNet frame by using a method of transfer learning, changing a convolution mode and adding a residual block;

s3, taking the training sample obtained in the step S1 and the lung X-ray segmentation image in the Kaggle data set as input data of the improved UNet, and training an improved UNet segmentation model;

s4, introducing the lung X-ray image test data to be segmented into a trained improved UNet model to obtain a segmented result;

s5, performing post-processing on the segmented result to obtain a final segmented result;

in step S1, the original X-ray lung image is preprocessed by using image edge filling, gaussian filtering, contrast-limited region histogram equalization CLAHE and data enhancement methods, where the preprocessing method specifically includes:

a1, expanding the original lung X-ray image and the corresponding tag image into a square image according to the length and width of the image, specifically:

when the length and the width of an original lung X-ray image are different, adding pixel points with the pixel value of 0 to ensure that the width and the length of the image are the same, thereby obtaining the lung X-ray image after edge filling and a label image corresponding to the lung X-ray image;

a2, performing Gaussian filtering on the original lung image processed in the A1 to obtain a denoised lung X-ray image, wherein a Gaussian kernel is a Gaussian template matrix with a standard deviation of 1 and a size of 3 multiplied by 3;

a3, making CLAHE on the processed image in A2 to obtain a lung X-ray image with high contrast, wherein the number of small areas divided by the image is 8 multiplied by 8, and the trimming limit system is 2% of the sum of pixel values of all pixel points of the image;

a4, performing data enhancement on the lung X-ray image processed in the A3 and the tag image processed in the a1 specifically:

the lung X-ray images processed in A3 and the label images processed in A1 are in one-to-one correspondence to form a data set N1, the images and labels in N1 are turned over up and down to obtain a data set N2, and the data sets N3 and N4 are obtained through two rotations with angle values of angle1 and angle2, wherein the data sets N3 and N4 are obtained

Finally, taking a data set composed of N1, N2, N3 and N4 as a data set for training the neural network;

in step S2, the method for constructing an improved UNet frame by using transfer learning, changing a convolution manner, and adding a residual block specifically includes:

b1, the concrete steps of improving UNet by using the transfer learning method are:

migrating a coding part of a MobileNet network model on the basis of a UNet segmentation model to obtain a Mobile _ UNet model; the coding part in the Mobile _ UNet model is 5 layers: the first layer comprises 2 convolution blocks convblock1 and convblock2, where convblock1 consists of 32 standard convolution cores of 2 steps and 3 × 3 size, and convblock2 consists of 64 depth separable convolution cores of 1 step and 3 × 3 size; the second layer comprises 2 convolution blocks convblock3 and convblock4, where convblock3 consists of 128 depth separable convolution kernels of size 3 x 3 with step size 2, convblock4 consists of 128 depth separable convolution kernels of size 3 x 3 with step size 1; the third layer comprises 2 convolution blocks convblock5 and convblock6, where convblock5 consists of 256 depth separable convolution cores of 2 steps and 3 × 3 size, and convblock6 consists of 256 depth separable convolution cores of 1 step and 3 × 3 size; the fourth layer comprises 6 convolution blocks convblock7-convblock12, wherein convblock7 is comprised of 512 depth-separable convolution kernels of step size 2 and size 3 × 3, and the remaining 5 convolution blocks are comprised of 512 depth-separable convolution kernels of step size 1 and size 3 × 3; the fifth layer comprises 2 convolution blocks convblock13-convblock14, where convblock13 consists of 1024 depth separable convolution kernels of step size 2 and size 3 × 3, and convblock14 consists of 1024 depth separable convolution kernels of step size 1 and size 3 × 3; the decoding part in the Mobile _ UNet model is 5 layers: the first four layers are all composed of an up-sampling module, a standard convolution module and a depth separable convolution module, the number of kernels in each module of each layer is (512, 512, 512), (256, 256, 256), (128, 128, 128), (64, 64, 64), and the kernel size is 3 x 3; the fifth layer comprises a block containing 2 volume blocks;

b2, the specific improvement of UNet by changing the convolution method is:

replacing the second convolution block of each layer in the decoded part of the Mobile _ UNet obtained in B1 with a depth separable convolution block;

b3, the method for improving UNet by adding residual block is specifically as follows:

two parallel residual blocks are added to the Mobile _ UNet obtained in B1, the first residual block consisting of a3 x 3 depth separable convolution block and the second residual block consisting of two consecutive empty convolution blocks with 2 and 3 expansion rates, respectively.

2. The UNet-based lung X-ray image segmentation method as claimed in claim 1, wherein the step S5 performs post-processing on the segmentation result obtained in S4, and the final segmentation result is specifically:

and performing post-processing on the segmented picture in the step S4, removing a part of the connected region with the area smaller than the size of the lung parenchymal region 1/5, obtaining a final lung segmentation image and calculating the segmentation accuracy of the lung X-ray image.

Images