CN111179275A

CN111179275A - A Medical Ultrasound Image Segmentation Method

Info

Publication number: CN111179275A
Application number: CN201911409096.6A
Authority: CN
Inventors: 车博; 袁浩瀚; 罗亮; 陈智; 方俊; 熊雯
Original assignee: University of Electronic Science and Technology of China; Sichuan Provincial Peoples Hospital
Current assignee: University of Electronic Science and Technology of China; Sichuan Provincial Peoples Hospital
Priority date: 2019-12-31
Filing date: 2019-12-31
Publication date: 2020-05-19
Anticipated expiration: 2039-12-31
Also published as: CN111179275B

Abstract

The invention belongs to the technical field of deep learning computer vision and medical information processing, and particularly relates to a medical ultrasonic image segmentation method. On the basis of a general image segmentation neural network model, the method disclosed by the invention integrates multiple input multiple output technology, hole convolution technology, small sample medical data enhancement and other novel technologies, mainly solves the difficult and painful point problems of small sample learning, low ultrasound image contrast, fuzzy nodule edge and the like, and obtains the optimal segmentation strategy disclosed by the invention.

Description

Medical ultrasonic image segmentation method

Technical Field

The invention belongs to the technical field of deep learning computer vision and medical information processing, and particularly relates to a medical ultrasonic image segmentation method.

Background

With the progress of science and technology, the medical imaging technology has been developed greatly, and the ultrasonic imaging technology has important value in prevention, diagnosis and treatment due to the advantages of simple operation, no radiation damage, low cost and the like. Currently, segmenting regions of interest in medical images is the basis for image analysis and lesion identification. An artificial segmentation method is widely adopted to segment the ultrasonic image clinically, and experienced clinicians manually delineate the interested field according to the professional knowledge of the clinicians. However, the manual segmentation is time-consuming and extremely depends on the professional skill and abundant experience of a doctor, and the ultrasonic images have the characteristics of fuzzy edges, low contrast and the like, so that the visual resolution of human eyes is very difficult. Therefore, how to automatically and efficiently segment the ultrasonic image has become a problem that needs to be solved urgently.

In recent years, a Convolutional Neural Network (CNN), a deep neural network model, provides great technical support for improving the segmentation performance of biomedical images. The convolutional neural network can automatically learn low-level visual features and high-level semantic features in the image, and the complex process of manually designing and extracting the image features in the traditional algorithm is avoided. However, conventional CNNs cannot reasonably propagate the underlying features to higher layers. In a semantic segmentation model (U-NET), channel fusion of low-dimensional features and high-dimensional features can be realized by methods such as jump connection and the like, and a good segmentation effect is achieved.

Disclosure of Invention

The invention aims to provide an ultrasonic image segmentation design scheme of a network Multi-scaled-Unet (MD-Unet) based on deep learning in ultrasonic medical image processing so as to obtain better segmentation performance.

The technical scheme adopted by the invention is as follows:

a medical ultrasound image segmentation method, comprising the steps of:

step 1, preprocessing ultrasonic image data to be segmented to obtain training set and verification set data;

step 2, performing data enhancement on the training set and the verification set data, including:

1) increasing the data volume of the training data by adopting offline enhancement: adopting rotation transformation and horizontal turning transformation to perform 10 times of enhancement;

2) enhancing the generalization of the network model by online enhancement: the memory pressure is reduced while the data diversity is enhanced by adopting rotation transformation, scale transformation, scaling transformation, translation transformation and color contrast transformation and adopting an online iterator mode;

step 3, constructing a multi-input multi-output cavity convolution U-shaped network, comprising the following steps:

1) a multi-input down-sampling module: the downsampling module has 4 layers in total, the multi-input adopts the image multi-scale thought, the input data is subjected to size scaling and changed into four pairs of data of 8:4:2:1, and the four pairs of data are respectively fused with a two-three downsampling layer of the network; the down-sampling module utilizes the convolution layer and the maximum pooling layer to complete the acquisition of bottom layer characteristics and sequentially obtain a characteristic diagram; the convolution kernel size of each layer is 3 × 3, a hole convolution r is 2, namely, an interval is added in a conventional convolution kernel so as to increase an image receptive field, and the number of the convolution kernels of the first layer to the fourth layer is 32, 64, 128 and 256 respectively;

2) an up-sampling module: the up-sampling module has 4 layers in total, adopts deconvolution as an up-sampling mode, sequentially enlarges the size of the characteristic image by utilizing the up-sampling module, reduces the number of channels and finally obtains a prediction image with the same size as the input data; the size of the convolution kernel of each layer is 3 multiplied by 3, and the number of the convolution kernels from the first layer to the fourth layer is respectively 256, 128, 64 and 32;

3) the deep supervision multi-output module: carrying out size transformation on the label for 4 times to form four pairs of data of 8:4:2:1, and sequentially using the four pairs of data as training labels of output layers sampled on 4 layers;

step 4, inputting training set data into the constructed U-shaped network for training to obtain a learned convolutional neural network model, and performing parameter adjustment on the verification set until an optimal model and corresponding parameters thereof are obtained to obtain a trained U-shaped network;

and 5, inputting the preprocessed ultrasonic image data to be segmented into the trained U-shaped network to obtain the segmentation result of each pixel.

The invention has the beneficial effects that: the invention provides a segmentation method for an ultrasonic medical image, which integrates multiple input multiple output technology, hole convolution technology, small sample medical data enhancement and other novel technologies on the basis of a general image segmentation neural network model, and mainly solves the difficult and painful point problems of small sample learning, low ultrasonic image contrast, fuzzy nodule edge and the like to obtain the optimal segmentation strategy.

Drawings

Fig. 1 is a design diagram of a medical image segmentation method according to an embodiment of the present invention.

Fig. 2 is a schematic diagram of a data processing module in step 1 according to an embodiment of the present invention.

Fig. 3 is a schematic diagram of a data enhancement module in step 2 according to an embodiment of the present invention.

Fig. 4 is a schematic diagram illustrating an overall structure of the MD-Unet in step 3 according to an embodiment of the present invention.

Fig. 5 is a diagram illustrating the correctness and loss of a training set and a verification set according to an embodiment of the present invention, where (a) is a diagram illustrating a loss function of the training set and the verification set obtained by training using an MD-Unet network, and (b) is a diagram illustrating the correctness of the training set and the verification set.

Fig. 6 is a schematic diagram illustrating an original label and a segmentation image provided by an embodiment of the present invention, where the left side of fig. 6 is the label image, and the right side of fig. 6 is the segmentation result.

Detailed Description

The invention is described in detail below with reference to the following figures and simulations:

the invention provides a network segmentation method based on thyroid nodule ultrasonic images, which comprises 5 steps, and mainly comprises 5 modules of data set acquisition, image preprocessing, network model construction, network training, network testing and evaluation, wherein a flow chart is shown in figure 1. In this embodiment, the specific steps are as follows:

1. the ultrasound image data to be segmented is preprocessed to obtain training set data and test set data, and the data processing flow is shown in fig. 2.

1) Removing the private information and the medical image instrument mark, and screening out an original ultrasonic image which is not manually marked by an imaging doctor;

2) manually marking the label under the guidance of an ultrasound imaging physician;

3) image quality enhancement under the premise of guaranteeing to keep image detail texture characteristics

3-1) reduction of sum noise and non-uniform plaque using adaptive mean filtering

3-2) improving the filtering effect by using two morphological operations-on and off operation

3-3) histogram equalization

3-4) Sobel operator edge enhancement

4) Dividing the data into a training set, a verification set and a test set in a ratio of 6:2:2

5) The image is decolored to obtain a gray image, and the resolution is unified into 256 × 256 by carrying out scale normalization

6) The data label is subjected to binarization processing and is normalized into a [0,1] interval

2. The data enhancement is performed on the training set small sample data, and the flow is shown in fig. 3.

Because the result of deep learning is closely related to the quality and the quantity of data, but medical samples are difficult to acquire, the data volume is small, in order to increase the data volume, avoid overfitting and improve the segmentation precision, two enhanced combination modes are adopted, and the defect of small sample data is overcome.

1) The data volume of training data is increased by adopting offline enhancement, and 10 times of enhancement is mainly performed by adopting rotation transformation and horizontal turnover transformation.

2) And enhancing the generalization of the network model by utilizing online enhancement. Mainly adopts rotation transformation, scale transformation, scaling transformation, translation transformation, color contrast transformation and the like, and reduces the memory pressure while enhancing the data diversity by using an online iterator mode.

3. A multi-input multi-output hole convolution U-shaped network is constructed, and the overall structure of the network is shown in figure 4.

1) Multi-input down-sampling module

The multi-input down-sampling module is shown in the left half of the U-network of fig. 4.

1-1) firstly, the multi-input image multi-scale concept is adopted, input data is subjected to size scaling and is changed into four pairs of data with the ratio of 8:4:2:1, and the four pairs of data are respectively fused with a two-one-three-four sampling layer of an input network.

1-2) the downsampling module has 4 layers in total, and the feature acquisition of the bottom layer is completed mainly by utilizing the convolution layer and the maximum pooling layer, so that feature maps with more channels and smaller size are obtained in sequence. The convolution kernel size of each layer is 3 × 3, and a hole convolution r ═ 2 is adopted, namely, a space is added in a conventional convolution kernel so as to increase the image receptive field. The sizes of the number of convolution kernels of the first layer to the fourth layer are 32, 64, 128 and 256 respectively.

2) Upsampling module

The upsampling module structure is shown in the right half of the U-type network of FIG. 4. The up-sampling module has 4 layers in total, and adopts deconvolution as an up-sampling mode. And the up-sampling module sequentially enlarges the size of the characteristic image, reduces the number of channels and finally obtains a prediction graph with the same size as the input data. The size of the convolution kernel of each layer is 3 × 3, and the number of convolution kernels from the first layer to the fourth layer is 256, 128, 64, 32, respectively.

3) Deep supervision multi-output module

And 4 times of size transformation is carried out on the label to form four pairs of data of 8:4:2:1, and the four pairs of data are sequentially used as training labels of output layers sampled on 4 layers.

4. Inputting training set data into the designed network for training to obtain the learned convolutional neural network model

1) And recording the loss and the segmentation accuracy of each training.

2) And modifying parameters and retraining the network according to the loss and the accuracy on the verification set. Until the best model and its corresponding parameters are selected.

5. Inputting the preprocessed ultrasonic image data to be segmented into the learned convolutional neural network model to obtain the segmentation result of each pixel.

The final results of the practice of the invention are shown here and the results are shown in figures 5 and 6. Fig. 5 is a schematic diagram of the accuracy and loss of a training set and a verification set provided in the embodiment of the present invention, where (a) is a schematic diagram of a loss function of the training set and the verification set obtained by training using an MD-Unet network, and (b) is a schematic diagram of the accuracy of the training set and the verification set. Fig. 6 is a schematic diagram of an original label and a segmentation image provided by an embodiment of the present invention, where the left side of fig. 6 is the label image, and the right side of fig. 6 is the segmentation result.

Claims

1. a medical ultrasound image segmentation method, is characterized in that, comprises the following steps:

Step 1. Preprocess the ultrasound image data to be segmented to obtain training set and validation set data;

Step 2. Perform data enhancement on the training set and validation set data, including:

1) Use offline enhancement to increase the amount of training data: use rotation transformation and horizontal flip transformation to do 10-fold enhancement;

2) Use online enhancement to enhance the generalization of the network model: use rotation transformation, scale transformation, scaling transformation, translation transformation, color contrast transformation, and use online iterators to enhance data diversity while reducing memory pressure;

Step 3. Build a multi-input and multi-output atrous convolutional U-shaped network, including:

1) Multi-input down-sampling module: The down-sampling module has a total of 4 layers. The multi-input adopts the idea of image multi-scale, and the size of the input data is scaled to become four pairs of data of 8:4:2:1, which are respectively related to the input data of the network. One, two, three, and four down-sampling layers are fused; the down-sampling module uses the convolution layer and the maximum pooling layer to complete the acquisition of the underlying features, and obtains feature maps in turn; the size of the convolution kernel of each layer is 3 × 3, and the hole convolution r= 2, that is, adding a space to the conventional convolution kernel to increase the image receptive field. The number of convolution kernels from the first layer to the fourth layer is 32, 64, 128, and 256 respectively;

2) Upsampling module: The upsampling module has a total of 4 layers, using deconvolution as the upsampling method, and using the upsampling module to sequentially expand the size of the feature image, reduce the number of channels, and finally obtain a prediction map of the same size as the input data; The size of the convolution kernel of the layer is 3×3, and the number of convolution kernels from the first layer to the fourth layer is 256, 128, 64, and 32 respectively;

3) In-depth supervision and multi-output module: make 4 size transformations on the label to form four pairs of data of 8:4:2:1, which are sequentially used as the training label of the output layer of the 4-layer upsampling;

Step 4. Input the training set data into the constructed U-shaped network for training, obtain the learned convolutional neural network model, and adjust the parameters on the verification set until the optimal model and its corresponding parameters are obtained, and the trained model is obtained. U-shaped network;

Step 5: Input the preprocessed ultrasound image data to be segmented into the trained U-shaped network to obtain the segmentation result of each pixel.

2. a kind of medical ultrasound image segmentation method according to claim 1 is characterized in that, in data enhancement and hollow convolution U-shaped network module, wherein data enhancement comprises:

1) Improve data utilization through offline enhancement of original data;

2) Through the online enhancement of the original data, the robustness of the network is further enhanced and the memory pressure of the server is reduced.

The atrous convolutional U-network module includes:

1) The image data is scaled by the multi-input module and fused with the down-sampling layer to further enhance the image utilization and improve the network's ability to extract image features;

2) A hole convolution layer is added in the process of downsampling and upsampling, thereby increasing the size of the receptive field and improving the loss of image details caused by convolution.