CN110782427A - Magnetic resonance brain tumor automatic segmentation method based on separable cavity convolution - Google Patents

Abstract

The invention belongs to the field of computer-aided medicine, in particular to a method for automatic segmentation of magnetic resonance brain tumor images based on separable hole convolution. The specific steps of the method of the present invention include: first, dividing the magnetic resonance brain tumor image data set into a training set and a test set, and performing preprocessing operations on the magnetic resonance brain tumor images in the training set; secondly, constructing a convolution based on separable holes MRI brain tumor image depth segmentation network architecture; thirdly, using the preprocessed training set MRI brain tumor images, the constructed separable cavity convolution brain tumor segmentation network is trained end-to-end to obtain an optimized brain tumor segmentation network. Tumor segmentation network model; finally, the trained brain tumor segmentation network model is used to segment the test set MRI brain tumor images. The method of the invention can obtain better segmentation results of the magnetic resonance brain tumor by strengthening the extraction of the discriminative depth feature and the integration of spatial multi-scale information for the magnetic resonance brain tumor image.

Description

Automatic segmentation of magnetic resonance brain tumors based on separable hole convolution

technical field

The invention relates to a medical image segmentation method, in particular to a magnetic resonance brain tumor automatic segmentation method based on separable hole convolution.

Background technique

Glioma is one of the most common and aggressive primary brain tumors, seriously endangering human health. The treatment of brain tumor is mainly surgery, supplemented by radiation therapy, chemotherapy and other comprehensive treatment measures. Magnetic resonance imaging technology has become the main reference for clinical diagnosis and treatment of brain tumors through the characteristics of non-access, non-invasive, multi-directional, multi-parameter imaging, and clear soft tissue display ability. Image analysis and clinical application research are of great significance. It is an indispensable means of extracting quantitative information of special tissues in images, and a prerequisite for realizing three-dimensional visualization and reconstruction of brain tissue. Based on the accurate segmentation results of brain tumors, doctors can obtain various information such as the shape, size, and location of the tumor, perform quantitative analysis, tracking and comparison, and grasp the development and growth status of tumor lesions. The existing brain tumor segmentation methods can be roughly divided into two categories, one is the segmentation method based on traditional machine learning, and the other is the segmentation method based on deep learning.

Traditional MRI brain tumor segmentation methods are mainly based on image processing, computer graphics and traditional artificial intelligence and other field models and methods, including threshold-based methods, region-based methods, model-based methods and classifier-based methods, etc. .

The automatic segmentation of MRI brain tumors based on deep learning has become a hot topic. In recent years, the deep neural convolutional network model has been successfully applied to many computer vision tasks, which can realize the automatic extraction of deep-level and high-discriminatory expressive features. It has been rapidly developed into the field of medical image processing and analysis. In the research of MRI brain image segmentation based on deep learning, a series of important research results have been developed in recent years, and the performance has been greatly improved compared with traditional brain image segmentation methods. However, the current deep learning methods are difficult to improve the ability to extract features and integrate information of various spatial scales. To solve this problem, an automatic segmentation method of brain tumors based on separable hole convolution is proposed.

For example, Chinese Patent Application No. 201580001261.8 discloses a fast magnetic resonance imaging method and device based on a deep convolutional neural network. The method includes: step S1, constructing a deep convolutional neural network; step S2, acquiring offline magnetic resonance image data , train the deep convolutional neural network, and learn the mapping relationship between the under-sampled magnetic resonance image and the full-sampling image; step S3, use the deep convolutional neural network learned in the step S2 to reconstruct the magnetic resonance image. The method utilizes a large number of offline magnetic resonance images to develop its prior information, so that its offline network can recover more fine structures and image features from the magnetic resonance data collected in the previous stage, and make the magnetic resonance sampling multiple and imaging accuracy better. improved. However, this technical solution does not actually design the process of automatically cutting brain tumors, and the information processing capability is also very limited.

SUMMARY OF THE INVENTION

In order to solve the above problems in the prior art, that is, to solve the problem that the current deep learning technology cannot extract accurate features and integrate scale information when performing brain tumor segmentation, the present invention provides a magnetic resonance brain tumor based on separable hole convolution Automatic segmentation method.

A method for automatic segmentation of magnetic resonance brain tumors based on separable hole convolution proposed by the present invention includes the following steps:

Step 1: Divide the magnetic resonance image data into a training set and a test set; the training set adopts a data preprocessing method to generate a processed magnetic resonance image, and the test set is used for model testing, and the segmentation process of the magnetic resonance brain tumor image, specifically including: :

Step 11: construct an image data set including magnetic resonance image data and labels, and divide it into two parts: training set and test set, the training set is used for model training of the present invention, and the test set is used for the model testing stage of the present invention;

Obtain image data and labels, X=[x ₁ , x ₂ ,...,x _N ] represents the sample set composed of all images, each case image is denoted as x _i , {i=1,2,..., N}, N is the number of image samples, each case image has four representations of different magnetic sequences, namely Flair, T1, T2 and T1c modes; Y=[y ₁ , y ₂ ,...,y _M ] represents the label corresponding to the image dataset X. Then, the sample set is divided, and a part is selected as the training sample set X _tr , and the other part is used as the test sample set X _te ;

Step 12: First remove the 1% highest and 1% lowest intensity regions of the training dataset images to get 3D images with a scale of 152×192×146, then cut each 3D image into a series of 2D slice images f1 with a scale of 152× 192; Then, according to the image tumor characteristics, these f1 images are processed in blocks to obtain an image f2 with a scale of 128×128 to solve the data imbalance;

Step 13: Using the z-score normalization process, the data pixel intensities of different magnitudes of the image f2 are converted into uniform intensities to ensure the normalization between the data. The specifics of step 13 include:

Step 131: Find the mean value μ of the overall data of the f2 image pixels; find the standard deviation σ of the overall data of the f2 image pixels;

Step 132: Obtain the f2 image pixel x; use the formula: Acquire, generate processed magnetic resonance brain tumor images.

Step 2, constructing a separable hole convolutional network for the segmentation of post-processing magnetic resonance brain tumor images, which specifically includes the following steps:

Step 21: Use the axial slice size obtained from the processed magnetic resonance image to train the fully convolutional neural network; sequentially obtain each axial slice of the four mode images of the secondary magnetic resonance image; T1, T2 and T1c four modalities of magnetic resonance images on the square block, the same label also uses this processing method; the axial slice size is 128 × 128 × 4, 4 represents Flair, T1, T2 and T1c modal;

Step 22: Input the axial slice obtained in Step 21 into the neural convolutional network with separable hole convolution for brain tumor segmentation;

In step 22, the construction of the separable atrous convolutional network mainly includes the following steps:

Step S221: The encoder network extracts features by constructing a separable hole convolution network. The separable hole convolution block consists of 3×3 separable convolutions and 3×3 convolutions with a hole ratio of 2, regularization and nonlinearity. The activation function is composed, and then the summation operation is performed through the shortcut connection to realize the extraction of brain tumor features;

The encoder network contains 3 separable atrous convolution blocks. The number of channels for extracting features of the 3 convolution blocks is 64, 128, and 256 respectively. The encoder network continuously performs downsampling to extract higher-order information of brain tumors. global semantic information;

Step S: 222: At the bottom layer of the separable hole convolutional network, two residual blocks composed of 3×3 and 1×1 convolution and summation operations are added respectively. The channel numbers of these two residual blocks are 512, 512, this structure is convenient to perceive global information and describe more detailed local feature information;

Step S: 223: In the decoder network, the separable hole convolution network restores features through upsampling and convolution operations, and completes the missing boundary information by merging the encoder network features mapped with it to improve the network features Information capture capability.

In the decoder network, three residual blocks are used for feature restoration, and the number of channels is 256, 128, and 64 respectively. The decoder network cooperates with the encoder network to complete the missing boundary information and perform pixel-level classification through the softmax layer.

Step 3, end-to-end training of the separable hole convolution network model, using the constructed network based on separable volume hole convolution, and sending the processed magnetic resonance image into the network to optimize the network process and improve the segmentation accuracy;

The separable convolutional network is optimized through step 3. During the training process, the loss function is composed of Dice loss and Cross Entropy Loss function, which is used to calculate the error of the network, and then the stochastic gradient descent method is used to continuously optimize the network until reach the optimum.

Step 4: Use the trained separable convolutional network model for MRI brain image segmentation, and send the test set to the network slice by slice, the slice size is 240×240, and the four modal data are sent together at the same time. In the trained model, after processing by the model, the output test segmentation result is generated.

In the above method, the automatic segmentation method of magnetic resonance brain tumor based on separable hole convolution is a deep neural convolution network; the feature extraction network is composed of a separable hole residual network. The MRI brain tumor automatic segmentation neural network with separable hole convolution is a convolutional neural network in series, which is composed of an encoder network and a decoder network, and the encoder network is realized by operations such as convolution and pooling. For the extraction of image features, the extracted features are sent to the decoder network, and the decoder network and the decoder network are connected through corresponding operations to restore the features together to obtain the final segmented magnetic resonance image.

After the above method outputs the brain tumor segmentation results in step 3, it is necessary to perform index evaluation of regional tumors to obtain segmentation effects of tumors in different regions.

The further described indicators are evaluated as: three morbidities in the brain tumor segmentation results corresponding to the four modal magnetic resonance images of Flair, T1, T2 and T1c, which are complete tumors, core tumors and enhanced tumors, respectively. relation. Magnetic resonance brain tumor images have four labels, 0, 1, 2, 4, which in turn indicate that voxels (x, y, z) are labeled as healthy tissue, necrotic and non-enhancing, edematous, and enhanced. The result of segmentation is represented by the evaluation index Dice parameter.

The beneficial effect of the MRI brain tumor automatic segmentation method with separable hole convolution of the present invention is that: the separable hole convolution network is used for feature extraction, and an encoding and decoding network structure is constructed to realize tumor feature extraction and feature fusion, and solve the problem of depth Learning brain tumor segmentation models is difficult in feature extraction and the ability to integrate information from multiple spatial scales. In addition, the verification results of the present invention on the BraTS 2018 challenge data set also achieved good results. The method can be processed and segmented slice by slice to achieve end-to-end segmentation effect, thereby reducing segmentation time and segmentation equipment performance, lowering the segmentation error rate of brain tumors, and reducing the cost of automatic brain tumor segmentation.

Description of drawings

The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments

Fig. 1 is the segmentation model schematic diagram of the magnetic resonance brain tumor automatic segmentation method of the separable hole convolution of the present invention;

Fig. 2 is the flow chart of the magnetic resonance brain tumor automatic segmentation method of separable hole convolution of the present invention;

Figure 3 is a comparison of the results of the MRI brain tumor automatic segmentation method with separable hole convolution and other methods in the BraTS2017 dataset;

Figure 4 is a comparison of the results of the MR brain tumor automatic segmentation method with separable hole convolution and other methods in the BraTS2018 dataset.

Detailed ways

Preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical solutions of the present invention, and are not intended to limit the protection scope of the present invention. For example, although the various steps of the method of the present invention are described in this application in a specific order, these orders are not limiting, and those skilled in the art may perform them in different orders without departing from the basic principles of the present invention. The steps and these transformations all fall within the protection scope of the present application.

Embodiment 1: First of all, please refer to FIG. 1, the present invention uses the neural network to realize the method of separable hole convolution network fusion to solve the difficulty of feature extraction and various spatial scale information in the current deep learning brain tumor segmentation method. Considering the problem of integration ability, this method has faster segmentation speed and higher segmentation accuracy. In the brain tumor segmentation network model in the embodiment, as shown in FIG. 1 , a U-shaped convolutional neural network can be formed by separating hole convolutions.

Please refer to Figure 2, the specific steps are as follows:

Step 1: division of magnetic resonance image data set and preprocessing operation of training data set: 2D slice processing and z-score method are used to perform data imbalance processing and data pixel intensity normalization processing operations to generate processed magnetic resonance images.

Step 2: Brain tumor segmentation: Realize the construction of an automatic segmentation network for magnetic resonance brain tumors based on separable atrous convolutions.

Step 3: Separable hole convolutional network parameter optimization to segment the processed magnetic resonance image.

Step 4: The trained separable atrous convolutional network model is used for segmentation of the MRI brain test set.

The details are as follows:

1. Processing of magnetic resonance images

In this step, the preprocessing steps including the division of the dataset and the images of the MRI brain tumor training set include:

Step 11: Divide the magnetic resonance image data set into training set and test set.

Step 12: For the training set of divided MRI brain images, first remove the 1% highest and 1% lowest intensity regions of the image to obtain 3D data of 152 × 192 × 146, and then cut each 3D image into a series of 2D slice images f1, the scale of f1 image is 152×192; then these f1 images are processed by sliding window to achieve block processing, and the image scale of f2 is 128×128.

Step 13: Perform z-score intensity normalization processing on the magnetic resonance image f2.

z-score normalization is a common method of data processing. It can convert data of different magnitudes into a unified z-score for comparison.

In step 13 of this embodiment, the pixel intensity normalization process is performed by Converting two or more sets of data into unit-free z-score scores unifies data standards, improves data comparability, and weakens data interpretability.

2. Construction of Brain Tumor Segmentation Network

Please refer to Figure 1. In step 2, the deep neural network for automatic segmentation of magnetic resonance brain tumors based on separable hole convolution is a convolutional neural network with an encoder-decoder structure, which is composed of an encoder network and a decoder network in series. The encoding network extracts image features through operations such as separable hole convolution and pooling, and the decoding network obtains image features through upsampling and convolution operations to achieve brain tumor segmentation.

In step 2, the separable convolution and the atrous convolution are borrowed from the networks of Chollet and Yu et al. The correlation and spatial correlation between the channels of the convolution layer can be decoupled, and they can be mapped separately to achieve better results. Effect. The fusion of separable convolution and atrous convolution obtains better local and global feature description capabilities. At the same time, this network makes full use of the channel and region information of the MRI brain image, which can better capture more pixel-level details and spatial information, which is conducive to the feature information extraction of MRI brain segmentation images, enhances the ability to capture features, and achieves more accurate segmentation.

In step 2, specifically, the construction process of the method for automatic segmentation of magnetic resonance brain tumors based on separable hole convolution is as follows:

In step S21, the network is trained by using the small square blocks taken from the axial slices of the processed magnetic resonance images; the square small blocks taken from the axial slices of the processed magnetic resonance images constitute training samples in the training stage, specifically A square block on the magnetic resonance image of Flair, T1, T1c, T2 four modalities centered on a specific pixel, the size is 128 × 128 × 4, and the four channels are Flair, T1, T1c, T2 four modalities respectively. Axial slices. Labels are also processed in this way, corresponding to the label values of the training data.

Step S22 , as shown in FIG. 1 , a schematic diagram of a brain tumor segmentation model of the method for automatic segmentation of magnetic resonance brain tumors with separable hole convolution of the present invention. Among them, the MRI brain tumor automatic segmentation network with separable convolution is trained with a scale of 128 × 128 × 4 in the training stage, and the feature information is restored by downsampling, and finally classified by the softmax layer.

3. Model optimization

The training prediction value will be used to modify the calculation of the Dice loss loss function. In the training phase, an image of a scale size is input, and an image of the same size is output, and the pixel classification prediction is performed, i.e. healthy tissue, necrotic and non-enhanced, edema and enhanced. probability value. The parameters in the entire separable atrous convolutional network are then trained using an error back-propagation algorithm.

It should be noted that the training samples are obtained by random sampling on the training data, and the data balance is processed by removing and blocking. The network is trained in small blocks of axial slices used in this stage, which can increase the number of training samples on the one hand, and control the number of samples of different categories on the other hand, which is conducive to achieving sample balance. The network is a fully convolutional neural network that implements an end-to-end approach and uses 2D slices for testing, which speeds up network testing.

Step 3, using the axial slices of the processed magnetic resonance images, to optimize the parameters of the automatic segmentation method of magnetic resonance brain tumors fused with separable hole convolutions. The network parameters for automatic segmentation of magnetic resonance brain tumors based on separable atrous convolutions are simultaneously tuned in the error back-propagation stage.

Step 4, using the trained separable atrous convolutional network model for the test set MRI brain images for segmentation.

To sum up, the automatic segmentation network of magnetic resonance brain tumor based on separable hole convolution of the present invention realizes the extraction of image features through separable hole convolution fusion, and solves the difficulty in feature extraction and various spatial problems of deep learning brain tumor segmentation model. In addition, good results have been obtained on both the BraTS 2017 dataset and the BraTS 2018 dataset.

Claims (6)

1. A magnetic resonance brain tumor image automatic segmentation method based on separable cavity convolution is characterized in that: the method comprises the following steps:

step 1: dividing a magnetic resonance brain tumor image data set into a training set and a testing set, wherein the training set is used for model training of the method, the testing data is used for model testing of the method, preprocessing operations are carried out on magnetic resonance brain tumor images in the training set, and a preprocessed magnetic resonance brain tumor image training set is generated;

step 2: constructing a magnetic resonance brain tumor image depth segmentation network framework based on separable cavity convolution;

and step 3: and performing end-to-end training on the constructed separable cavity convolution brain tumor segmentation network by adopting the preprocessed training set magnetic resonance brain tumor image to obtain an optimized brain tumor segmentation network model.

And 4, step 4: and segmenting the test set magnetic resonance brain tumor image by adopting the trained brain tumor segmentation network model.

2. The method of claim 1, wherein the method comprises: the preprocessing operation of the training set magnetic resonance brain tumor image in the step 1 comprises the following steps:

removing the areas of the invalid highest and lowest intensities of the three-dimensional data of the magnetic resonance brain tumor image, namely removing invalid background pixels, and then cutting the processed three-dimensional image into a series of two-dimensional images f1 along the Z axis; then, the two-dimensional f1 images are sliced in a sliding window mode, the size of the slice data is determined by a tumor area, 3 slice images f2 are obtained by one two-dimensional f1 image slice, and the labels are processed in the same mode to realize block processing so as to solve data imbalance; and finally, converting the intensities of the data pixels with different magnitudes of the image f2 into uniform intensities by utilizing a z-score method for normalization processing so as to ensure comparability between data and generate a processed magnetic resonance image.

3. The method of claim 2, wherein the method comprises: in step 1, the preprocessed magnetic resonance image includes magnetic resonance images of four modalities, Flair, T1, T2, and T1 c.

4. The method of claim 1, wherein the method comprises: and 2, constructing a magnetic resonance brain tumor image depth segmentation network architecture based on separable cavity convolution, wherein the deep learning network architecture is formed by connecting separable convolution blocks, double residual blocks and residual blocks in series. The separable convolution block and the double-residual block have the following specific structures:

s31, separable rolling block

(1) The separable convolution block comprises two convolution layers, wherein the first convolution layer consists of a regularization function, an activation function and a separation convolution, the second convolution layer consists of a regularization function, an activation function and a hole convolution, and the number of channels of the two convolution layers is consistent;

(2) then, using a residual error thought shortcut connection mode to perform summation operation on the input features and the features processed by the separable rolling blocks;

(3) finally, combining the two parts to obtain a final separable rolling block;

s32. double-residual block module

(1) The double-residual-block module is arranged at the bottom layer of the separable cavity convolution network and comprises two residual blocks, the two double-residual-block processing modes are different, the first residual block is constructed by two convolutions, the extraction of features is realized, and the shortcut principle of a residual net is mainly utilized;

(2) the second residual block is constructed by two convolutions, each convolution layer is regularized into and activated with a function, then processed by convolution, and finally summed with the features processed by the residual block.

5. The method of claim 1, wherein the method comprises: in step 3, the end-to-end training including the implementation of the separable hole convolution network model is optimized: based on the inverse propagation of the separable hole convolutional network, a modified Diceloss loss function is used as a target function, and the modified Diceloss loss function is mapped to the prediction probability of each label through the operation of an activation function softmax; based on the method, a random gradient descent method is adopted in the back propagation, the deviation of each parameter is calculated layer by using a chain type rule, the back propagation error is determined, the parameters of the network model are continuously updated, and the separable cavity convolution network is optimized.

6. The method of claim 1, wherein: in step 4, the trained brain tumor segmentation network model is adopted to segment the test set magnetic resonance brain tumor image, and the test network process is as follows:

firstly, 3D of test set data is converted into 2D data through Z-axis slicing, and then the slice data is sent into a training model to carry out test data segmentation.

Images