CN112734755A - Lung lobe segmentation method based on 3D full convolution neural network and multitask learning - Google Patents

Abstract

The invention discloses a lung lobe segmentation method based on a 3D full convolution neural network and multitask learning, and belongs to the field of automatic lung lobe segmentation. The invention comprises the following steps: preparing and calibrating lung lobe related data; preprocessing original lung lobe CT image data and lung lobe related data with accurate labels to remove redundant background information; constructing a 3D full convolution neural network based on multitask learning; training the constructed 3D full convolution neural network by using the preprocessed calibrated lung lobe related data and the synthesized learning error; and performing lung lobe segmentation on the input three-dimensional lung lobe CT image by using the trained 3D full convolution neural network, and outputting a predicted lung lobe label. The invention can receive the lung lobe CT image data with original size and automatically and quickly generate an accurate lung lobe segmentation result.

Description

Lung lobe segmentation method based on 3D full convolution neural network and multitask learning

Technical Field

The invention relates to the field of automatic lung lobe segmentation, in particular to a lung lobe segmentation method based on a 3D full convolution neural network and multitask learning.

Background

With the widespread use of Computed Tomography (CT) technology in hospitals, CT has become one of the main technologies for diagnosis and treatment of lung diseases. Lung lobe segmentation is crucial in the qualitative and quantitative analysis of lung diseases, such as the localization of lung nodules and the generation of diagnosis and treatment reports. The lung can be anatomically divided into five functionally independent parts: the upper left lung lobe, the lower left lung lobe, the upper right lung lobe, the middle right lung lobe, and the lower right lung lobe. The boundaries of the lung lobes are called as fissures, and since the fissures are usually incomplete and unclear, it is cumbersome to segment the lung lobes manually, and labeling a CT typically takes 2-4 hours. Through a deep learning method, the constructed neural network can rapidly and accurately perform automatic lung lobe segmentation from CT, and the method has great significance for auxiliary diagnosis and treatment of lung diseases.

In recent years, there have been many studies on methods for automatic lung lobe segmentation. These methods can be mainly divided into two parts: a conventional method and a deep learning method. The traditional method mainly detects the lung lobe boundary, namely the lung fissure, through an algorithm of graphic imaging, so that the segmentation result of the lung lobes is finally obtained.

Where Zhou et al perform a search for lung fissures based on anatomical features where vessels connect the lobes but do not cross the lobe boundaries, they assign segmented vessels to the lobes through bronchial information, the gaps between the lobes are defined as target areas of lung fissures, they use edge detection filtering to detect lung fissure points among these target areas and connect them with morphological operations. Kuhnigk et al also used vessel information, but in contrast, they used Euclidean distance conversion and watershed conversion algorithms on vessels to enhance the gaps between lung lobes that guided the distribution of lobes during the watershed conversion through marker points of the bronchial tree. Wei et al use the estimated fissure direction to search for regions of interest without vessels and bronchi and then use a two-dimensional wavelet transform to detect possible fissures. Pu et al apply a threshold to detect candidate lung break points, which are used to construct a 3D geometric mesh. They use a continuous smoothing operation to reduce the mesh into a series of approximate lung segment segments, and separate and label them according to their orientation. The detected fissures segments are then expanded using radial basis functions to form a complete fissures surface.

In recent years, a method for extracting deep features of an image based on a neural network is rapidly developed, the deep neural network is successfully applied to the field of medical image segmentation, and the full convolution neural network is particularly widely applied to the field of segmentation by adopting input with any size, effectively deducing and learning a feature hierarchical structure and generating output with corresponding size in an end-to-end mode. Harrison et al uses a 2D region of interest of a 2D progressive and multi-path whole nested neural network (P-HNN) fissuring lung, and then a 3D random walk algorithm is applied to the resulting fissuring map to perform segmentation of the segmentation. Park et al used a 3D version of the typical network U-Net-3D U-Net to segment lung lobes. Ferreira et al propose regularized V-Net (FRV-Net) and combine Dice loss function with Focal loss function to obtain reliable lung lobe segmentation results. Lee et al propose segmentation of lung lobes based on 3D depth separable convolution, a hole residual dense convolution block, and an input enhanced PLS-Net.

The conventional method for segmenting the lung lobes by detecting the boundaries between the lung lobes, namely the lung fissures, based on the image graphics has obvious limitations. These methods require information on auxiliary structures such as blood vessels, bronchi, lungs, etc. in order to reinforce the fissured areas, and the acquisition of these auxiliary structures is often difficult in practice. The traditional method has a complex flow for segmenting the lung lobes and long time required in the whole process, so that the traditional method is difficult to be applied to a real-time medical auxiliary system. In addition, when the lung of a patient has a focus area, which causes the lung fissure to be cut off, the traditional method has a poor effect of segmenting the lung lobes. The deep learning method based on the neural network overcomes the defect that the traditional methods need auxiliary structure information, and the methods can learn characteristics from data in a supervised learning mode so as to automatically segment lung lobes. However, the neural network based on the U-Net and V-Net structures has a large number of parameters and a large video memory requirement, so that the input is generally resampled to lose a lot of original information, and meanwhile, the receptive field acquired by the network is limited, and the defects make the network difficult to generate an accurate lung lobe segmentation result. PLS-Net based on 3D depth separable convolution, hole residual dense convolution block and input enhancement can extract more extensive context information quickly due to the fact that the hole convolution increases the receptive field, and the result is still not accurate enough in segmentation of the lung lobe boundary region.

Disclosure of Invention

The invention aims to provide a lung lobe segmentation method based on a 3D full convolution neural network and multitask learning, which can receive lung lobe CT image data with original size and automatically and quickly generate an accurate lung lobe segmentation result.

The invention solves the technical problem, and adopts the technical scheme that: a lung lobe segmentation method based on a 3D full convolution neural network and multitask learning comprises the following steps:

a. preparing and calibrating lung lobe related data;

b. preprocessing original lung lobe CT image data and lung lobe related data with accurate labels to remove redundant background information;

c. constructing a 3D full convolution neural network based on multitask learning;

d. training the constructed 3D full convolution neural network by using the preprocessed calibrated lung lobe related data and the synthesized learning error;

e. and performing lung lobe segmentation on the input three-dimensional lung lobe CT image by using the trained 3D full convolution neural network, and outputting a predicted lung lobe label.

Further, in step a, the prepared lung lobe related data is imported from a lung lobe CT image data system.

Further, in the step a, when the lung lobe related data is calibrated, the used data labels include a lung lobe label and a lung lobe boundary label;

for the lung lobe label, the image of each lung lobe CT image inspection is calibrated manually;

for the lung lobe boundary label, the lung lobe label is automatically obtained through calibration, and the specific method comprises the following steps: generating a lung lobe boundary label from the lung lobe label by using a boundary searching method of an image processing pack Skimage, and applying Gaussian filtering to the generated lung lobe boundary label to smooth a boundary;

for incomplete lung fissure in a lung lobe CT image, calibrating the incomplete lung fissure according to relevant anatomical structures of the lung, wherein the relevant anatomical structures are blood vessels and/or bronchus, and the lung fissure refers to: the boundary between the lobes of the lungs.

Further, the step b is specifically as follows: and carrying out crop processing and normalization processing on the original lung lobe CT image data and the lung lobe related data with the accurate labels, and respectively cutting off 20 pixel values of the original lung lobe CT image data and the lung lobe related data with the accurate labels along the edges of three dimensions, so that the original lung lobe CT image data and the lung lobe related data with the accurate labels are normalized to be between [0-1 ].

Further, in step c, the multi-task learning specifically includes: adding a convolution of 1 x 1 and sigmoid activation function to the last layer of a decoder of the 3D full convolution neural network to generate a segmentation result of the lung lobe boundary, and training by using a Focal learning error so that the network can learn the lung lobe boundary segmentation task and the lung lobe segmentation task simultaneously in a back propagation process.

Further, in the step c, the constructed 3D full convolution neural network is a 3D-Unet network, a V-net network or a lightweight three-dimensional full convolution network.

Further, when the constructed 3D full convolution neural network is a lightweight three-dimensional full convolution network, the network can separate convolution, cavity residual dense convolution blocks and input enhancement based on 3D depth, and is used for extracting multi-scale and multi-type data features in data on the premise of using a small amount of parameters and video memory;

the 3D deep separable convolution is used for dividing general convolution operation into two steps to reduce the parameter quantity of the network and reduce the video memory requirement when training three-dimensional data;

the cavity residual error dense convolution block is used for increasing the receptive field of a network through the separation convolution with the connection expansion rate increasing progressively, expanding the range of captured spatial information, and simultaneously applying dense connection and residual error learning for back propagation of learning errors;

the input enhancement is used to supplement the information lost by the data during the down-sampling process.

Furthermore, in the step d, cross entry classification learning errors are used for the lung lobe segmentation task, Focal learning errors are used for the lung lobe boundary segmentation task, the weights occupied by the two tasks in the training process are adjusted by using two variable weight parameters, and the whole learning errors of the network are synthesized.

Further, the learning error of the synthetic network as a whole specifically includes:

setting P as the lung lobe prediction generated by the network in the lung lobe segmentation task, G as the lung lobe label in the lung lobe segmentation task, C as the number of total classes, wherein the total classes in the lung lobe segmentation task are composed of five lung lobe classes and one background class, namely C is 6, N is the total number of pixels, and setting the ith pixel in the lung lobe label

Belong to class c, then

Is 1, if not c, is 0;

for the probability that the ith pixel in the lung lobe prediction belongs to the class c, the range is [0, 1]]Therefore, the Cross entry classification learning error of the lung lobe segmentation task is:

setting up

Lung lobe boundary prediction results generated for the sigmoid layer in the lung lobe boundary task,

setting the ith pixel in the lung lobe boundary label for the lung lobe boundary label automatically generated by the lung lobe label

Belong toBoundary of lung lobes

Is 1, otherwise is 0,

the probability that the ith pixel in the lung lobe boundary prediction is the lung lobe boundary is in the range of 0,1]And gamma is a modulation coefficient for controlling the weight of the easily classified sample, so that the learning error in the lung lobe boundary task is defined as:

setting up

And

in order to adjust the parameters of the weights of the lung lobe segmentation task and the lung lobe boundary segmentation task, the learning error of the whole network is as follows:

。

further, in the step D, when the constructed 3D full convolution neural network is trained, the Adam optimization algorithm is adopted as the training optimization algorithm, the initial learning rate is set to be 0.001, and the weight attenuation parameter is set to be 0

If the error of a single case is not reduced after the training of 20 cases of data continuously, the learning rate is multiplied by the attenuation coefficient of 0.8;

and updating parameters once for each batch by network learning, judging the total error of the lung boundary detection result by the model after each iterative learning, if the current error is smaller than the error of the last iteration, saving the current model, continuing training, and if the training reaches the maximum iteration number or the total error does not decrease after 10 iterations, stopping the training.

The lung lobe segmentation method based on the 3D full convolution neural network and the multitask learning has the advantages that the multitask learning integrates the lung lobe segmentation task and the lung lobe boundary segmentation task, the lung lobe information and the lung lobe boundary information are learned at the same time to generate a lung lobe segmentation result with an accurate boundary, the advantages of a deep neural network are fully exerted, and the lung lobe segmentation method based on the 3D full convolution neural network and the multitask learning can be used for automatically, quickly and accurately segmenting healthy and diseased lungs without the need of deducing an auxiliary structure.

Drawings

Fig. 1 is a flowchart of a lung lobe segmentation method based on a 3D full convolution neural network and multi-task learning according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a lung boundary detection model based on a 3D full convolution neural network and multi-task learning according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a 3D depth separable convolution according to an embodiment of the present invention;

FIG. 4 is a diagram of a hole dense residual convolution block according to an embodiment of the present invention.

Detailed Description

The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

Examples

The embodiment of the invention provides a lung lobe segmentation method based on a 3D full convolution neural network and multitask learning, a flow chart of the method is shown in figure 1, wherein the method comprises the following steps:

a. preparing and calibrating lung lobe related data;

b. preprocessing original lung lobe CT image data and lung lobe related data with accurate labels to remove redundant background information;

c. constructing a 3D full convolution neural network based on multitask learning;

d. training the constructed 3D full convolution neural network by using the preprocessed calibrated lung lobe related data and the synthesized learning error;

e. and performing lung lobe segmentation on the input three-dimensional lung lobe CT image by using the trained 3D full convolution neural network, and outputting a predicted lung lobe label.

On the basis of the above steps, in the implementation process of the embodiment, the specific steps are as follows:

s1, data preparation

The deep neural network approach requires high quality data for training, so the data needs to be prepared first for training of the model. The 3D full convolution network adopted in this embodiment can learn the characteristics of data from a small amount of data, and the data used is a lung CT image of the department of imaging in western hospital, and thirty-two patient data are collected.

In this embodiment, the method used for training the model is a method based on supervised learning, and the iterative update of the model requires data with accurate labels. Two kinds of tags are required for the data used: a lobe label and a label of a lobe boundary. For the labels of the lung lobes, for the image of each lung lobe CT examination, the labeling and calibration are carried out by the imaging experts of the Western Wash Hospital, so that the accuracy and the objectivity of each label are ensured. For incomplete lung fissure in lobe CT, the labeling is performed according to other anatomical structures of the lung, such as blood vessels and bronchi.

The lung lobe boundary label is automatically obtained through the labeled lung lobe label, and the specific method is that the lung lobe boundary label is generated from the lung lobe label by using a public boundary searching method of the image processing package Skyimage, and then the generated lung lobe boundary label is applied with Gaussian filtering to smooth the boundary.

S2, preprocessing data

The lung lobe segmentation method in the embodiment does not need a complex preprocessing flow. In the data preprocessing stage, simple crop processing and normalization processing are carried out on original CT image data, and since the external region of a CT human body is redundant to the segmentation task of lung lobes, in order to reduce the video memory amount and accelerate the training and deducing time on the premise of keeping a complete lung region, the original CT data are respectively cut off by 20 pixel values along the edges of three dimensions. For example, a CT data size of 300 × 512 is converted into a CT size of (320-20 × 2) ((512-20 × 2)), or 280 × 472, (by crop).

In order to ensure the consistency of the CT data and the calibration data, the calibration data is subjected to the same crop step. The CT data obtained in the above steps are then normalized to between [0-1 ].

S3 construction of lung lobe segmentation model and application of multi-task learning

The lung nodule candidate detection model used in the present embodiment consists of two parts: the method comprises the following steps that firstly, a lightweight full convolution neural network (PLS-Net) provided by Lee et al is used as a main feature extraction network; the other is Multi-task Learning (Multi-task Learning) which is used for simultaneously Learning the lung lobe and the lung lobe boundary and improving the accuracy of the lung lobe segmentation in the lung lobe boundary region from the shared information of the lung lobe boundary task. Fig. 2 shows a schematic diagram of a lung boundary detection model, and because only a three-dimensional CT image is considered in this embodiment, the full convolution neural network structure used by the lung lobe segmentation model in this embodiment includes different types of network layers, and because input data has features of three dimensions, all modules used by the network model are operated in three dimensions. The detailed arrangement of each section is as follows:

1. 3D full convolution neural network: the three-dimensional full convolution network structure adopted in the embodiment refers to a lightweight three-dimensional full convolution network structure proposed by Lee and the like. The network enhances extraction of multi-scale data features based on 3D depth separable convolution, hole residual dense convolution blocks and input. The 3D depth separable convolution performs the conventional 3 x 3 convolution operation in two steps, i.e., depth convolution and dot convolution. The depth convolution performs 3D spatial convolution independently on each input channel, while the point-to-point convolution applies a 1 × 1 × 1 convolution to combine the depth convolution outputs and project them onto the new channel space, ultimately producing an output through batch normalization and ReLU. The conventional 3D convolution operation has high calculation cost and a large number of parameters. The 3D deep separable convolution captures spatial and cross-channel correlations separately by decomposing it explicitly into two simpler steps, making this operation more efficient, and greatly reducing the number of parameters of the network, alleviating the video memory requirements for training three-dimensional data, see fig. 3; the hole residual dense convolutional block increases the Receptive field (Receptive field) of the network by a separate convolution with increasing concatenation Dilation rate (scaling rate), which contains four densely concatenated 3 × 3 × 3 convolutional layers, with a Dilation rate r = (1,2,3,4), followed by 1 × 1 × 1 convolutional layers and a residual learning mechanism. The increasing expansion ratio is such that the receptive field of each expanded convolutional layer arranged in a stacked manner completely covers a cubic region without any voids. In this way, the layers can work interdependently to mitigate the gridding problem caused by the dilation convolution, while exponentially expanding the field of view without the need for additional parameters and calculations. In order to merge multi-scale context features, the dense convolution cascade module introduces dense connection, namely each layer is directly connected to all subsequent layers, captured spatial information is wider, and meanwhile backward propagation of a loss function is facilitated. Referring to fig. 4, D represents the number of input channels of the hole residual dense convolution block, g represents the growth rate, i.e., the size of the number of channels of the output generated by the 3D separable convolution in the module, and g is set to 12 in this network; the whole network is shown in fig. 2, which is divided into two parts, namely an encoder for extracting context features of deep-level features and a decoder for decoding corresponding features and generating lung lobe segmentation with corresponding sizes. Each cube represents a three-dimensional data input, and the number at the bottom right of the cube represents the number of channels for that input. The input enhancement then complements part of the information lost in the data down-sampling process, specifically the output resulting from each down-sampling operation (3D separable convolution with stride of 2) at the encoder is concatenated with the original input of the corresponding size. The spatial resolution of the feature map generated by the last layer of the encoder is one eighth of the original resolution, the decoder generates 12 channels of output by convolution of the outputs of the x layer and the x-1 layer of the encoder, the feature map of the x-1 layer is spliced with the feature map of the x layer which is up-sampled by 2 times through trilinear interpolation, the convolution, up-sampling and splicing processes are repeated until the resolution of the feature map is consistent with the original input, and finally the feature map is used for generating 6 channels of lung lobe segmentation prediction results (respectively representing 5 lung lobes and the background) through 1 × 1 × 1 convolution and a softmax activation function. The network can rapidly extract multi-scale and multi-type features in data on the premise of using a small amount of parameters and video memory, and has strong learning ability.

2. Multi-task learning: multi-task Learning (Multi-task Learning) is a method that enables models to achieve better performance of the original task by sharing expressions among related tasks. Multi-task learning improves the model generation capability by utilizing domain-specific information contained in the relevant task training. In the lung lobe segmentation, the lung fissure near the lung lobe boundary is often incomplete, and in addition, the boundary of the lung lobe is difficult to distinguish due to some pathological changes, so that the segmentation effect of the current lung lobe segmentation method on the lung lobe boundary is generally poorer than that of the lung lobe inner region. In order to solve the problem, the lung lobe boundary segmentation task is combined with the lung lobe segmentation task as an auxiliary task, and the aim is to improve the segmentation performance of the model in the lung lobe boundary region through the shared information expression of the auxiliary task.

Specifically, the last layer of the decoder of the 3D full convolution neural network is added with a convolution of 1 x 1 and sigmoid activation function to generate a segmentation result of the lung lobe boundary, and the segmentation result is trained by using a corresponding learning error, so that the network can learn the lung lobe boundary segmentation task and the lung lobe segmentation task simultaneously in a back propagation process. Since the lung lobe boundary data has the phenomenon of extreme imbalance, namely the lung lobe boundary area only occupies a small part, the Focal loss function is used for training the lung lobe boundary task.

S4 training of network

The present embodiment performs training based on the network structure designed in step S3. The network training is mainly divided into the following steps:

s401, Cross Engine Classification learning error and Focal learning error. Learning errors directly affect how well the model is trained. The lung lobe boundary data is extremely unbalanced, that is, the lung lobe boundary area only occupies a small part.

In the embodiment, cross entry classification learning errors are used for a lung lobe segmentation task, Focal learning errors are used for a lung lobe boundary task to pay attention to a small number of samples and difficultly-divided samples, and two variable parameters are used for adjusting the weight occupied by the two tasks in the training process.

In this embodiment, P is the lung lobe prediction generated by the network in the lung lobe segmentation task, and G is the lung lobe segmentationIn the lung lobe segmentation task, the total class is composed of five lung lobe classes and a background class, namely C is 6, N is the total number of pixels, and if the ith pixel in the lung lobe label is set

Belong to class c, then

Is 1, if not c, is 0;

for the probability that the ith pixel in the lung lobe prediction belongs to the class c, the range is [0, 1]]Therefore, the Cross entry classification learning error of the lung lobe segmentation task is:

setting up

Lung lobe boundary prediction results generated for the sigmoid layer in the lung lobe boundary task,

setting the ith pixel in the lung lobe boundary label for the lung lobe boundary label automatically generated by the lung lobe label

The boundary of lung lobe

Is 1, otherwise is 0,

the probability that the ith pixel in the lung lobe boundary prediction is the lung lobe boundary is in the range of 0,1]And gamma is a modulation coefficient for controlling the weight of the easily classified sample, so that the lung lobe boundary task middle schoolThe learning error is defined as:

setting up

And

in order to adjust the parameters of the weights of the lung lobe segmentation task and the lung lobe boundary segmentation task, the learning error of the whole network is as follows:

。

s402, network training

Since the input is three-dimensional data, in order to alleviate the requirement of the network on video memory, a hybrid precision training method and a breakpoint training method are used for training the network. The training optimization algorithm adopts an Adam optimization algorithm, the initial learning rate is set to be 0.001, and the weight attenuation parameter is set to be

. If the error of a single case does not decrease after 20 consecutive data training, the learning rate is multiplied by the attenuation coefficient of 0.8. Since the dimensions of each instance of input data are different. The training batch is set to 1, and the number of learning iterations is 100. The expansion ratio in the hole residual dense volume block is set to (1,2,3, 4).

The network training adopts a BP feedback propagation algorithm, meanwhile learns a lung lobe segmentation task by using a classification error, and learns a lung lobe boundary segmentation task by using a Focal error. The parameters are updated once per batch by web learning. After each iteration learning, the model judges the total error of the lung boundary detection result, if the current error is smaller than the error of the last iteration, the current model is saved, and then the training is continued. If the training reaches the maximum iteration number or the total error does not decrease after 10 iterations, the training is stopped.

Therefore, the method of the embodiment can accept the original size of the CT data and automatically and quickly generate an accurate lung lobe segmentation result. The lightweight 3D full convolution neural network can quickly and efficiently extract multi-scale context information from CT data; the multi-task learning integrates the lung lobe segmentation task and the lung lobe boundary segmentation task, and meanwhile, the lung lobe information and the lung lobe boundary information are learned to generate a lung lobe segmentation result with accurate boundary.

Claims (10)

1. A lung lobe segmentation method based on a 3D full convolution neural network and multitask learning is characterized by comprising the following steps:

a. preparing and calibrating lung lobe related data;

b. preprocessing original lung lobe CT image data and lung lobe related data with accurate labels to remove redundant background information;

c. constructing a 3D full convolution neural network based on multitask learning;

d. training the constructed 3D full convolution neural network by using the preprocessed calibrated lung lobe related data and the synthesized learning error;

e. and performing lung lobe segmentation on the input three-dimensional lung lobe CT image by using the trained 3D full convolution neural network, and outputting a predicted lung lobe label.

2. The method for segmenting lung lobes based on 3D full convolution neural network and multitask learning according to claim 1, characterized in that in step a, the prepared lung lobe related data is imported from a lung lobe CT image data system.

3. The method for segmenting lung lobes based on 3D full convolution neural network and multitask learning as claimed in claim 1, wherein in the step a, when the lung lobe related data is calibrated, the labels of the used data comprise lung lobe labels and lung lobe boundary labels;

for the lung lobe label, the image of each lung lobe CT image inspection is calibrated manually;

for the lung lobe boundary label, the lung lobe label is automatically obtained through calibration, and the specific method comprises the following steps: generating a lung lobe boundary label from the lung lobe label by using a boundary searching method of an image processing pack Skimage, and applying Gaussian filtering to the generated lung lobe boundary label to smooth a boundary;

for incomplete lung fissure in a lung lobe CT image, calibrating the incomplete lung fissure according to relevant anatomical structures of the lung, wherein the relevant anatomical structures are blood vessels and/or bronchus, and the lung fissure refers to: the boundary between the lobes of the lungs.

4. The lung lobe segmentation method based on the 3D full convolution neural network and the multitask learning according to claim 1, wherein the step b specifically comprises: and carrying out crop processing and normalization processing on the original lung lobe CT image data and the lung lobe related data with the accurate labels, and respectively cutting off 20 pixel values of the original lung lobe CT image data and the lung lobe related data with the accurate labels along the edges of three dimensions, so that the original lung lobe CT image data and the lung lobe related data with the accurate labels are normalized to be between [0-1 ].

5. The method for segmenting lung lobes based on the 3D full convolution neural network and the multitask learning according to claim 1, wherein in the step c, the multitask learning specifically includes: adding a convolution of 1 x 1 and sigmoid activation function to the last layer of a decoder of the 3D full convolution neural network to generate a segmentation result of the lung lobe boundary, and training by using a Focal learning error so that the network can learn the lung lobe boundary segmentation task and the lung lobe segmentation task simultaneously in a back propagation process.

6. The method for segmenting lung lobes based on 3D full convolution neural network and multitask learning according to claim 1 or 5, characterized in that in step c, the constructed 3D full convolution neural network is 3D-Unet network or V-net network or lightweight three-dimensional full convolution network.

7. The lung lobe segmentation method based on 3D full convolution neural network and multitask learning as claimed in claim 6, characterized in that when the constructed 3D full convolution neural network is a lightweight three-dimensional full convolution network, the network is based on 3D depth separable convolution, cavity residual dense convolution block and input enhancement, and is used for extracting multi-scale and multi-type data features in data on the premise of using a small amount of parameters and video memory;

the 3D deep separable convolution is used for dividing general convolution operation into two steps to reduce the parameter quantity of the network and reduce the video memory requirement when training three-dimensional data;

the cavity residual error dense convolution block is used for increasing the receptive field of a network through the separation convolution with the connection expansion rate increasing progressively, expanding the range of captured spatial information, and simultaneously applying dense connection and residual error learning for back propagation of learning errors;

the input enhancement is used to supplement the information lost by the data during the down-sampling process.

8. The method for segmenting lung lobes based on 3D full convolution neural network and multitask learning according to claim 1 or 5, characterized in that in step D, cross Encopy classification learning errors are used for lung lobe segmentation tasks, Focal learning errors are used for lung lobe boundary segmentation tasks, and two variable weight parameters are used for adjusting weights occupied by the two tasks in a training process to synthesize the whole learning errors of the network.

9. The method for segmenting lung lobes based on 3D full convolution neural network and multitask learning according to claim 8, wherein a learning error of the synthetic network as a whole specifically includes:

setting P as the lung lobe prediction generated by the network in the lung lobe segmentation task, G as the lung lobe label in the lung lobe segmentation task, C as the number of total classes, wherein the total classes in the lung lobe segmentation task are composed of five lung lobe classes and one background class, namely C is 6, N is the total number of pixels, and setting the ith pixel in the lung lobe label

Belong to class c, then

Is 1, if not c, is 0;

for the probability that the ith pixel in the lung lobe prediction belongs to the class c, the range is [0, 1]]Therefore, the Cross entry classification learning error of the lung lobe segmentation task is:

setting up

Lung lobe boundary prediction results generated for the sigmoid layer in the lung lobe boundary task,

setting the ith pixel in the lung lobe boundary label for the lung lobe boundary label automatically generated by the lung lobe label

The boundary of lung lobe

Is 1, otherwise is 0,

the probability that the ith pixel in the lung lobe boundary prediction is the lung lobe boundary is in the range of 0,1]And gamma is a modulation coefficient for controlling the weight of the easily classified sample, so that the learning error in the lung lobe boundary task is defined as:

setting up

And

in order to adjust the parameters of the weights of the lung lobe segmentation task and the lung lobe boundary segmentation task, the learning error of the whole network is as follows:

。

10. the method for segmenting lung lobes based on 3D full convolution neural network and multitask learning as claimed in claim 9, wherein in step D, when training the constructed 3D full convolution neural network, the Adam optimization algorithm is adopted as the training optimization algorithm, the initial learning rate is set to 0.001, and the weight attenuation parameter is set to

If the error of a single case is not reduced after the training of 20 cases of data continuously, the learning rate is multiplied by the attenuation coefficient of 0.8;

and updating parameters once for each batch by network learning, judging the total error of the lung boundary detection result by the model after each iterative learning, if the current error is smaller than the error of the last iteration, saving the current model, continuing training, and if the training reaches the maximum iteration number or the total error does not decrease after 10 iterations, stopping the training.

Images