CN109584252B

CN109584252B - Lung lobe segment segmentation method and device of CT image based on deep learning

Info

Publication number: CN109584252B
Application number: CN201811505228.0A
Authority: CN
Inventors: 郑永升; 戎术
Original assignee: Hangzhou Yitu Medical Technology Co ltd
Current assignee: Hangzhou Yitu Medical Technology Co ltd
Priority date: 2017-11-03
Filing date: 2017-11-03
Publication date: 2020-08-14
Anticipated expiration: 2037-11-03
Also published as: CN109615636B; CN109636811B; CN111709953B; CN111709953A; CN109636811A; CN107909581A; CN109584252A; CN107909581B; CN109615636A

Abstract

The invention relates to a lung lobe segment segmentation method, a device, a system, a storage medium and equipment of a CT image, wherein the method comprises the following steps: detecting, in the CT image, a lung contour including an intra-lung region and an extra-lung region is detected and output; a screening step, in the lung contour, selecting a machine segmentation mode to screen out an area in the lung, and taking the area as a candidate area; a segmentation step, in a 3D layer of the candidate region, simultaneously performing blood vessel segmentation and lung fissure segmentation on the lung segment and the lung lobe; a construction step, namely constructing a blood vessel tree according to a blood vessel segmentation result to obtain three-dimensional blood vessel distribution of the lung; and an integration step, namely combining the blood vessel tree and the lung fissure segmentation result, and performing lung lobe segmentation to obtain a final segmentation result of the candidate region. The method, the device, the system, the storage medium and the equipment for segmenting the lung lobe sections of the CT image based on the deep learning effectively reduce errors and improve the diagnosis rate and the accuracy rate, and are not limited by individual lung morphological differences.

Description

Lung lobe segment segmentation method and device of CT image based on deep learning

The application is a divisional application of Chinese patent application with the application number of 201711070729.6, the application date of 2017, 11 and 03, and the name of CT image lung lobe segmentation method, device, system, storage medium and equipment.

Technical Field

The present invention relates to the field of image segmentation, and in particular, to a method, an apparatus, a system, a storage medium, and a device for segmenting lung lobe segments of CT images.

Background

Currently, lung cancer is the cancer with the highest mortality rate among all cancers. The lung nodules are image representation forms of lung cancer, are represented by shadows with increased density in CT imaging, and have important significance for early screening and evaluation of the lung cancer through detection and segmentation of the lung nodules in lung lobes and lung segments in the CT images. In the existing detection, 3D CT (Emmenlauer, m., etc.: free, fast and reliable fixing of large 3D databases, J Microscopy,233, No.1, pp.42-60,2009.) has a large image data volume and large individual difference, which brings certain difficulty to the segmentation technology of the lung lobes and the lung segments.

Unlike generalized computer vision technology applications, in order to reduce the complexity of Deep Convolutional Neural Network (DCNN) computation and efficiently utilize limited training data, the prior art employs segmentation combining medical background information with deep learning, for example, 1) the best current technique for segmenting lung lobes and lung segments is segmentation performed in a layer-by-layer progressive manner (k.hayashi, et., radio and CT applications of the major responses, "Radiographics, vol, 21, No.4, pp.861-874,2001), lung contours are detected and positioned first, and then lung lobes are segmented by using a deep learning method (s.hutch, et., Automatic segmentation for acquisition and quantification of volumetric X-ray images," IEEE transactions, med.20, pp.490, jc. j. n. map, et. CT, CT coordinates of CT, et. 2001, CT, et. r. n. CT, for segmentation of lung lobes, CT coordinates, et. 12, CT, et. emission, et. 498, et. CT, et. r. CT, et. c. r. CT, et. r. CT, et. 12, et. c. r. g. 1, CT, et. r. 1, CT, et. r. g. 1, "IEEE trans. med.image., vol.27, No.1, pp.1-10, jan.2008"). This approach ignores the difference in lung structure between different patients, with uncertainty in the detection results (i.c. slurry, etc.: for "means automatic segmentation of the clinical lung in ct," IEEE trans. med. image, vol.24, No.8, pp.1025-1038, aug.2005.). 2) A water-wash deformation is established by analyzing the trachea and the vessels, which has a better effect on visible fissures segmentation but a poorer effect on partially invisible segmentations (E.van Rikxort, etc.: Automatic segmentation of pulmony patches and obtained in not complete details, "IEEE Transactions on medical Imaging, vol.29, No.6, pp.1286-1296,2010"). The methods are tested on a large amount of data, and the segmentation effect is poor.

Chinese patent document CN103035009A discloses a reconstruction and segmentation method based on CT image lung nodule edges, which first performs spatial transformation on an image by using a transformation mode with sparse representation capability on gradient features; and then establishing a CT image segmentation algorithm evaluation system, mainly amplifying edge information and reconstructing the edges of the lung nodules.

Although the above patent improves the accuracy of the detection and segmentation of the pulmonary nodule edge in the CT image, before the detection of the edge of the image is started, the detection needs to be compared with an initially set threshold value, and the detection can be started only when the condition is met, otherwise, the image needs to be reprocessed.

As a latest technology LDCT (low-dose helical CT) for Screening Lung cancer, the LDCT radiation dose is lower, is reduced by 75-90% compared with the conventional CT dose, does not affect Lung imaging, has the sensitivity of CT, and has the advantages that after the diagnosis rate of early Lung cancer can be improved by LDCT and the death rate of Lung cancer is effectively reduced by data of National Lung Screening Trial (NLST) in 2011, all clinical researches on Screening of Lung cancer give out exciting results successively, and the application of LDCT in Screening of Lung cancer is effectively popularized. Therefore, how to fully activate and utilize the nearly silence stored large data of the breast LDCT is one of the problems that those skilled in the art need to solve.

In summary, how to improve the accuracy and precision of detecting and segmenting lung nodules in CT images and how to fully utilize and activate the big data of CT images is one of the problems to be solved by those skilled in the art.

Disclosure of Invention

In order to solve the above problems, the present invention provides a method, an apparatus, a system, a storage medium and a device for segmenting a lung lobe segment of a CT image based on deep learning.

In order to achieve the above object, a first aspect of the present invention provides a method for segmenting lung segments of a CT image, the method comprising the steps of:

detecting, namely detecting and outputting a lung contour in the CT image, wherein the lung contour comprises an intra-lung region and an extra-lung region;

a screening step, in the lung contour, selecting a machine segmentation mode to screen out an area in the lung, and taking the area in the lung as a candidate area;

a segmentation step, in a 3D layer of the candidate region, simultaneously performing blood vessel segmentation and lung fissure segmentation on the lung segment and the lung lobe;

a construction step, namely constructing a blood vessel tree according to a blood vessel segmentation result to obtain three-dimensional blood vessel distribution of the lung;

and an integration step, combining the vessel tree and the lung fissure segmentation result by adopting a lung lobe segmentation algorithm, and performing lung lobe segmentation to obtain a final segmentation result of the candidate region.

Furthermore, in the detection step, the FCN network which takes the multilayer convolution layer as a main structure is adopted for detection and output.

Specifically, in the segmentation step, the lung segments and the lung lobes are respectively subjected to blood vessel segmentation and lung fissure segmentation by using 3D U-net along two branch lines.

Further, in the constructing step, the trachea distribution is deduced according to the three-dimensional blood vessel distribution, so that the bronchial tree is obtained.

Specifically, in the integration step, the lung lobe segmentation algorithm integrates the vessel tree and the lung fissure segmentation result by inputting the lung lobe classification model, and outputs the final segmentation result.

The invention provides a lung lobe segment segmentation device based on CT image in second aspect, the device includes:

the detection module is used for detecting and outputting a lung contour in the CT image, wherein the lung contour comprises an intra-lung region and an extra-lung region;

the screening module is used for screening out the area in the lung in a machine segmentation mode in the lung contour, and taking the area in the lung as a candidate area;

the segmentation module is used for simultaneously performing blood vessel segmentation and lung fissure segmentation on the lung segment and the lung lobe in a 3D layer of the candidate region;

the construction module is used for constructing a blood vessel tree according to the blood vessel segmentation result to obtain the three-dimensional blood vessel distribution of the lung;

and the integration module is used for combining the blood vessel tree and the lung fissure segmentation result by adopting a lung lobe segmentation algorithm, and performing lung lobe segmentation to obtain a final segmentation result of the candidate region.

Furthermore, the detection module adopts the FCN network which takes the multilayer convolution layer as a main structure to carry out detection and output.

Specifically, in the segmentation module, the lung segments and the lung lobes are respectively subjected to blood vessel segmentation and lung fissure segmentation by using 3D U-net along two branches.

Further, in the construction module, the trachea distribution is deduced according to the three-dimensional blood vessel distribution, and the bronchial tree is obtained.

Specifically, in the integration module, the lung lobe segmentation algorithm integrates the vessel tree and the lung fissure segmentation result by inputting the lung lobe classification model, and outputs the final segmentation result.

The invention provides a system for realizing the segmentation of the lung lobe segment based on the CT image in a third aspect, which comprises the lung lobe segment segmentation device based on the CT image in any one of the second aspect.

A fourth aspect of the present invention provides a non-volatile storage medium having instructions stored therein, which when executed, cause a processor to perform a CT image-based lung segment segmentation method, the instructions comprising:

detecting and outputting a lung contour in the CT image, wherein the lung contour comprises an intrapulmonary area and an extrapulmonary area;

screening instructions, selecting a machine segmentation mode to screen out an area in the lung contour, and taking the area in the lung as a candidate area;

a segmentation instruction, which is used for simultaneously carrying out blood vessel segmentation and lung fissure segmentation on the lung segment and the lung lobe in a 3D layer of the candidate region;

constructing an instruction, namely constructing a blood vessel tree according to a blood vessel segmentation result to obtain the three-dimensional blood vessel distribution of the lung;

and integrating the instruction, combining the blood vessel tree and the lung fissure segmentation result by adopting a lung lobe segmentation algorithm, and segmenting the lung lobes to obtain a final segmentation result of the candidate region.

A fifth aspect of the invention provides an apparatus comprising a memory storing computer executable instructions and a processor configured to execute the instructions to perform a process for CT image based segmentation of lung lobes, the process comprising:

The invention provides a CT image lung lobe segment segmentation method, a device, a system, a storage medium and equipment, which adopt a plurality of layers of convolution layers as an FCN network of a main body structure to accurately detect and output a lung contour in a CT image; secondly, screening candidate regions, selecting 3D U-net for expansion, carrying out blood vessel segmentation and lung fissure segmentation on lung segments and lung lobes, constructing a blood vessel tree according to a blood vessel segmentation result, and deducing the distribution condition of the trachea according to three-dimensional blood vessel distribution; the method can efficiently control the precision and speed of the segmentation process, the accuracy of the finally obtained segmentation result is high, errors which easily exist in the traditional segmentation mode are greatly reduced, the diagnosis rate is improved, and the method is not limited by individual lung morphological differences.

Drawings

Fig. 1 is a flowchart of a lung lobe segmentation method based on CT images according to the present invention;

FIG. 2 is a block diagram of a lung lobe segmentation apparatus based on CT image according to the present invention;

fig. 3 is a schematic view of a lung lobe segmentation process based on CT images according to the present invention.

Detailed Description

The following description of the embodiments of the present invention is provided for illustrative purposes, and other advantages and effects of the present invention will become apparent to those skilled in the art from the present disclosure. While the invention will be described in conjunction with the preferred embodiments, it is not intended that features of the invention be limited to these embodiments. Rather, the invention has been described in connection with embodiments thereof for the purpose of covering all alternatives and modifications which may fall within the scope of the invention as defined by the appended claims. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. The invention may be practiced without these particulars. Moreover, some of the specific details have been left out of the description in order to avoid obscuring or obscuring the focus of the present invention.

Deep learning is a new area in machine learning research, with the motivation to build, simulate, human brain neural networks for analytical learning, which mimic the mechanisms of the human brain to interpret data such as images, sounds, and text ("what can deep learning be done. The invention provides a CT image lung lobe segmentation method, a device, a system, a storage medium and equipment based on deep learning.

The embodiment of the invention provides a CT image lung lobe segment segmentation method, as shown in fig. 1, the method comprises a detection step 1, a screening step 2, a segmentation step 3, a construction step 4 and an integration step 5. Specifically, in the detection step 1, CT image data is input, and a lung contour in the CT image is detected and output, where the output lung contour includes an intra-lung region and an extra-lung region; in the screening step 2, the lung contour is input, the lung region is screened out by selecting a machine segmentation mode, and the lung region is used as a candidate region; in the segmentation step 3, performing blood vessel segmentation and lung fissure segmentation on lung segments and lung lobes of the candidate region on a 3D level to obtain blood vessel segmentation and lung fissure segmentation results, wherein the specific format is a three-dimensional lattice which corresponds to a three-dimensional position in the CT image, and each element value in the lattice represents whether the lattice is judged as a blood vessel/lung fissure by a blood vessel/lung fissure algorithm; in the construction step 4, according to the blood vessel segmentation result, a blood vessel tree is constructed to obtain the three-dimensional blood vessel distribution of the lung; in the integration step 5, the vessel tree and the lung fissure segmentation result are combined by adopting a lung lobe segmentation algorithm, and the lung lobe segment is segmented to obtain a final segmentation result of the candidate region. The invention adopts layer-by-layer segmentation, efficiently controls the accuracy and speed of segmentation, improves the segmentation efficiency and is not limited by the individual lung morphological difference.

In recent years, image segmentation techniques based on deep learning (harihanan, b., label _ aez, p., Girshick, r., Malik, j.: Hypercolumns for object segmentation and _ ne-gradeable localization. in: proc.cvpr.pp.447-456(2015)) have been used for mass-play in various international memories, and in general Deep Convolutional Neural Networks (DCNN), several fully-connected layers are usually added after the convolutional network so that the features of the convolutional network can be mapped into vectors of fixed length (Io _ e, s., szegy, c.: balance transformation: accurate computing word tracking by reducing the global convolutional coefficients/shifts/1502.03167 (1502.03167)), and finally utilized. Whereas full-volume networks such as FCN (Long, J., Shell, E., Darrell, T.: full connected network for a discrete segment. in: Proc.CVPR.pp.3431-3440 (g)) or U-net (Ronneberger, O., Fischer, P., Brox, T.: U-net: connected network for a binary image segment. in: MICCAI.LNCS, vol.9351, pp.234-241.Springer (2015)), can achieve a pixel-level classification of the images (Milletari, F., Ahmax, di, S., Hough-cnn. collection for a segment of the segment. F., Ahmax, S., Hough-c.: left-green for a segment of the segment. for a final classification of the images (copy. R, P. sub., P. for a segment. sub.: a final classification of the images (B. R, P. sub.: a final classification of the size of the images (B. R, P. sub.: a final classification of the sample of the volume, P. RTM. A. sub., P, m. Sajjadi, M. and Tasdizen, T. Image segmentation with classified digital models and local discrete normal networks in Proc. ICCV. pp.2168-2175(2013)), and this classification can not only classify the Image to be detected into two classes of foreground and background, but also automatically give the classification of the object. U-net is also one of the more common image segmentation networks, and lung segment segmentation has great advantages in medical images, especially lung lobes: first, U-net can be converted into 3-D (Fedorov, A., Beichel, R., et al: 3D slicer as an image computing platform for the quantitative Imaging network J. Magn Reson Imaging 30(9), 1323-. By using the image segmentation algorithm in the field of deep learning, the error can be greatly reduced, and the diagnosis efficiency and accuracy are improved.

Specifically, those skilled in the art will appreciate that the conventional segmentation method is a classic Lung segmentation process, i.e., grayscale threshold segmentation, connectivity analysis, left and right Lung separation, morphological occlusion of high density structures in the Lung, initiated by Hushiying et al (Hushiying, Hoffman E A, Reinhardt J M. automatic segmentation for access to quantitative X-ray CT images [ J ]. IEEE transactions on Medical Imaging,2001,20(6): 490-498.). The flow adopts an iterative threshold optimization method, then the maximum two connected components are searched, and the two connected components are separated by a method of continuous morphological erosion during lung adhesion, and the method is called as a classical lung segmentation method. In the detection step 1, a full Convolutional neural network (FCN) different from a classical segmentation method is adopted, and compared with a traditional method of segmenting an image by using a Convolutional Neural Network (CNN), the FCN has obvious advantages: (1) any size of input image can be accepted without requiring all training images and test images to be the same size; (2) more efficient because the problems of repeated storage and computation of the convolution due to the use of pixel blocks are avoided. The invention adopts a mature FCN network for segmentation, wherein the multi-convolution layer is used as a main body structure of the FCN network, and replaces the traditional neural network full link layer with the convolution layer, thereby ensuring that the whole network is a full convolution network structure.

In order to ensure that the resolution of an output result is consistent with that of an input image, the output characteristic diagram of the network is expanded to be consistent with that of the input image through the last several layers of the scrolling layers. In addition, the invention performs retraining or fine-tuning on the lung CT image to improve the segmentation accuracy through the following aspects: (1) taking a section of the lung CT image in the XY direction (shown in figure 3) as a training image, and marking out an area in the lung as a training label; (2) training data volume: randomly sampling from 10K-magnitude CT to generate a 100K-magnitude CT sectional image; (3) and (3) marking accuracy: the training annotation error is required to be guaranteed to be within 3 mm; (4) the data volume of a specific CT region (lung apex, septum, extrapulmonary) needs to be guaranteed on a targeted basis; (5) the distribution of specific CT types (different ages, symptoms, CT doses and the like of patients) needs to be ensured so as to output the lung contour efficiently, and the process of the detection step 1 is very fast due to the high similarity of the distribution structure of the human body and the relatively clear lung contour in the CT image.

And then in a screening step 2, selecting a machine segmentation mode to screen out the lung region, wherein the machine segmentation is automatic lung segmentation based on machine learning, and the lung region is used as a candidate region for key screening of subsequent further segmentation, so that the noise is effectively reduced, and the detection speed is accelerated. The machine learning is a multi-field cross subject and relates to a plurality of subjects such as probability theory, statistics, approximation theory, convex analysis and algorithm complexity theory. The special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve the performance of the computer.

Further, in the segmentation step 3, the lung segment and the lung lobe are respectively subjected to blood vessel segmentation and lung fissure segmentation by using 3D U-net along two branch lines. Two of the branches are obtained by screening the candidate region (the lung interior region), i.e. the edges of the candidate region. Those skilled in the art will appreciate that the encoder gradually reduces the spatial dimensions of the pooling layer and the decoder gradually restores the details and spatial dimensions of the object. There is usually a fast connection between the encoder and the decoder, thus helping the decoder to better repair the details of the target. U-Net is the most commonly used structure in this approach. 3D U-net is a kind of full-convolution deep neural network, and for an implementation of a conventional FCN network, a top-down network and a bottom-up network are connected by Skip connection (Skip connection) to ensure that a final output layer can simultaneously obtain high-order semantic and low-order local texture information. The blood vessel segmentation and the lung fissure segmentation are both segmented by 3D U-net, the two segments adopt the same algorithm framework and network structure, and the difference is that the input labeling semantics in the training process are different, the former needs to label blood vessels, and the latter needs to label lung fissure. Segmentation of the lung segments and lobes by 3D U-net resulted in vascularity and 5 independent spaces (i.e. lobes) and septa in the lung segments.

Specifically, in the constructing step 4, on the basis of the blood vessel segmentation result, a blood vessel tree is constructed to obtain the three-dimensional blood vessel distribution inside the lung segment, and since 3D U-net is a 3D segmentation model (as shown in fig. 3), the input is the segmentation result of a 3D space, i.e. the confidence of whether each point in the 3D space is a blood vessel. When constructing a blood vessel tree, firstly, the segmentation result is binarized according to a fixed threshold value, smoothing is carried out, then the maximum connected component of a blood vessel region is worked out through a flood fill algorithm to be used as the constructed blood vessel tree, namely, the final blood vessel segmentation result is produced, binarization processing is carried out based on the confidence coefficient of a 3D space point, and the position where the blood vessel is specifically determined. According to the anatomical structure in the lung, the trend of pulmonary blood vessels and tracheas in the lung has strong consistency, and blood vessels are often accompanied near the bronchi of all levels of the lung. However, in the lung CT imaging, the pulmonary vessels are more visually significant, so the algorithm is to extract the characteristic information such as the distribution and shape of the pulmonary bronchial tree by detecting the vessel tree and obtaining the pulmonary bronchial distribution approximately according to the position of the vessels, so as to be used for information integration in the subsequent steps. The flodfil algorithm is a classical algorithm for extracting a plurality of connected points from one area and distinguishing the connected points from other adjacent areas (or respectively dyeing the connected points into different colors), and is named because the idea of the flodfil algorithm is similar to that of flood which is spread from one area to all reachable areas.

Further, in the integration step 5, the lung lobe segmentation algorithm is a neural network based classification model. The algorithm inputs vector (position + distance) information characterized by the relative position of a specific point in the lung (as shown in fig. 3, the relative proportion of the region in XYZ direction is calculated based on the lung segmentation result), the blood vessel and lung fissure result (represented by a 64 × 64 × 64 cube) in the vicinity of the region, and the distance to the nearest lung fissure and blood vessel. The algorithm trains a lung lobe segment classification model by learning the input, so that a blood vessel tree and a lung fissure segmentation result are combined. The algorithm outputs the classification result of the lung lobes and the lung segments at the position. The invention can accurately control the accuracy and speed of the segmentation process, for example, the training data volume and the marking accuracy need to be ensured, the training data volume needs to ensure 100K-magnitude lung interior points (lung lobes and lung segments to which the marking belongs), 10K-magnitude CT data and CT type distribution (different age segments of patients, symptoms, CT dose and the like) are ensured; the marking precision is required to be ensured to be more than 95%. Acceleration can be achieved by using a suitably compact network structure (reducing convolutional layer depth and width), operating with shared neighborhood features to reduce the amount of computation.

For the CT type distribution, before the lung lobe segmentation of the CT image, the patient should be classified according to the age, disease condition, and CT dose (such as conventional CT and LDCT); and secondly, the same CT type is segmented, so that errors caused by different CT types can be effectively reduced, data statistics can be performed in a targeted manner, and a valuable reference basis is provided for diagnosing related diseases (such as metastatic tumors, mesothelioma, lung nodules and the like).

As shown in fig. 3, the preferred embodiment of the present invention firstly inputs CT image data, and detects and outputs the lung contour in the CT image by using the FCN network, where the output lung contour includes the lung interior region and the lung exterior region; secondly, screening out an area in the lung as a candidate area by adopting a machine segmentation mode; thirdly, respectively performing vessel segmentation and lung fissure segmentation on the lung segment and the lung lobe of the candidate region by adopting 3D U-net on a 3D layer to respectively obtain 5 independent spaces (namely lung lobes), diaphragms and vessel distribution in the lung segment, and constructing a vessel tree according to the vessel distribution result in the lung segment; and finally, combining the blood vessel segmentation result with the lung fissure segmentation result and carrying out lung lobe segmentation by using a lung lobe segmentation algorithm, wherein the lung lobe segmentation algorithm is a classification model based on a neural network, and the lung lobe classification model is trained by learning the input by the algorithm, so that the blood vessel tree and the lung fissure segmentation result are combined. The algorithm outputs the classification result of the lung lobes and the lung segments at the position, and the result with a plurality of segmentation regions (such as

regions

1, 3, 4 and the like in fig. 3) is obtained accurately and quickly.

Therefore, the lung lobe segment segmentation method for the CT image provided by the invention adopts a layer-by-layer segmentation mode, and introduces FCN and 3D U-net for segmentation respectively, so that the finally obtained segmentation result has high accuracy, errors which easily exist in the traditional segmentation mode are greatly reduced, and the diagnosis rate is improved.

The embodiment of the invention provides a lung lobe segmentation device based on a deep learning CT image, and as shown in FIG. 2, the device comprises a detection module 10, a screening module 20, a segmentation module 30, a construction module 40 and an integration module 50.

A detecting module 10, configured to detect and output a lung contour in the CT image, where the lung contour includes an intra-lung region and an extra-lung region;

a screening module 20, configured to select an intra-lung region in the lung contour by using a machine segmentation method, and take the intra-lung region as a candidate region;

a segmentation module 30, configured to perform vessel segmentation and lung segmentation on lung lobes and lung segments simultaneously in a 3D level of the candidate region;

a construction module 40, configured to construct a blood vessel tree according to the blood vessel segmentation result, so as to obtain three-dimensional blood vessel distribution of the lung;

and the integration module 50 is configured to combine the vessel tree and the lung fissure segmentation result by using a lung lobe segmentation algorithm, and perform lung lobe segmentation to obtain a final segmentation result of the candidate region.

Further, in the detection module 10, a mature FCN network is used for segmentation, wherein the multi-convolution layer is used as a main structure of the FCN network, and the conventional neural network full link layer is replaced with the convolution layer, so that the whole network is guaranteed to be a full convolution network structure, and the lung contour in the CT image is efficiently and quickly output.

In the screening step 2, the lung region is screened out by selecting a machine segmentation mode, and the lung region is used as a candidate region for the key screening of subsequent further segmentation.

Further, in the segmentation module 30, the vessel segmentation and the fissure segmentation are performed on the lung segment and the lung lobe respectively along two branch lines by using 3D U-net. That is, in the 3D layer of the candidate region, the lung segment and the lung lobes are segmented by 3D U-net, resulting in the vascularity and 5 independent spaces (i.e., lung lobes) and septa in the lung segment.

Further, in the construction module 40, based on the result of the vessel segmentation, a vessel tree is constructed to obtain the three-dimensional vessel distribution inside the lung segment. According to the anatomical structure in the lung, the trend of pulmonary blood vessels and tracheas in the lung has strong consistency, and blood vessels are often accompanied near the bronchi of all levels of the lung. However, in pulmonary CT imaging, the pulmonary vessels are more visually prominent, so the algorithm is to detect the vessel tree and approximate the pulmonary bronchial distribution according to the position of the vessels.

Further, in the integration module 50, the lung lobe segmentation algorithm is a neural network-based classification model. The algorithm inputs vector (position + distance) information characterized by the relative position of a specific point in the lung (as shown in fig. 3, the relative proportion of the region in XYZ direction is calculated based on the lung segmentation result), the blood vessel and lung fissure result (represented by a 64 × 64 × 64 cube) in the vicinity of the region, and the distance to the nearest lung fissure and blood vessel. The algorithm trains a lung lobe segment classification model by learning the input, so that a blood vessel tree and a lung fissure segmentation result are combined. The algorithm outputs the classification result of the lung lobes and the lung segments at the position.

Therefore, the lung lobe segment segmentation device based on the CT image adopts a layer-by-layer segmentation mode, and the FCN and the 3D U-net are introduced for segmentation respectively, so that the finally obtained segmentation result is high in accuracy, errors which easily exist in a traditional segmentation mode are greatly reduced, and the diagnosis rate is improved.

The embodiment of the invention also provides a system for realizing the segmentation of the lung lobe segment based on the CT image, which comprises the lung lobe segment segmentation device based on the CT image in the embodiment.

An embodiment of the present invention further provides a non-volatile storage medium having instructions stored therein, where the instructions, when executed, cause a processor to execute a CT image-based lung segment segmentation method, where the instructions include:

a segmentation instruction, which is used for simultaneously carrying out blood vessel segmentation and lung fissure segmentation on the lung lobes and the lung segments in a 3D layer of the candidate region;

An embodiment of the present invention also provides an apparatus, including a memory storing computer executable instructions and a processor configured to execute the instructions to perform a process of CT image-based lung lobe segmentation, the process including:

a segmentation step, in a 3D layer of the candidate region, simultaneously performing blood vessel segmentation and lung fissure segmentation on lung lobes and lung segments;

In summary, according to the method, the apparatus, the system, the storage medium and the device for segmenting the lung lobe segment of the CT image based on the deep learning provided by the present invention, the FCN network with the multi-layered convolution layer as the main structure is adopted to accurately detect and output the lung contour in the CT image; secondly, screening candidate regions and selecting 3D U-net for expansion to carry out blood vessel segmentation and lung fissure segmentation on lung segments and lung lobes, constructing a blood vessel tree according to a blood vessel segmentation result, and deducing the distribution condition of the trachea by virtue of blood vessel distribution; the method can efficiently control the precision and speed of the segmentation process, the accuracy of the finally obtained segmentation result is high, errors which easily exist in the traditional segmentation mode are greatly reduced, the diagnosis rate is improved, and the method is not limited by individual lung morphological differences.

The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.

Claims

1. A lung lobe segment segmentation method of a CT image based on deep learning is characterized by comprising the following steps:

detecting and outputting a lung contour in the CT image, wherein the lung contour is used for segmenting an area in the lung and an area outside the lung;

a screening step, in the lung contour, selecting the lung region by a machine segmentation mode, and taking the lung region as a candidate region;

a segmentation step, in the 3D layer of the candidate region, simultaneously performing blood vessel segmentation and lung fissure segmentation on the lung segment and the lung lobe;

a construction step, namely constructing a blood vessel tree according to the blood vessel segmentation result to obtain the three-dimensional blood vessel distribution of the lung;

an integration step, combining the vessel tree and the lung fissure segmentation result by adopting a lung lobe segmentation algorithm, and performing lung lobe segmentation to obtain a final segmentation result of the candidate region,

in the segmentation step, the lung segments and lung lobes of the candidate region are subjected to vessel segmentation and lung fissure segmentation on a 3D level to obtain vessel segmentation and lung fissure segmentation results, the specific format is a three-dimensional lattice which corresponds to a three-dimensional position in the CT image, and each element value in the lattice represents whether the cell is judged as a vessel/lung fissure by a vessel/lung fissure algorithm.

2. The method for segmenting lung segments based on deep learning CT images as claimed in claim 1, wherein said detecting step comprises detecting and outputting by using FCN network with multi-layer convolution layer as main structure.

3. The method for segmenting lung segments based on the deep learning CT image as claimed in claim 1, wherein in the segmenting step, the vessel segmentation and the fissure segmentation are performed on the lung segments and the lung lobes respectively along two branch lines by using 3D U-net.

4. The method as claimed in claim 1, wherein the step of constructing estimates the distribution of trachea according to the three-dimensional blood vessel distribution to obtain the bronchial tree.

5. The method as claimed in claim 1, wherein in the integrating step, the lung segmentation algorithm integrates the vessel tree and the fissure segmentation result by inputting a lung segmentation classification model, and outputs a final segmentation result.

6. A lung lobe segmentation device of CT image based on deep learning is characterized in that the device includes:

the detection module is used for detecting and outputting a lung contour in the CT image, wherein the lung contour is used for segmenting an intra-lung region and an extra-lung region;

the screening module is used for screening the lung region in the lung contour by adopting a machine segmentation mode and taking the lung region as a candidate region;

the segmentation module is used for simultaneously performing blood vessel segmentation and lung fissure segmentation on the lung segment and the lung lobe in the 3D layer of the candidate region;

an integration module for combining the vessel tree and the lung fissure segmentation result by adopting a lung lobe segmentation algorithm and performing lung lobe segmentation to obtain a final segmentation result of the candidate region,

in the segmentation module, vessel segmentation and lung fissure segmentation are performed on lung segments and lung lobes of the candidate region on a 3D level to obtain vessel segmentation and lung fissure segmentation results, a specific format is a three-dimensional lattice which corresponds to a three-dimensional position in the CT image, and each element value in the lattice represents whether the cell is judged as a vessel/lung fissure by a vessel/lung fissure algorithm.

7. The apparatus for segmenting lung lobes according to claim 6, wherein the detection module employs an FCN network with a plurality of convolutional layers as a main structure for detection and output.

8. The apparatus for segmenting lung lobe segment based on deep learning CT image as claimed in claim 6, wherein in the segmentation module, the blood vessel segmentation and the lung lobe segmentation are performed respectively along two branch lines by using 3D U-net.

9. The apparatus of claim 6, wherein the bronchial tree is obtained by inferring a distribution of a trachea from the three-dimensional vessel distribution in the constructing module.

10. The apparatus of claim 6, wherein in the integration module, the lung segmentation algorithm integrates the vessel tree and the fissure segmentation result by inputting a lung segmentation classification model, and outputs a final segmentation result.