CN112184657A

CN112184657A - Pulmonary nodule automatic detection method, device and computer system

Info

Publication number: CN112184657A
Application number: CN202011018485.9A
Authority: CN
Inventors: 黄钢; 聂生东; 陈阳
Original assignee: Shanghai University of Medicine and Health Sciences; University of Shanghai for Science and Technology
Current assignee: Shanghai University of Medicine and Health Sciences; University of Shanghai for Science and Technology
Priority date: 2020-09-24
Filing date: 2020-09-24
Publication date: 2021-01-05
Also published as: WO2022063199A1

Abstract

The invention relates to a method, a device and a computer system for automatically detecting pulmonary nodules, wherein the method comprises the following steps: acquiring a CT image to be detected; carrying out filtering enhancement processing on the CT image to be detected to obtain a lung enhanced CT image sequence; segmenting the CT image sequence by adopting a threshold method to obtain an image only comprising a lung parenchymal region; cutting the image obtained by the lung parenchyma segmentation module into a plurality of image blocks, and obtaining an interested area through a multi-scale feature-fused U-Net network model; and (3) automatically detecting and identifying the region of interest by adopting a 3D CNN model to obtain a pulmonary nodule detection result. Compared with the prior art, the invention has the advantages of high detection sensitivity and precision and the like.

Description

Pulmonary nodule automatic detection method, device and computer system

Technical Field

The invention relates to the field of computer-aided detection, in particular to an automatic pulmonary nodule detection method, device and computer system.

Background

Lung cancer is one of the most deaths of neoplastic diseases. More than 130 million people die each year worldwide as a result of lung cancer. In the early stage of lung cancer, its manifestation is not obvious, and over 70% of patients diagnosed with lung cancer are essentially in the advanced stage of lung cancer. According to the relevant medical statistical data, if lung cancer patients can obtain proper intervention treatment in the early stage of cancer, the 5-year survival rate of the lung cancer patients can reach more than 90 percent, but the survival rate of the lung cancer patients in the 2-3 stage is reduced to 40-5 percent. Therefore, the 'early discovery, early diagnosis and early treatment' is the key for improving the survival rate of the lung cancer patients. Early lung cancer is generally characterized by lung nodules, and detection of lung nodules is the primary step in the early diagnosis of lung cancer. Currently, the most common early detection of lung cancer is imaging examination by CT tomography and then screening by a radiologist, but a large amount of images generated by CT aggravates the radiograph reading burden of the radiologist, which not only wastes time and labor, but also reduces the efficiency and accuracy of diagnosis with a long radiograph reading time. In this context, a computer aided detection system (CAD) assisted radiologist for lung cancer based on CT images has been proposed as a "second opinion" to assist clinical diagnosis.

From the current research situation at home and abroad, a plurality of researchers are dedicated to detecting lung nodules, but the existing method has the situations of low sensitivity and excessive false positive.

Disclosure of Invention

The present invention aims to overcome the defects of the prior art and provide an automatic pulmonary nodule detection method, device and computer system with high detection sensitivity and high detection precision.

The purpose of the invention can be realized by the following technical scheme:

a method for automatic detection of pulmonary nodules, the method comprising the steps of:

acquiring a CT image to be detected;

carrying out filtering enhancement processing on the CT image to be detected to obtain a lung enhanced CT image sequence;

segmenting the CT image sequence by adopting a threshold method to obtain an image only comprising a lung parenchymal region;

cutting the image obtained by the lung parenchyma segmentation module into a plurality of image blocks, and obtaining an interested area through a multi-scale feature-fused U-Net network model;

and (3) automatically detecting and identifying the region of interest by adopting a 3D CNN model to obtain a pulmonary nodule detection result.

Further, the filter enhancement processing includes image filtering and window width and level adjustment.

Further, the segmentation process specifically includes:

roughly dividing the CT image by using a threshold value method, filling holes, and selecting an initial lung parenchymal region by using a three-dimensional connected region marking method;

and repairing the segmented initial lung parenchymal region, adjusting the edge of the lung by using morphological closure operation, and expanding a plurality of pixel points outwards at the edge to obtain a final lung parenchymal region.

Further, the patch is performed on the initial lung parenchymal region using a convex hull algorithm.

Further, random inactivation is introduced into the multi-scale feature fusion U-Net network model, the multi-scale feature fusion U-Net network model comprises a multi-scale feature fusion unit, a plurality of convolution maximum pooling units and a plurality of up-sampling units, and feature graphs of the same scale and size obtained by the convolution maximum pooling units and the up-sampling units are spliced.

Further, the multi-scale feature fusion unit and the convolution maximum pooling unit both comprise a combination of convolution layers and batch normalization layers.

Furthermore, the 3D CNN model comprises a plurality of submodels consisting of a Dense Block unit and a pooling layer, the output ends of the submodels are connected, and the final lung nodule detection result is output after the prediction results of the submodels are fused.

Further, the multi-scale feature fused U-Net network model and the 3D CNN model adopt sample data set during training, each sample comprises a CT image and a corresponding lung nodule marking result, and the lung nodule marking result is the fusion of at least three manual marking results.

The present invention also provides an automatic pulmonary nodule detection apparatus, including:

the CT image acquisition module is used for acquiring a CT image to be detected;

the preprocessing module is used for carrying out filtering enhancement processing on the CT image to be detected to obtain a lung enhanced CT image sequence;

a lung parenchyma segmentation module, configured to segment the CT image sequence by using a threshold method, and obtain an image only including a lung parenchyma region;

the interested region extraction module is used for cutting the image obtained by the lung parenchyma segmentation module into a plurality of image blocks and obtaining an interested region through a multi-scale feature fused U-Net network model;

and the classification module is used for automatically detecting and identifying the region of interest by adopting a 3D CNN model to obtain a lung nodule detection result.

The invention also provides a computer system for automatically detecting pulmonary nodules, comprising:

a processor;

a memory storing processor-executable instructions;

wherein the processor is coupled to the memory for reading program instructions stored by the memory and, in response, performing steps in a method as described above.

Compared with the prior art, the invention has the following beneficial effects:

1. the lung nodule detection method provided by the invention is realized through a U-Net network model and a 3D CNN model which are fused by multi-scale features, firstly, a segmentation model is constructed to position candidate regions of lung nodules, and then, a classification model is constructed to classify and identify the nodules and non-nodules, thereby achieving the detection of the lung nodules. The sensitivity of the invention can reach more than 90% under the condition of keeping 8FPs/Scan on the processed data, and the invention has better detection result.

2. The invention extracts and fuses the multi-scale characteristics of the pulmonary nodules, thereby improving the sensitivity of pulmonary nodule detection, having higher detection sensitivity and being capable of accurately and efficiently detecting the pulmonary nodules.

3. The invention has the advantages of preprocessing the CT image, effectively reducing the noise of the image, improving the quality of the image and further improving the detection precision.

4. The multi-scale feature fusion U-Net network model designed by the invention can extract and fuse multi-scale feature information of lung nodules, can realize the repair of detail information, has high feature accuracy, and can segment the nodules as much as possible.

5. Random inactivation is introduced into the U-Net model established by the invention, and the constructed convolution layers in the network all comprise the combination of the convolution layer and the Batch Normalization (BN) layer, so that overfitting of the model is reduced, and the accuracy of lung nodule segmentation is effectively improved.

6. The 3D CNN model established by the invention outputs the prediction results after each pooling layer, and the final detection result is obtained by fusing the prediction results, thereby further improving the detection precision.

Drawings

Fig. 1 is a flow chart of deep learning pulmonary nodule detection based on multi-scale information;

FIG. 2 is a schematic diagram of a designed multi-scale feature fusion unit;

FIG. 3 is a schematic diagram of a designed U-Net network structure;

fig. 4 is a schematic diagram of a designed 3D CNN network structure;

fig. 5 is a schematic diagram of a density Block structure in a 3D CNN network.

Detailed Description

The invention is described in detail below with reference to the figures and specific embodiments. The present embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation manner and a specific operation process are given, but the scope of the present invention is not limited to the following embodiments.

Example 1

As shown in fig. 1, the present embodiment provides an automatic lung nodule detection method, including:

step 1, CT image data acquisition.

And 2, sample preprocessing, including image filtering and window width and window level adjustment.

In this embodiment, the lung CT image is filtered by using median filtering, mean filtering, and the like, so as to reduce noise of the image and improve quality of the image, and then window width and window position adjustment operations are performed on the lung CT image to enhance contrast of the lung parenchymal region, so as to obtain a lung enhanced CT image sequence. The specific operation of adjusting the window width and the window level may be to set the pixels with HU values greater than 400 to 400 and to set the pixels with HU values less than-1000 to-1000.

And 3, segmenting lung parenchyma.

The lung parenchyma segmentation is mainly used for segmenting the lung parenchyma through a threshold method, and repairing segmentation results to accurately obtain a lung parenchyma region. Specifically, firstly, converting a gray value of a CT image into an HU value, normalizing, roughly dividing lung parenchyma by using a set threshold value T, filling holes, selecting a lung parenchyma region by using a three-dimensional communication region marking method, repairing the divided lung parenchyma region by using a convex hull algorithm, finally adjusting the edge of a lung by using a morphological closing operation, and expanding the obtained lung parenchyma region outwards by a plurality of pixel points to obtain a final lung parenchyma region. In this embodiment, the number of pixel expansion may be set to 10.

And 4, extracting the region of interest.

And constructing a multi-scale feature-fused U-Net network model, wherein the multi-scale feature-fused U-Net network model can extract multi-scale lung nodule information and fuse the extracted multi-scale lung nodule information, so that the accuracy of lung nodule segmentation is improved.

As shown in fig. 3, the multi-scale feature fusion U-Net network model of the present embodiment introduces random inactivation, and includes a multi-scale feature fusion unit, a plurality of convolution maximum pooling units, and a plurality of upsampling units, and feature maps of the same scale size obtained by the convolution maximum pooling units and the upsampling units are spliced. As shown in fig. 2, the multi-scale feature fusion unit includes four channels, the sizes of convolution kernels of the four channels are 1 × 1, 3 × 3, 5 × 5, and 7 × 7, and then the multi-scale feature information extracted by the four channels is spliced and input into the convolution maximum pooling unit. The multi-scale feature fusion unit and the convolution unit both comprise a combination of a convolution layer and a Batch Normalization (BN) layer, so that the accuracy of lung nodule segmentation is improved.

The specific process of extracting the region of interest comprises the following steps:

and 4.1, firstly inputting the input m x n image blocks into a U-Net network model, then extracting and fusing multi-scale characteristic information of lung nodules through a multi-scale characteristic fusion unit, and finally extracting feature maps of the nodules in different scales through four convolution and maximum pooling units.

And 4.2, performing upsampling operation opposite to pooling operation on the extracted nodule feature map, gradually restoring the feature map to the scale of the original image, simultaneously splicing the feature map with the same scale size in the feature extraction process and the upsampled feature map for repairing detailed information, introducing random inactivation into the built U-Net model for reducing overfitting of the model, and outputting the model as a segmentation result of m n, namely a lung nodule segmentation result.

And 5, classifying by a classifier.

And storing the obtained region of interest into an image block with the size of x w, and automatically detecting and identifying the region of interest by adopting a 3D CNN model to obtain a lung nodule detection result.

As shown in fig. 4, the 3D CNN model of this embodiment includes a plurality of submodels composed of a density Block unit and pooling layers, and outputs a prediction result after each pooling layer, and finally fuses the prediction results to obtain a final lung nodule detection result. The 3D CNN model extracts features through a plurality of Dense Block units and a pooling layer, and finally, obtained feature information is fused and classified, so that nodules and non-nodules are identified. The structure of the Dense Block cell is shown in FIG. 5.

The multi-scale feature fused U-Net network model and the 3D CNN model adopt sample data set during training, each sample comprises a CT image and a corresponding lung nodule marking result, and the lung nodule marking result is the fusion of at least three manual marking results.

In this embodiment, the construction of the sample data set used for training specifically includes: removing cases with the thickness larger than 2.5mm from a lung image database alliance LIDC data set, screening 888 sets of CT images for training, providing an XML (extensive Makeup language) labeling file through the 888 sets of CT images, extracting coordinate information of lung nodules from the XML labeling file, fusing labeling results of four radiologists, storing the nodules labeled by at least three radiologists as labeling results, and not using the nodules labeled by less than three radiologists as labeling results, and forming a sample data set by the obtained CT image data and the labeling results together. When training is carried out, model training is realized after the samples in the sample data set are processed in the steps 2 and 3.

According to the method, the detection of the lung nodule is researched through a design experiment, a lung nodule detection model based on deep learning is constructed, firstly, a segmentation model is constructed to position the candidate region of the lung nodule, then, a classification model is constructed to classify and identify the nodule and the non-nodule, and therefore the detection of the lung nodule is achieved. The sensitivity of the method can reach more than 90% under the condition of keeping 8FPs/Scan on the processed data, and the method has a better detection result.

Example 2

The present embodiment provides an automatic pulmonary nodule detection apparatus, including: the CT image acquisition module is used for acquiring a CT image to be detected; the preprocessing module is used for carrying out filtering enhancement processing on the CT image to be detected to obtain a lung enhanced CT image sequence; a lung parenchyma segmentation module, configured to segment the CT image sequence by using a threshold method, and obtain an image only including a lung parenchyma region; the interested region extraction module is used for cutting the image obtained by the lung parenchyma segmentation module into a plurality of image blocks and obtaining an interested region through a multi-scale feature fused U-Net network model; and the classification module is used for automatically detecting and identifying the region of interest by adopting a 3D CNN model to obtain a lung nodule detection result. The rest is the same as example 1.

Example 3

The present embodiments provide a pulmonary nodule automated detection computer system comprising a processor and a memory storing processor-executable instructions; wherein the processor is coupled to the memory for reading program instructions stored in the memory and, in response, performing the steps of the method of embodiment 1.

The foregoing detailed description of the preferred embodiments of the invention has been presented. It should be understood that numerous modifications and variations could be devised by those skilled in the art in light of the present teachings without departing from the inventive concepts. Therefore, the technical solutions available to those skilled in the art through logic analysis, reasoning and limited experiments based on the prior art according to the concept of the present invention should be within the scope of protection defined by the claims.

Claims

1. A method for automatically detecting pulmonary nodules, the method comprising the steps of:

acquiring a CT image to be detected;

2. The method of claim 1, wherein the filter enhancement process comprises image filtering and window width and level adjustment.

3. The method according to claim 1, wherein the segmentation process specifically comprises:

4. The method of claim 3, wherein the patching of the initial lung parenchymal region is performed using a convex hull algorithm.

5. The method according to claim 1, wherein the multi-scale feature fusion U-Net network model introduces random inactivation, and comprises a multi-scale feature fusion unit, a plurality of convolution maximum pooling units and a plurality of upsampling units, and feature maps of the same scale size obtained by the convolution maximum pooling units and the upsampling units are spliced.

6. The method of claim 5, wherein the multi-scale feature fusion unit and the convolution max-pooling unit each comprise a combination of a convolution layer and a batch normalization layer.

7. The method according to claim 1, wherein the 3D CNN model comprises a plurality of submodels composed of a Dense Block unit and a pooling layer, the output ends of the submodels are connected, and the final pulmonary nodule detection result is output after the prediction results of the submodels are fused.

8. The method according to claim 1, wherein the multi-scale feature fused U-Net network model and the 3D CNN model are collected in sample data during training, each sample comprises a CT image and a corresponding lung nodule labeling result, and the lung nodule labeling result is a fusion of at least three manual labeling results.

9. An automatic pulmonary nodule detection apparatus, comprising:

10. A pulmonary nodule automated detection computer system, comprising:

a processor;

a memory storing processor-executable instructions;

wherein the processor is coupled to the memory for reading program instructions stored by the memory and, in response, performing the steps of the method of any one of claims 1-8.