CN112669284A

CN112669284A - Method for realizing pulmonary nodule detection by generating confrontation network

Info

Publication number: CN112669284A
Application number: CN202011595134.4A
Authority: CN
Inventors: 尚菲菲; 陈锦言; 于永新
Original assignee: Tianjin University
Current assignee: Tianjin University
Priority date: 2020-12-29
Filing date: 2020-12-29
Publication date: 2021-04-16

Abstract

The invention discloses a method for realizing lung nodule detection by using a generative adversarial network, and proposes an end-to-end nodule detection framework, which integrates nodule candidate screening and false positive reduction into a model for joint training. As follows: use the generator as a nodule candidate screening model to do a primary screening; then use the discriminator to make a true and false positive discrimination; the generator is transformed into a single-input single-output model, and the input is a lung CT image, through the generator The output of G1 is the template image of the marked lung nodule, and the segmented lung nodule image is obtained through the generator G2; the output of the discriminator is transformed into the probability that the input lung nodule is a true nodule; the appropriate loss function is set, and the common Train to achieve balance. The detection model is optimized to improve the accuracy of the overall algorithm.

Description

Method for realizing pulmonary nodule detection by generating confrontation network

Technical Field

The invention relates to medical image computer-aided detection, in particular to a lung CT (computed tomography) pulmonary nodule detection method based on generation of an antagonistic network.

Background

Detection of a lung nodule is the identification of the exact location of the lung nodule from a set of Computed Tomography (CT) slices. The lung nodule detection method is mainly divided into a detection method based on traditional manual operation and a detection method based on deep learning. The traditional manual detection frame is mainly divided into four steps: preprocessing, lung segmentation, nodule detection, and classification.

(1) The purpose of the pre-processing is to reduce noise and enhance nodular structures by applying different filters to the input lung CT images. Of these, median filters, point-enhancement filters, logarithmic filters and filters based on histogram equalization are widely used at this stage.

(2) Lung segmentation mainly reduces the search space by extracting only lung regions, and a threshold-based segmentation method is mainly used.

(3) Nodule detection is a major module aimed at detecting potential lung nodules using image processing techniques, including two steps of nodule candidate and false positive reduction. The framework for nodule candidate detection is based on threshold, region, cluster, mixture or mixture segmentation algorithms, partial differential equations, models, maps, template matching, filtering, features, concepts, and other methods. In the false positive reduction phase, different intensity-based, morphology-based, shape-based, and texture-based features are extracted from the detected nodule candidate regions as feature vectors and fed to a classifier for true nodule detection.

(4) In the classification phase, the identification of lung nodules is already completed with the help of different classifiers, and the output of the final result is realized.

In recent years, deep learning methods have become valuable tools in the field of medical imaging due to their higher detection and classification accuracy, and most of the work has chosen to use CNN as a detection architecture for lung nodules. The detection architecture of CNN-based pulmonary nodules can be mainly classified into four categories: (1) the basic convolution operation type used to implement the network, such as 2-D or 3-D or a combination of both; (2) optimizing CNN network parameters based on an evolutionary algorithm; (3) based on the fused CNN algorithm, the handicraft-based features and the CNN-based extracted features can be combined to form a final feature vector; (4) a pre-trained CNN network is used. CNN is trained on a large and diverse data set, fine-tuned for final weights, and then used for object detection or classification, which can achieve higher classification accuracy.

The traditional manual detection method not only needs to design a model manually according to experience and select proper parameter standards, but also consumes a large amount of time, has poor expansibility and poor convergence on a concave boundary. The method based on deep learning solves the problems that the traditional manual design is complex, appropriate parameters are generated in a self-adaptive mode, and the accuracy of the model is guaranteed, but the CNN-based method needs a large amount of marked training data but does not have a large amount of marked data sets. All some consider generating an antagonistic network to simulate the generation of lung nodules, which are involved in training to improve the classification accuracy of CNN networks.

Lung nodules are lung tissue abnormalities found in lung cancer patients, and early detection of lung nodules has been observed in various research works to improve 5-year survival in lung cancer patients. Pulmonary nodule detection is therefore an important stage in the early diagnosis and proper treatment planning of lung cancer, however, pulmonary nodules are relatively small, difficult to find by some physicians, and large in number of patients. With the rapid development of computers, computer-aided diagnosis (CAD) systems can detect the position and size of lung nodules more accurately in a shorter time.

Disclosure of Invention

The present invention aims to overcome the defects of the prior art and provide a method for realizing pulmonary nodule detection by generating an antagonistic network.

The technical scheme of the invention is a method for realizing pulmonary nodule detection by utilizing a generated countermeasure network, and provides an end-to-end nodule detection framework, so that nodule candidate screening and false positive reduction are integrated into a model and are trained together, and the method specifically comprises the following steps:

1) taking the generator as a nodule candidate screening model to perform primary screening; then, a discriminator is used for discriminating true and false positives;

2) the generator is transformed into a single-input single-output model, the input is a lung CT image, the lung CT image is output as a template image for marking lung nodules through the generator G1, and then a segmented lung nodule image is obtained through the generator G2; transforming the output of the discriminator into the probability of inputting the pulmonary nodule as a true nodule; and setting a proper loss function, and achieving balance through co-training.

The generator G1 of the present invention prioritizes the use of the U-Net network.

The discriminator of the invention adopts VGG19 as a backbone network, and most of the largest pooling is deleted from the convolutional layers, and only the first convolutional layer is left.

The invention reduces to two fully connected layers Fc6 and Fc7, the input is a super-resolution image, and the output of Fc7 is the probability that the input is a true nodule.

According to the design of the loss function, the loss function mainly comprises three kinds of losses, namely content loss, countermeasure loss and classification loss;

the content loss is for the generator G₁The generated template is consistent with the real template and is defined as:

wherein I is the image of the input generator, I_MA template marked for the CT image;

the countermeasure loss is to discriminate an image generated by the generator as false and an actual image as true, and is defined as:

wherein I is the image of the input generator, I_TSegmenting the marked CT image to obtain a real lung nodule image; the classification loss is defined to make it easier to classify the images reconstructed by the generator network:

when the lung nodule image is true positive, y is 1; when the lung nodule image is false positive, y is 0.

The method comprises the following specific steps:

a. pretreatment: selecting a LUNA16 data set, wherein 888 CT data are contained in the data set, 36378 lung nodules are marked out in total, and 1186 nodules marked by three experts are selected as a final region to be detected;

at this stage, the CT image in the data set is first converted from dicom format to png format, the image is 512 × 512 pixels; then, a median filter is adopted to enhance the input CT image;

b. and (3) lung region segmentation: extracting a lung region from the preprocessed image by adopting a threshold segmentation method;

c. training a pulmonary nodule detection model;

d. automatic detection of pulmonary nodules: after the detection model is obtained, the CT file of the patient is input into a generator for generating an confrontation network after two steps of preprocessing and lung region segmentation, the generator directly outputs a lung nodule mark image, the mark image is fused with the original CT image to obtain a CT image finally provided with a lung nodule mark, the mark information comprises position and size information of a lung nodule, and finally, the automatic detection of the lung nodule on the CT file of the patient by the detection model is realized.

Advantageous effects

The method for realizing the lung nodule detection by utilizing the generation countermeasure network is an end-to-end lung nodule detection framework, not only reduces the expenditure of resources, but also further optimizes a detection model by the mutual countermeasure of a generator and a discriminator, and improves the accuracy of the whole algorithm.

Drawings

FIG. 1 is a proposed framework for generation of an antagonistic network for pulmonary nodule detection, showing the process of generating an antagonistic network training and testing.

Figure 2 generator network architecture.

FIG. 3 arbiter network architecture.

Detailed Description

The invention is further described below with reference to the accompanying drawings.

The method comprises the following steps of a, preprocessing: the LUNA16 dataset was selected, which contained 888 CT's, and a total of 36378 lung nodules were labeled, and 1186 nodules labeled by three experts were selected as the last region to be examined. At this stage, we first convert the CT image in the data set from dicom format to png format, the image being 512 × 512 pixels, and then use a median filter to enhance the input CT image and remove unwanted noise pixels from the CT image. b. And (3) lung region segmentation: and extracting a lung region from the preprocessed image by adopting a threshold segmentation method. c. Training a pulmonary nodule detection model: since we adopt an end-to-end pulmonary nodule detection framework, lung nodule detection and reduction of false positive rate are completed in one step. At this stage, a generation model and a discrimination model in the generated countermeasure network are trained simultaneously, the generation model and the discrimination model are enabled to spread a mutual game, and finally a balance point is reached, so that the labeled image generated by the generator can directly eliminate the lung nodules with false positives, and the false positive probability output by the discriminator can further reduce the false positive rate of the lung nodules. d. Automatic detection of pulmonary nodules: after the detection model is obtained, the CT file of the patient is input into a generator for generating the countermeasure network after two steps of preprocessing and lung region segmentation, the generator directly outputs a lung nodule mark image, the mark image is fused with the original CT image to obtain a CT image with a lung nodule mark finally, and the mark information comprises the position and size information of the lung nodule. And finally, the automatic detection of the pulmonary nodules of the CT file of the patient by using the detection model is realized.

The pulmonary nodule detection scheme is implemented primarily by generating an antagonistic network. Most of the existing deep learning nodule detection systems are mostly composed of two steps: a) nodule candidate screening and b) false positive reduction, two different models trained separately were used. Although a two-step approach is typically employed, the two-step approach not only adds considerable resource overhead in training two independent deep learning models, but also prevents cross-talk between the two. In this work, we propose an end-to-end nodule detection framework that integrates nodule candidate screening and false positive reduction into one model, co-trained. The generator is used as a nodule candidate screening model to perform primary screening; and then, a discriminator is used for discriminating true and false positives, so that false positives are reduced, and the detection efficiency is improved. To achieve this goal, we have improved the original generation of the countermeasure network. As shown in fig. 1, the generator is modified into a single-input single-output model, the input is a lung CT image, the generator G1 outputs a template image for marking lung nodules, and the generator G1 preferentially considers the use of a U-Net network to ensure a good segmentation effect; obtaining a segmented lung nodule image through a generator G2; the discriminator output is modified to input the probability that the lung nodule is a true nodule. And a proper loss function is set, and through co-training, balance is achieved, and the accuracy of the whole algorithm is improved.

Design of network architecture

Generating a countermeasure network architecture can be divided into a generator and an arbiter

A generator

We adopt the U-net network as the backbone network in the generator due to the excellent performance of the U-net network in the split domain. As shown in fig. 2.

B discriminator

We use VGG19 as the backbone network in the arbiter. Since the lung nodules are very small, to avoid excessive downsampling of the lung nodules, we have removed most of the largest pooling from the convolutional layers, leaving only the first convolutional layer. Furthermore, we reduce to two fully connected layers fc6 and fc 7. The input is a super-resolution image and the output of fc7 is the probability that the input is a true nodule. As shown in fig. 3.

Design of (II) loss function

Our loss function includes mainly three kinds of losses, content loss, countermeasure loss and classification loss, respectively.

The content loss is for the generator G₁The generated template is consistent with the real template. Is defined as:

wherein I is the image of the input generator, I_MA labeled template for a CT image.

The countermeasure loss is to discriminate an image generated by the generator as false and an actual image as true. Is defined as:

wherein I is the image of the input generator, I_TThe marked CT image is a real lung nodule image obtained by segmentation.

The classification penalty is to make it easier to classify the images reconstructed by the generator network. Is defined as:

Claims

1. A method for lung nodule detection using a generative adversarial network, characterized in that an end-to-end nodule detection framework is proposed, which integrates nodule candidate screening and false positive reduction into a model and trains together. ,details as follows:

1) Use the generator as a nodule candidate screening model to do a primary screening; then use the discriminator to make a true and false positive discrimination;

2) The generator is transformed into a single-input single-output model, the input is a lung CT image, and the generator G1 is used to output a template image of a marked lung nodule, and then a segmented lung nodule image is obtained through the generator G2;

Transform the discriminator output into the probability that the input lung nodule is a true nodule;

Set an appropriate loss function and achieve balance through joint training.

2 . The method according to claim 1 , wherein the generator G1 uses a U-Net network.

3. a kind of method that utilizes generative adversarial network to realize lung nodule detection according to claim 1, it is characterized in that, adopt VGG19 as backbone network in the discriminator, delete the majority maximum pooling from the convolutional layer, only The first convolutional layer is left.

4. The method according to claim 1, characterized in that, reducing to two fully connected layers Fc6 and Fc7, the input is a super-resolution image, and the output of Fc7 is the probability that the input is a true nodule.

5. The method according to claim 1, characterized in that, in the design of the loss function, the loss function mainly includes three kinds of losses, namely content loss, confrontation loss and classification. loss;

The content loss is to make the template generated by generator G ₁ consistent with the real template, which is defined as:

Wherein, I is the image input to the generator, and I _M is the labelled template of the CT image;

The adversarial loss is to discriminate the images generated by the generator as fake and real images as real, which is defined as:

Among them, I is the image input to the generator, and I _T is the real lung nodule image obtained by the segmentation of the labeled CT image; the classification loss is to make the image reconstructed by the generator network easier to classify, and is defined as:

Among them, when the lung nodule image is true positive, then y=1; when the lung nodule image is false positive, then y=0.

6. a kind of method utilizing Generative Adversarial Network to realize lung nodule detection according to any one of claim 1 to 5, is characterized in that, concrete steps are as follows:

a. Preprocessing: The LUNA16 data set is selected, which contains 888 CTs, a total of 36,378 lung nodules are marked, and 1,186 nodules marked by three experts are selected as the final area to be detected;

At this stage, the CT images in the dataset are first converted from dicom format to png format, and the images are 512 × 512 pixels; then a median filter is used to enhance the input CT images;

b. Lung area segmentation: using a threshold segmentation method, extract the lung area from the preprocessed image;

c. Training the pulmonary nodule detection model;

d. Automatic detection of pulmonary nodules: After the detection model is obtained, the patient's CT file is input into the generator of the generative adversarial network after two steps of preprocessing and lung area segmentation, and the generator directly outputs the marked image of pulmonary nodules , fuse the marked image with the original CT image to obtain the final CT image with lung nodule marking, the marking information includes the position and size information of the lung nodule, and finally realize the automatic detection of the lung nodule in the patient CT file with the detection model.