CN112669284A - Method for realizing pulmonary nodule detection by generating confrontation network - Google Patents
Method for realizing pulmonary nodule detection by generating confrontation network Download PDFInfo
- Publication number
- CN112669284A CN112669284A CN202011595134.4A CN202011595134A CN112669284A CN 112669284 A CN112669284 A CN 112669284A CN 202011595134 A CN202011595134 A CN 202011595134A CN 112669284 A CN112669284 A CN 112669284A
- Authority
- CN
- China
- Prior art keywords
- image
- nodule
- lung
- generator
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Apparatus For Radiation Diagnosis (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
Abstract
The invention discloses a method for realizing pulmonary nodule detection by generating an antagonistic network, which provides an end-to-end nodule detection framework, integrates nodule candidate screening and false positive reduction into a model, and performs co-training, and specifically comprises the following steps: taking the generator as a nodule candidate screening model to perform primary screening; then, a discriminator is used for discriminating true and false positives; the generator is transformed into a single-input single-output model, the input is a lung CT image, the lung CT image is output as a template image for marking lung nodules through the generator G1, and then a segmented lung nodule image is obtained through the generator G2; transforming the output of the discriminator into the probability of inputting the pulmonary nodule as a true nodule; and setting a proper loss function, and achieving balance through co-training. The detection model is optimized, and the accuracy of the whole algorithm is improved.
Description
Technical Field
The invention relates to medical image computer-aided detection, in particular to a lung CT (computed tomography) pulmonary nodule detection method based on generation of an antagonistic network.
Background
Detection of a lung nodule is the identification of the exact location of the lung nodule from a set of Computed Tomography (CT) slices. The lung nodule detection method is mainly divided into a detection method based on traditional manual operation and a detection method based on deep learning. The traditional manual detection frame is mainly divided into four steps: preprocessing, lung segmentation, nodule detection, and classification.
(1) The purpose of the pre-processing is to reduce noise and enhance nodular structures by applying different filters to the input lung CT images. Of these, median filters, point-enhancement filters, logarithmic filters and filters based on histogram equalization are widely used at this stage.
(2) Lung segmentation mainly reduces the search space by extracting only lung regions, and a threshold-based segmentation method is mainly used.
(3) Nodule detection is a major module aimed at detecting potential lung nodules using image processing techniques, including two steps of nodule candidate and false positive reduction. The framework for nodule candidate detection is based on threshold, region, cluster, mixture or mixture segmentation algorithms, partial differential equations, models, maps, template matching, filtering, features, concepts, and other methods. In the false positive reduction phase, different intensity-based, morphology-based, shape-based, and texture-based features are extracted from the detected nodule candidate regions as feature vectors and fed to a classifier for true nodule detection.
(4) In the classification phase, the identification of lung nodules is already completed with the help of different classifiers, and the output of the final result is realized.
In recent years, deep learning methods have become valuable tools in the field of medical imaging due to their higher detection and classification accuracy, and most of the work has chosen to use CNN as a detection architecture for lung nodules. The detection architecture of CNN-based pulmonary nodules can be mainly classified into four categories: (1) the basic convolution operation type used to implement the network, such as 2-D or 3-D or a combination of both; (2) optimizing CNN network parameters based on an evolutionary algorithm; (3) based on the fused CNN algorithm, the handicraft-based features and the CNN-based extracted features can be combined to form a final feature vector; (4) a pre-trained CNN network is used. CNN is trained on a large and diverse data set, fine-tuned for final weights, and then used for object detection or classification, which can achieve higher classification accuracy.
The traditional manual detection method not only needs to design a model manually according to experience and select proper parameter standards, but also consumes a large amount of time, has poor expansibility and poor convergence on a concave boundary. The method based on deep learning solves the problems that the traditional manual design is complex, appropriate parameters are generated in a self-adaptive mode, and the accuracy of the model is guaranteed, but the CNN-based method needs a large amount of marked training data but does not have a large amount of marked data sets. All some consider generating an antagonistic network to simulate the generation of lung nodules, which are involved in training to improve the classification accuracy of CNN networks.
Lung nodules are lung tissue abnormalities found in lung cancer patients, and early detection of lung nodules has been observed in various research works to improve 5-year survival in lung cancer patients. Pulmonary nodule detection is therefore an important stage in the early diagnosis and proper treatment planning of lung cancer, however, pulmonary nodules are relatively small, difficult to find by some physicians, and large in number of patients. With the rapid development of computers, computer-aided diagnosis (CAD) systems can detect the position and size of lung nodules more accurately in a shorter time.
Disclosure of Invention
The present invention aims to overcome the defects of the prior art and provide a method for realizing pulmonary nodule detection by generating an antagonistic network.
The technical scheme of the invention is a method for realizing pulmonary nodule detection by utilizing a generated countermeasure network, and provides an end-to-end nodule detection framework, so that nodule candidate screening and false positive reduction are integrated into a model and are trained together, and the method specifically comprises the following steps:
1) taking the generator as a nodule candidate screening model to perform primary screening; then, a discriminator is used for discriminating true and false positives;
2) the generator is transformed into a single-input single-output model, the input is a lung CT image, the lung CT image is output as a template image for marking lung nodules through the generator G1, and then a segmented lung nodule image is obtained through the generator G2; transforming the output of the discriminator into the probability of inputting the pulmonary nodule as a true nodule; and setting a proper loss function, and achieving balance through co-training.
The generator G1 of the present invention prioritizes the use of the U-Net network.
The discriminator of the invention adopts VGG19 as a backbone network, and most of the largest pooling is deleted from the convolutional layers, and only the first convolutional layer is left.
The invention reduces to two fully connected layers Fc6 and Fc7, the input is a super-resolution image, and the output of Fc7 is the probability that the input is a true nodule.
According to the design of the loss function, the loss function mainly comprises three kinds of losses, namely content loss, countermeasure loss and classification loss;
the content loss is for the generator G1The generated template is consistent with the real template and is defined as:
wherein I is the image of the input generator, IMA template marked for the CT image;
the countermeasure loss is to discriminate an image generated by the generator as false and an actual image as true, and is defined as:
wherein I is the image of the input generator, ITSegmenting the marked CT image to obtain a real lung nodule image; the classification loss is defined to make it easier to classify the images reconstructed by the generator network:
when the lung nodule image is true positive, y is 1; when the lung nodule image is false positive, y is 0.
The method comprises the following specific steps:
a. pretreatment: selecting a LUNA16 data set, wherein 888 CT data are contained in the data set, 36378 lung nodules are marked out in total, and 1186 nodules marked by three experts are selected as a final region to be detected;
at this stage, the CT image in the data set is first converted from dicom format to png format, the image is 512 × 512 pixels; then, a median filter is adopted to enhance the input CT image;
b. and (3) lung region segmentation: extracting a lung region from the preprocessed image by adopting a threshold segmentation method;
c. training a pulmonary nodule detection model;
d. automatic detection of pulmonary nodules: after the detection model is obtained, the CT file of the patient is input into a generator for generating an confrontation network after two steps of preprocessing and lung region segmentation, the generator directly outputs a lung nodule mark image, the mark image is fused with the original CT image to obtain a CT image finally provided with a lung nodule mark, the mark information comprises position and size information of a lung nodule, and finally, the automatic detection of the lung nodule on the CT file of the patient by the detection model is realized.
Advantageous effects
The method for realizing the lung nodule detection by utilizing the generation countermeasure network is an end-to-end lung nodule detection framework, not only reduces the expenditure of resources, but also further optimizes a detection model by the mutual countermeasure of a generator and a discriminator, and improves the accuracy of the whole algorithm.
Drawings
FIG. 1 is a proposed framework for generation of an antagonistic network for pulmonary nodule detection, showing the process of generating an antagonistic network training and testing.
Figure 2 generator network architecture.
FIG. 3 arbiter network architecture.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
The method comprises the following steps of a, preprocessing: the LUNA16 dataset was selected, which contained 888 CT's, and a total of 36378 lung nodules were labeled, and 1186 nodules labeled by three experts were selected as the last region to be examined. At this stage, we first convert the CT image in the data set from dicom format to png format, the image being 512 × 512 pixels, and then use a median filter to enhance the input CT image and remove unwanted noise pixels from the CT image. b. And (3) lung region segmentation: and extracting a lung region from the preprocessed image by adopting a threshold segmentation method. c. Training a pulmonary nodule detection model: since we adopt an end-to-end pulmonary nodule detection framework, lung nodule detection and reduction of false positive rate are completed in one step. At this stage, a generation model and a discrimination model in the generated countermeasure network are trained simultaneously, the generation model and the discrimination model are enabled to spread a mutual game, and finally a balance point is reached, so that the labeled image generated by the generator can directly eliminate the lung nodules with false positives, and the false positive probability output by the discriminator can further reduce the false positive rate of the lung nodules. d. Automatic detection of pulmonary nodules: after the detection model is obtained, the CT file of the patient is input into a generator for generating the countermeasure network after two steps of preprocessing and lung region segmentation, the generator directly outputs a lung nodule mark image, the mark image is fused with the original CT image to obtain a CT image with a lung nodule mark finally, and the mark information comprises the position and size information of the lung nodule. And finally, the automatic detection of the pulmonary nodules of the CT file of the patient by using the detection model is realized.
The pulmonary nodule detection scheme is implemented primarily by generating an antagonistic network. Most of the existing deep learning nodule detection systems are mostly composed of two steps: a) nodule candidate screening and b) false positive reduction, two different models trained separately were used. Although a two-step approach is typically employed, the two-step approach not only adds considerable resource overhead in training two independent deep learning models, but also prevents cross-talk between the two. In this work, we propose an end-to-end nodule detection framework that integrates nodule candidate screening and false positive reduction into one model, co-trained. The generator is used as a nodule candidate screening model to perform primary screening; and then, a discriminator is used for discriminating true and false positives, so that false positives are reduced, and the detection efficiency is improved. To achieve this goal, we have improved the original generation of the countermeasure network. As shown in fig. 1, the generator is modified into a single-input single-output model, the input is a lung CT image, the generator G1 outputs a template image for marking lung nodules, and the generator G1 preferentially considers the use of a U-Net network to ensure a good segmentation effect; obtaining a segmented lung nodule image through a generator G2; the discriminator output is modified to input the probability that the lung nodule is a true nodule. And a proper loss function is set, and through co-training, balance is achieved, and the accuracy of the whole algorithm is improved.
Design of network architecture
Generating a countermeasure network architecture can be divided into a generator and an arbiter
A generator
We adopt the U-net network as the backbone network in the generator due to the excellent performance of the U-net network in the split domain. As shown in fig. 2.
B discriminator
We use VGG19 as the backbone network in the arbiter. Since the lung nodules are very small, to avoid excessive downsampling of the lung nodules, we have removed most of the largest pooling from the convolutional layers, leaving only the first convolutional layer. Furthermore, we reduce to two fully connected layers fc6 and fc 7. The input is a super-resolution image and the output of fc7 is the probability that the input is a true nodule. As shown in fig. 3.
Design of (II) loss function
Our loss function includes mainly three kinds of losses, content loss, countermeasure loss and classification loss, respectively.
The content loss is for the generator G1The generated template is consistent with the real template. Is defined as:
wherein I is the image of the input generator, IMA labeled template for a CT image.
The countermeasure loss is to discriminate an image generated by the generator as false and an actual image as true. Is defined as:
wherein I is the image of the input generator, ITThe marked CT image is a real lung nodule image obtained by segmentation.
The classification penalty is to make it easier to classify the images reconstructed by the generator network. Is defined as:
when the lung nodule image is true positive, y is 1; when the lung nodule image is false positive, y is 0.
Claims (6)
1. A method for realizing pulmonary nodule detection by using a generation confrontation network is characterized in that an end-to-end nodule detection framework is provided, nodule candidate screening and false positive reduction are integrated into a model and are trained together, and the method comprises the following specific steps:
1) taking the generator as a nodule candidate screening model to perform primary screening; then, a discriminator is used for discriminating true and false positives;
2) the generator is transformed into a single-input single-output model, the input is a lung CT image, the lung CT image is output as a template image for marking lung nodules through the generator G1, and then a segmented lung nodule image is obtained through the generator G2;
transforming the output of the discriminator into the probability of inputting the pulmonary nodule as a true nodule;
and setting a proper loss function, and achieving balance through co-training.
2. The method of claim 1, wherein the generator G1 uses a U-Net network.
3. The method of claim 1, wherein the discriminator uses VGG19 as the backbone network, and most of the largest pooling is removed from the convolutional layers, leaving only the first convolutional layer.
4. The method of claim 1, wherein reduced to two fully connected layers Fc6 and Fc7, the input is a super-resolution image, and the output of Fc7 is the probability that the input is a true nodule.
5. The method for detecting pulmonary nodules by generating an antagonistic network according to claim 1, wherein the loss function is designed to include three kinds of losses, namely content loss, antagonistic loss and classification loss;
the content loss is for the generator G1The generated template is consistent with the real template and is defined as:
wherein I is the image of the input generator, IMA template marked for the CT image;
the countermeasure loss is to discriminate an image generated by the generator as false and an actual image as true, and is defined as:
wherein I isImage of the input generator, ITSegmenting the marked CT image to obtain a real lung nodule image; the classification loss is defined to make it easier to classify the images reconstructed by the generator network:
when the lung nodule image is true positive, y is 1; when the lung nodule image is false positive, y is 0.
6. The method for detecting pulmonary nodules by generating an antagonistic network according to any of the claims 1 to 5, characterized by the following steps:
a. pretreatment: selecting a LUNA16 data set, wherein 888 CT data are contained in the data set, 36378 lung nodules are marked out in total, and 1186 nodules marked by three experts are selected as a final region to be detected;
at this stage, the CT image in the data set is first converted from dicom format to png format, the image is 512 × 512 pixels; then, a median filter is adopted to enhance the input CT image;
b. and (3) lung region segmentation: extracting a lung region from the preprocessed image by adopting a threshold segmentation method;
c. training a pulmonary nodule detection model;
d. automatic detection of pulmonary nodules: after the detection model is obtained, the CT file of the patient is input into a generator for generating an confrontation network after two steps of preprocessing and lung region segmentation, the generator directly outputs a lung nodule mark image, the mark image is fused with the original CT image to obtain a CT image finally provided with a lung nodule mark, the mark information comprises position and size information of a lung nodule, and finally, the automatic detection of the lung nodule on the CT file of the patient by the detection model is realized.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011595134.4A CN112669284A (en) | 2020-12-29 | 2020-12-29 | Method for realizing pulmonary nodule detection by generating confrontation network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011595134.4A CN112669284A (en) | 2020-12-29 | 2020-12-29 | Method for realizing pulmonary nodule detection by generating confrontation network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112669284A true CN112669284A (en) | 2021-04-16 |
Family
ID=75411926
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011595134.4A Pending CN112669284A (en) | 2020-12-29 | 2020-12-29 | Method for realizing pulmonary nodule detection by generating confrontation network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112669284A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113554612A (en) * | 2021-07-20 | 2021-10-26 | 中国医科大学 | Automatic segmentation and texture extraction method for abnormal lung region |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107016665A (en) * | 2017-02-16 | 2017-08-04 | 浙江大学 | A kind of CT pulmonary nodule detection methods based on depth convolutional neural networks |
| CN108171700A (en) * | 2018-01-12 | 2018-06-15 | 西安电子科技大学 | Medical image pulmonary nodule detection method based on confrontation network |
| CN108389190A (en) * | 2018-02-08 | 2018-08-10 | 贵州联科卫信科技有限公司 | A kind of Lung neoplasm automatic testing method based on deep learning method |
| CN108986073A (en) * | 2018-06-04 | 2018-12-11 | 东南大学 | A kind of CT image pulmonary nodule detection method based on improved Faster R-CNN frame |
| CN110246143A (en) * | 2019-06-14 | 2019-09-17 | 吉林大学第一医院 | Lung CT image assists detection processing device |
| CN111105032A (en) * | 2019-11-28 | 2020-05-05 | 华南师范大学 | Chromosome structure abnormality detection method, system and storage medium based on GAN |
| CN111127346A (en) * | 2019-12-08 | 2020-05-08 | 复旦大学 | Multi-level image restoration method based on partial-to-integral attention mechanism |
| CN111275651A (en) * | 2020-02-25 | 2020-06-12 | 东南大学 | Face bright removal method based on antagonistic neural network |
| CN111667491A (en) * | 2020-05-09 | 2020-09-15 | 中山大学 | Breast mass image generation method with marginal landmark annotation information based on depth countermeasure network |
| CN111899193A (en) * | 2020-07-30 | 2020-11-06 | 湖北工业大学 | Criminal investigation photography system and method based on low-illumination image enhancement algorithm |
| CN111986142A (en) * | 2020-05-23 | 2020-11-24 | 冶金自动化研究设计院 | Unsupervised enhancement method for surface defect image data of hot-rolled plate coil |
| CN112133326A (en) * | 2020-09-08 | 2020-12-25 | 东南大学 | Gunshot data amplification and detection method based on antagonistic neural network |
-
2020
- 2020-12-29 CN CN202011595134.4A patent/CN112669284A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107016665A (en) * | 2017-02-16 | 2017-08-04 | 浙江大学 | A kind of CT pulmonary nodule detection methods based on depth convolutional neural networks |
| CN108171700A (en) * | 2018-01-12 | 2018-06-15 | 西安电子科技大学 | Medical image pulmonary nodule detection method based on confrontation network |
| CN108389190A (en) * | 2018-02-08 | 2018-08-10 | 贵州联科卫信科技有限公司 | A kind of Lung neoplasm automatic testing method based on deep learning method |
| CN108986073A (en) * | 2018-06-04 | 2018-12-11 | 东南大学 | A kind of CT image pulmonary nodule detection method based on improved Faster R-CNN frame |
| CN110246143A (en) * | 2019-06-14 | 2019-09-17 | 吉林大学第一医院 | Lung CT image assists detection processing device |
| CN111105032A (en) * | 2019-11-28 | 2020-05-05 | 华南师范大学 | Chromosome structure abnormality detection method, system and storage medium based on GAN |
| CN111127346A (en) * | 2019-12-08 | 2020-05-08 | 复旦大学 | Multi-level image restoration method based on partial-to-integral attention mechanism |
| CN111275651A (en) * | 2020-02-25 | 2020-06-12 | 东南大学 | Face bright removal method based on antagonistic neural network |
| CN111667491A (en) * | 2020-05-09 | 2020-09-15 | 中山大学 | Breast mass image generation method with marginal landmark annotation information based on depth countermeasure network |
| CN111986142A (en) * | 2020-05-23 | 2020-11-24 | 冶金自动化研究设计院 | Unsupervised enhancement method for surface defect image data of hot-rolled plate coil |
| CN111899193A (en) * | 2020-07-30 | 2020-11-06 | 湖北工业大学 | Criminal investigation photography system and method based on low-illumination image enhancement algorithm |
| CN112133326A (en) * | 2020-09-08 | 2020-12-25 | 东南大学 | Gunshot data amplification and detection method based on antagonistic neural network |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113554612A (en) * | 2021-07-20 | 2021-10-26 | 中国医科大学 | Automatic segmentation and texture extraction method for abnormal lung region |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111476292B (en) | Small sample element learning training method for medical image classification processing artificial intelligence | |
| CN109493308B (en) | Medical image synthesis and classification method for generating confrontation network based on condition multi-discrimination | |
| CN107464250B (en) | Automatic breast tumor segmentation method based on three-dimensional MRI (magnetic resonance imaging) image | |
| Agarwal et al. | Lung cancer detection and classification based on alexnet CNN | |
| CN108171232B (en) | Deep learning algorithm-based bacterial and viral pneumonia classification method for children | |
| CN111401480B (en) | Novel mammary gland MRI automatic auxiliary diagnosis method based on fusion attention mechanism | |
| CN108564026B (en) | Network construction method and system for thyroid tumor cytology smear image classification | |
| CN114897914B (en) | Semi-supervised CT image segmentation method based on countermeasure training | |
| CN112150442A (en) | New crown diagnosis system based on deep convolutional neural network and multi-instance learning | |
| CN110503630A (en) | A kind of cerebral hemorrhage classification, positioning and prediction technique based on three dimensional depth learning model | |
| CN112700461B (en) | System for pulmonary nodule detection and characterization class identification | |
| CN111767952B (en) | Interpretable lung nodule benign and malignant classification method | |
| CN115272196B (en) | Method for predicting focus area in histopathological image | |
| CN108010013A (en) | A kind of lung CT image pulmonary nodule detection methods | |
| CN116030325A (en) | Lung nodule CT image recognition method based on deep hybrid learning framework | |
| Huang et al. | Segmentation of cervical cell images based on generative adversarial networks | |
| CN110570405A (en) | pulmonary nodule intelligent diagnosis method based on mixed features | |
| CN112419396A (en) | Thyroid ultrasonic video automatic analysis method and system | |
| CN117036288A (en) | Tumor subtype diagnosis method for full-slice pathological image | |
| Tian et al. | Radiomics and its clinical application: artificial intelligence and medical big data | |
| Shao et al. | Application of U-Net and Optimized Clustering in Medical Image Segmentation: A Review. | |
| CN113902676A (en) | Lung nodule image detection method and system based on attention mechanism neural network | |
| CN112669284A (en) | Method for realizing pulmonary nodule detection by generating confrontation network | |
| Pan et al. | A review of machine learning approaches, challenges and prospects for computational tumor pathology | |
| CN116029994A (en) | Brain glioma MR image segmentation method based on edge loss and knowledge fusion decision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20210416 |