CN110838114A - Pulmonary nodule detection method, device and computer storage medium - Google Patents
Pulmonary nodule detection method, device and computer storage medium Download PDFInfo
- Publication number
- CN110838114A CN110838114A CN201911095432.4A CN201911095432A CN110838114A CN 110838114 A CN110838114 A CN 110838114A CN 201911095432 A CN201911095432 A CN 201911095432A CN 110838114 A CN110838114 A CN 110838114A
- Authority
- CN
- China
- Prior art keywords
- lung nodule
- region
- image
- lung
- candidate region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10104—Positron emission tomography [PET]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30061—Lung
- G06T2207/30064—Lung nodule
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/03—Recognition of patterns in medical or anatomical images
- G06V2201/032—Recognition of patterns in medical or anatomical images of protuberances, polyps nodules, etc.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Evolutionary Biology (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Apparatus For Radiation Diagnosis (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a pulmonary nodule detection method, a pulmonary nodule detection device and a computer storage medium. The method comprises the following steps: performing primary nodule detection on the acquired CT image and PET image of the target lung based on a Faster R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image; performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule region in the first lung nodule candidate region and a second lung nodule region in the second lung nodule candidate region; and acquiring the shape feature of the first lung nodule region and the metabolic feature of the second lung nodule region, and classifying the first lung nodule region and the second lung nodule region according to the shape feature and the metabolic feature by using an SVM (support vector machine) classifier to determine a final lung nodule region in the CT image and the PET image. By utilizing the technical scheme disclosed by the application, the accuracy of the diagnosis result of a doctor can be improved.
Description
Technical Field
The present application relates to the field of medical image data processing technologies, and in particular, to a pulmonary nodule detection method, apparatus, and computer storage medium.
Background
The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.
Lung cancer is one of the most rapidly growing malignancies and has posed a great threat to human health and life. Compared with other cancers, the biological characteristics of lung cancer are very complex, no obvious symptoms exist in early stage, most of lung cancer is found to be in middle and late stage, and the treatment cost is high and the effect is poor. Therefore, early screening appears to be of paramount importance. Early lung cancer generally manifests as malignant lung nodules in the lung, and therefore, early screening generally begins with the detection of lung nodules.
Early detection of lung nodules was dominated by electron Computed Tomography (CT) scanning, and in recent years Positron Emission Tomography (PET) was also used as a screening tool for cancer. PET is an imaging device that reflects the genetic, molecular, metabolic and functional status of lesions. It uses radioactive nuclide with positron to mark human body compound or metabolite, such as glucose, protein, nucleic acid, fatty acid, receptor ligand and water, etc. human body metabolite, and after it is injected into the body of examined person, the positive electronic nuclide is associated with the marked object and participates in the cell metabolic activity of human body tissue and organ in the course of physiological and pathological process, and is recollected and distributed, then it uses PET external detection nerve device to make collection so as to obtain the information of human body metabolic activity molecular level, and can provide the biological metabolic information of disease for clinic.
PET/CT integrated machine, namely positron emission tomography and electron computer tomography system, organically integrates PET and CT, and complements the advantages of the PET and CT. The system is used for scanning the whole body of a patient quickly once, a CT anatomical image and a PET functional metabolic image can be obtained simultaneously, biological metabolic information of a focus and accurate anatomical positioning and structure can be presented to a doctor simultaneously through the two images, and therefore the doctor can judge diseases earlier and faster. Therefore, the combination of PET and CT can provide great help for doctors to discover lung cancer in early stage.
However, this is accompanied by a large number of PET/CT images, which greatly increases the workload of the physician. Moreover, the pulmonary nodules vary in variety, size and shape, and are easily missed and misdiagnosed if observed only by the naked eye of a doctor.
Disclosure of Invention
An embodiment of the present application provides a pulmonary nodule detection method, an apparatus and a computer storage medium, so as to solve at least one problem in the prior art.
In order to solve the above technical problem, an embodiment of the present application provides a pulmonary nodule detection method, which may include:
performing preliminary nodule detection on the acquired CT image and PET image of the target lung based on a Faster R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image;
performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule candidate region in the first lung nodule candidate region and a second lung nodule candidate region in the second lung nodule candidate region;
and acquiring the shape feature of the first lung nodule region and the metabolic feature of the second lung nodule region, and classifying the first lung nodule region and the second lung nodule region according to the shape feature and the metabolic feature by utilizing an SVM (support vector machine) classifier so as to determine a final lung nodule region in the CT image and the PET image.
Optionally, the step of performing preliminary nodule detection on the acquired CT image and PET image of the target lung based on the fast R-CNN model comprises:
extracting lung nodule characteristics in the CT image and the PET image by using a characteristic extraction network;
determining a plurality of potential lung nodule regions in the CT image and the PET image according to the extracted lung nodule features by using a region suggestion network;
classifying the determined plurality of potential lung nodule regions using a classification regression network and performing bounding box regression using a loss function to determine the first and second candidate lung nodule regions of the plurality of potential lung nodule regions.
Optionally, the step of extracting the lung nodule features in the CT image and the PET image by using a feature extraction network includes:
performing convolution processing on the CT image and the PET image by using VGG-16 containing multilayer convolution layers to obtain a convolution characteristic map;
and performing deconvolution processing on the obtained convolution feature map by using a deconvolution layer to extract the lung nodule feature.
Optionally, the lung nodule features include shape features in the CT image and metabolic features in the PET image.
Optionally, before the classifying the determined plurality of potential lung nodule regions by using a classification regression network and performing bounding box regression by using a loss function, the method further comprises:
pooling the plurality of potential lung nodule regions with a pooling layer of interest to make the plurality of potential lung nodule regions the same size.
Optionally, the first lung nodule region candidate includes a lung nodule region of a CT axial image and a lung nodule region of a CT sagittal image, and the second lung nodule region candidate includes an initial lung nodule region of the PET image.
Optionally, the step of performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model comprises:
performing feature extraction on the first lung nodule candidate region and the second lung nodule candidate region by using a convolutional layer;
performing pooling processing on the first lung nodule candidate region and the second lung nodule candidate region after feature extraction by using a pooling layer;
and performing full-connection processing on the first lung nodule candidate region and the second lung nodule candidate region processed by the pooling layer by using a full-connection layer to obtain a judgment probability of the lung nodule region, and determining the first lung nodule region and the second lung nodule region according to the obtained judgment probability.
An embodiment of the present application further provides a pulmonary nodule detection apparatus, which may include:
a preliminary detection unit configured to perform preliminary nodule detection on the acquired CT image and PET image of the target lung based on a fast R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image;
an identification unit configured to perform false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule region in the first lung nodule candidate region and a second lung nodule region in the second lung nodule candidate region;
a determination unit configured to obtain a shape feature of the first lung nodule region and a metabolic feature of the second lung nodule region, and classify the first lung nodule region and the second lung nodule region according to the shape feature and the metabolic feature by using an SVM classifier to determine a final lung nodule region in the CT image and the PET image.
Optionally, the preliminary detection unit includes:
a feature extraction subunit configured to extract lung nodule features in the CT image and the PET image using a feature extraction network;
a determining subunit configured to determine a plurality of potential lung nodule regions in the CT image and the PET image from the extracted lung nodule features using a region suggestion network;
a classification regression subunit configured to classify the determined plurality of potential lung nodule regions using a classification regression network and perform bounding box regression using a loss function to determine the first and second candidate lung nodule regions of the plurality of potential lung nodule regions.
Embodiments of the present application further provide a computer storage medium having program instructions stored thereon, where the program instructions, when executed, implement the following functions:
performing preliminary nodule detection on the acquired CT image and PET image of the target lung based on a Faster R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image;
performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule region of the first lung nodule candidate region and a second lung nodule region of the second lung nodule candidate region;
and acquiring the shape feature of the first lung nodule region and the metabolic feature of the second lung nodule region, and classifying the first lung nodule region and the second lung nodule region according to the shape feature and the metabolic feature by utilizing an SVM (support vector machine) classifier to determine a final lung nodule region in the CT image and the PET image.
According to the technical scheme provided by the embodiment of the application, the embodiment of the application carries out primary nodule detection on the acquired CT image and PET image of the target lung based on the fast R-CNN model so as to determine a first candidate lung nodule region in the CT image and a second candidate lung nodule region in the PET image; performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule region in the first lung nodule candidate region and a second lung nodule region in the second lung nodule candidate region; and classifying the first lung nodule region and the second lung nodule region by using an SVM classifier according to the acquired shape characteristics of the first lung nodule region and the acquired metabolic characteristics of the second lung nodule region to determine the final lung nodule region in the CT image and the PET image, so that the film reading time of a doctor can be greatly shortened, and the accuracy of a diagnosis result can be improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without any creative effort.
Fig. 1 is a flowchart of a pulmonary nodule detection method provided in an embodiment of the present application;
FIG. 2 is a schematic structural diagram of the Faster R-CNN model;
fig. 3 is a diagram illustrating an effect of an application example of the lung nodule detection method provided in the embodiment of the present application;
fig. 4 is a schematic structural diagram of a pulmonary nodule detection apparatus according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only used for explaining a part of the embodiments of the present application, but not all embodiments, and are not intended to limit the scope of the present application or the claims. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It will be understood that when an element is referred to as being "disposed on" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected/coupled" to another element, it can be directly connected/coupled to the other element or intervening elements may also be present. The term "connected/coupled" as used herein may include electrical and/or mechanical physical connections/couplings. The term "comprises/comprising" as used herein refers to the presence of features, steps or elements, but does not preclude the presence or addition of one or more other features, steps or elements. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.
In addition, in the description of the present application, the terms "first", "second", "third", and the like are used for descriptive purposes only and to distinguish similar objects, and there is no order of precedence between the two, and no indication or implication of relative importance is to be inferred.
The lung nodule detection method, apparatus and computer storage medium provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.
As shown in fig. 1, an embodiment of the present application provides a pulmonary nodule detection method, which may include the following steps:
s1: and performing primary lung nodule detection on the acquired CT image and the PET image of the target lung based on a Faster R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image.
The CT image and the PET image may be images obtained by scanning lungs of a living body (e.g., a human) using a PET/CT scanner. Among them, the CT image mainly represents anatomical features of a subject, and the PET image mainly represents functional metabolic features of the subject.
As shown in FIG. 2, the Faster R-CNN model is a neural network model for target detection, and may generally include a feature extraction network, a region suggestion network (RPN), and a classification regression network. The feature extraction network may be used to extract lung nodule features in CT images and PET images, and may include multiple convolutional layers, e.g., VGG-16 (i.e., a network with a depth of 16) containing multiple convolutional layers, among others. Each convolutional layer generates an abstract description by using the network information output by the last convolutional layer, for example, the first convolutional layer can be used for learning edge information of an input image, and the second convolutional layer can be used for learning pattern information in the edge to learn more complex shape and other information. In addition, the feature extraction network may further include a deconvolution layer connected to the last convolution layer, which may improve the accuracy of the lung nodule detection result. The kernel size, stride size, pad size, and number of kernels of the deconvolution layer may be 4, and 512, respectively. The region suggestion network may be used to determine a plurality of potential lung nodule regions in the CT image and the PET image based on the shape features and the metabolic features extracted by the feature extraction network. The classification regression network may be configured to classify a plurality of potential lung nodule regions determined by the region suggestion network and perform bounding box regression using a loss function, so as to determine candidate lung nodule regions in the potential lung nodule regions, and the classification regression network may further perform fine tuning on positions of the candidate lung nodule regions. In addition, the Faster R-CNN model may also include a pooling of interest (RoI) layer, which may be used to pool multiple potential lung nodule regions to make them the same size prior to performing a classification regression on the potential lung nodule regions.
After acquiring the CT image and the PET image of the target lung, preliminary lung nodule detection may be performed on the acquired two images based on the fast R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image. Specifically, the method comprises the following steps:
first, after receiving the CT image and the PET image, the feature extraction network may be used to extract lung nodule features in the two images, such as shape features in the CT image (e.g., a volume, a surface region, and cross-sectional regions on different dimensional planes (e.g., XY, YZ, and XZ), contour pixels, and CT values, etc.) and metabolic features in the PET image (e.g., Standardized Uptake Values (SUV) of the lung nodule, a volume, a surface region, cross-sectional regions on different dimensional planes, etc.). Specifically, the CT image and the PET image may be subjected to convolution calculation using VGG-16 including 5 convolutional layers to obtain a convolutional feature map deeper than the received CT image and PET image but smaller in spatial resolution, and then the obtained convolutional feature map may be subjected to deconvolution calculation using a deconvolution layer to extract lung nodule features in the CT image and the PET image. By mapping the convolution characteristic map to the original image space by using the deconvolution layer, the influence of background factors on the extraction of candidate lung nodules can be reduced, and the accuracy of the detection result can be improved.
Then, after extracting the lung nodule features in the CT image and the PET image, a plurality of potential lung nodule regions in the CT image and the PET image can be determined according to the extracted lung nodule features by using the region suggestion network. Specifically, the RPN may determine a series of good target candidate regions (nanoposes) using all bounding boxes or Anchors (Anchors), and then may determine a plurality of potential lung nodule regions in the CT image and the PET image, which may include regions without confidence scores and suspected lung nodule regions, according to the determined coordinates of the target candidate regions and their corresponding target confidence scores. The target confidence score may be obtained by utilizing a fully connected layer.
Finally, the determined plurality of potential lung nodule regions may be classified by a classification regression network and bounding box regression may be performed by a loss function to determine a first lung nodule candidate region and a second lung nodule candidate region of the potential lung nodule regions. Specifically, first, the convolution feature maps of multiple potential lung nodule regions may be cropped, and then each cropped region may be scaled to a fixed size (e.g., 14x14) using a bilinear interpolation algorithm; maximum pooling may then be performed using a convolution kernel (e.g., of size 2x2) to obtain a feature map for each potential lung nodule region; next, features may be extracted from the obtained feature map to classify a plurality of potential lung nodule regions, where the classification may include a lung nodule target class and a background class, and finally, a frame regression prediction may be performed by using a loss function to obtain center coordinates, widths, and heights of regions where the two classes are located, so as to determine a first lung nodule candidate region and a second lung nodule candidate region. The first lung nodule candidate region may include a lung nodule region of a CT axial image (i.e., a CT image obtained by scanning a subject in a transverse direction parallel to a ground plane, which may also be referred to as a CT transverse scan image) and a lung nodule region of a CT sagittal image (i.e., a CT image obtained by scanning a subject in an anteroposterior direction), and the second lung nodule candidate region may include an initial lung nodule region in which a lung nodule may exist in a PET image.
In addition, before determining the first lung nodule candidate region and the second lung nodule candidate region of the plurality of potential lung nodule regions, the potential lung nodule regions may be pooled by using the RoI pooling layer to make the potential lung nodule regions have the same size, if necessary.
S2: and performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on the convolutional neural network model to determine a first lung nodule candidate region in the first lung nodule candidate region and a second lung nodule candidate region in the second lung nodule candidate region.
False positive regions may occur in CT and PET images taken due to environmental, operational, and/or patient-specific factors. For example, for lung nodules, false positive areas occur primarily at the stenotic bronchi and blood vessels in the lung. The occurrence of false positive regions may affect the final diagnosis result, so in order to improve the accuracy of the diagnosis result, the false positive regions in the first lung nodule candidate region and the second lung nodule candidate region need to be removed.
The convolutional neural network model may generally include convolutional layers, pooled layers, and fully-connected layers connected to each other, e.g., may include three convolutional layers, three pooled layers, and two fully-connected layers. Wherein the convolutional layer may include a plurality of filters and convolutional kernels, and the fully-connected layer may include a multi-layer perceptron.
After determining the first lung nodule candidate region in the CT image and the second lung nodule candidate region in the PET image, false positive region identification may be performed on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine the first lung nodule candidate region in the first lung nodule candidate region and the second lung nodule candidate region in the second lung nodule candidate region. Specifically, feature extraction may be performed on the first lung nodule candidate region and the second lung nodule candidate region by using the convolutional layer; then, overlapping sampling can be performed on the first lung nodule candidate region and the second lung nodule candidate region after feature extraction by using a pooling layer, so that feature selection and information filtering of the first lung nodule candidate region and the second lung nodule candidate region are realized; then, the first lung nodule candidate region and the second lung nodule candidate region after the pooling layer processing may be subjected to full-connection processing by using a full-connection layer to obtain a determination probability of the lung nodule region, so that a first lung nodule candidate region in the first lung nodule candidate region and a second lung nodule candidate region in the second lung nodule candidate region may be determined according to the obtained determination probability, that is, non-lung nodule regions in the first lung nodule candidate region and the second lung nodule candidate region are removed.
For example, for a 3-channel color image with 32x32 pixels (width x height) of the input CT image and PET image, the convolutional layer may use 32 filters and a kernel of 5x5x3 to perform feature extraction, and finally may obtain a feature mapping matrix with 32x32x32 pixels; the pooling layer performs pooling on the feature mapping matrix, reduces the feature mapping matrix to 16x16x32, and finally outputs a matrix with the size of 8 x 64; the fully connected layer may classify the matrix output by the pooling layer using softmax regression, that is, classify the first lung nodule candidate region and the second lung nodule candidate region into a lung nodule region and a non-lung nodule region, and output the determination probabilities of the two regions, so that the first lung nodule candidate region in the first lung nodule candidate region and the second lung nodule candidate region in the second lung nodule candidate region may be determined according to the output determination probabilities.
S3: and acquiring the shape feature of the first lung nodule region and the metabolic feature of the second lung nodule region, and classifying the first lung nodule region and the second lung nodule region according to the acquired shape feature and metabolic feature by using an SVM (support vector machine) classifier to determine a final lung nodule region in the CT image and the PET image.
A Support Vector Machine (SVM) classifier is a two-class classification model whose basic model is defined as a linear classifier with the largest spacing in feature space.
After determining a first lung nodule region of the first lung nodule candidate region and a second lung nodule region of the second lung nodule candidate region, shape features of the first lung nodule candidate region, such as a volume, a surface region, cross-sectional regions on three planes (e.g., XY, XZ, and YZ), fusion, contour pixels, and CT values (a maximum value, a center value, and a standard deviation of the first lung nodule candidate region) may be obtained from the CT image, and metabolic features of the second lung nodule candidate region, such as a normalized uptake (SUV) of the center of the second lung nodule candidate region, a maximum value and an average value of an SUV in the second lung nodule candidate region, a volume, a surface region, and cross-sectional regions on three planes, may also be obtained from the PET image. The SVM classifier may then classify the first lung nodule region and the second lung nodule region based on the acquired shape features and metabolic features. In particular, the SVM classifier may train shape features of a first lung nodule region and metabolic features in a second lung nodule region, and in combination with changes in some features (e.g., for the second lung nodule region in a PET image, changes in its shape may typically be observed in the corresponding CT image, while for the first lung nodule region in the CT image, if there are no high absorption regions in the corresponding PET image, many features in the second lung nodule region may be set to 0), implement classification of the first lung nodule region and the second lung nodule region, thereby determining a final lung nodule region.
The specific process of how the SVM classifier classifies the inputted features can refer to the related description in the prior art, and is not described in detail herein.
By observing the final lung nodule region in both the PET and CT images, the physician can quickly make his diagnosis.
As can be seen from the above description, in the embodiment of the present application, preliminary nodule detection is performed on the acquired CT image and PET image of the target lung based on the fast R-CNN model, so as to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image; performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule candidate region in the first lung nodule candidate region and a second lung nodule candidate region in the second lung nodule candidate region; and acquiring the shape feature of the first lung nodule region and the metabolic feature of the second lung nodule region, and classifying the first lung nodule region and the second lung nodule region according to the acquired shape feature and metabolic feature by using an SVM classifier to determine the final lung nodule region in the CT image and the PET image, so that the film reading time of a doctor can be greatly shortened, and the accuracy of a diagnosis result of the doctor is improved.
The following is a specific example to illustrate the beneficial effects of the lung nodule detection method provided by the embodiments of the present application. As shown in fig. 3(a) -3(d), wherein fig. 3(a) and 3(b) are raw PET and CT images, respectively; fig. 3(c) and 3(d) are respectively a PET image and a CT image output after being processed by the lung nodule detection method provided by the embodiment of the present application, wherein the regions a and B are lung nodule regions. As can be seen from these figures, the lung nodule regions that are not detected in the CT image are detected in the PET image, and the lung nodule regions that are not apparent in the PET image are clearly detected in the CT image. Therefore, the lung nodule detection method provided by the embodiment of the application can achieve the effect of assisting a doctor in diagnosing lung nodules, reduces missed diagnosis or misdiagnosis and improves diagnosis efficiency.
An embodiment of the present application further provides a pulmonary nodule detection apparatus, as shown in fig. 4, the pulmonary nodule detection apparatus may include:
a preliminary detection unit 310, which may be configured to perform preliminary nodule detection on the acquired CT image and PET image of the target lung based on the fast R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image;
an identifying unit 320, which may be configured to perform false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule region in the first lung nodule candidate region and a second lung nodule region in the second lung nodule candidate region; and
a determining unit 330, which may be configured to obtain a shape feature of the first lung nodule region and a metabolic feature of the second lung nodule region, and classify the first lung nodule region and the second lung nodule region according to the obtained shape feature and metabolic feature by using an SVM classifier to determine a final lung nodule region in the CT image and the PET image.
In an embodiment, the preliminary detection unit 310 may include (not shown in the figure): a feature extraction subunit, which may be configured to extract lung nodule features in the CT image and the PET image using a feature extraction network; a determining subunit, which may be configured to determine a plurality of potential lung nodule regions in the CT image and the PET image from the extracted lung nodule features using a region suggestion network; a classification regression subunit configured to classify the determined plurality of potential lung nodule regions using a classification regression network and perform bounding box regression using a loss function to determine a first lung nodule candidate region and the second lung nodule candidate region of the plurality of potential lung nodule regions.
With regard to the detailed description of the preliminary detection unit 310, the identification unit 320 and the classification unit 330, reference may be made to the description of steps S1 to S3 in the above method embodiment, which is not described redundantly herein.
The device may be a server, an electronic device, or the like, or may be any device capable of performing data processing based on a neural network, and is not limited thereto. In addition, the functions realized by the units in the above-described apparatus may be executed by a processor in the computer according to program instructions stored in the memory.
Embodiments of the present application further provide a computer storage medium having program instructions stored thereon, where the program instructions, when executed, may implement the following functions: performing primary nodule detection on the acquired CT image and PET image of the target lung based on a Faster R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image; performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule candidate region in the first lung nodule candidate region and a second lung nodule candidate region in the second lung nodule candidate region; and acquiring the shape feature of the first lung nodule region and the metabolic feature of the second lung nodule region, and classifying the first lung nodule region and the second lung nodule region according to the acquired shape feature and metabolic feature by using an SVM (support vector machine) classifier to determine a final lung nodule region in the CT image and the PET image.
The program instructions may be executed by a processor or other processing device.
The apparatuses, units and the like explained in the above embodiments may be specifically implemented by a computer chip and/or an entity, or implemented by a product with certain functions. For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functions of the various elements may be implemented in the same or multiple computer chips when implementing the present application.
Although the present application provides method steps as described in the above embodiments or flowcharts, additional or fewer steps may be included in the method, based on conventional or non-inventive efforts. In the case of steps where no necessary causal relationship exists logically, the order of execution of the steps is not limited to that provided by the embodiments of the present application.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments.
The embodiments described above are described in order to enable those skilled in the art to understand and use the present application. It will be readily apparent to those skilled in the art that various modifications to these embodiments may be made, and the generic principles described herein may be applied to other embodiments without the use of the inventive faculty. Therefore, the present application is not limited to the above embodiments, and those skilled in the art should make improvements and modifications within the scope of the present application based on the disclosure of the present application.
Claims (10)
1. A pulmonary nodule detection method, comprising:
performing preliminary nodule detection on the acquired CT image and PET image of the target lung based on a Faster R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image;
performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule candidate region in the first lung nodule candidate region and a second lung nodule candidate region in the second lung nodule candidate region;
and acquiring the shape feature of the first lung nodule region and the metabolic feature of the second lung nodule region, and classifying the first lung nodule region and the second lung nodule region according to the shape feature and the metabolic feature by utilizing an SVM (support vector machine) classifier so as to determine a final lung nodule region in the CT image and the PET image.
2. The method as set forth in claim 1, wherein the step of performing preliminary nodule detection on the acquired CT and PET images of the target lung based on the Faster R-CNN model comprises:
extracting lung nodule characteristics in the CT image and the PET image by using a characteristic extraction network;
determining a plurality of potential lung nodule regions in the CT image and the PET image according to the extracted lung nodule features by using a region suggestion network;
classifying the determined plurality of potential lung nodule regions using a classification regression network and performing bounding box regression using a loss function to determine the first and second candidate lung nodule regions of the plurality of potential lung nodule regions.
3. The method according to claim 2, wherein the step of extracting lung nodule features in the CT image and the PET image by using a feature extraction network comprises:
performing convolution processing on the CT image and the PET image by using VGG-16 containing multilayer convolution layers to obtain a convolution characteristic map;
and performing deconvolution processing on the obtained convolution feature map by using a deconvolution layer to extract the lung nodule feature.
4. The method of claim 3, wherein the lung nodule features include shape features in the CT image and metabolic features in the PET image.
5. The method of claim 2, wherein prior to classifying the determined plurality of potential lung nodule regions using a classification regression network and performing bounding box regression using a loss function, the method further comprises:
pooling the plurality of potential lung nodule regions with a pooling layer of interest to make the plurality of potential lung nodule regions the same size.
6. The method of claim 2 or 5, wherein the first lung nodule region candidate comprises a lung nodule region of a CT axial image and a lung nodule region of a CT sagittal image, and the second lung nodule region candidate comprises an initial lung nodule region of the PET image.
7. The method of claim 1, wherein the step of performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model comprises:
performing feature extraction on the first lung nodule candidate region and the second lung nodule candidate region by using a convolutional layer;
performing pooling processing on the first lung nodule candidate region and the second lung nodule candidate region after feature extraction by using a pooling layer;
and performing full-connection processing on the first lung nodule candidate region and the second lung nodule candidate region processed by the pooling layer by using a full-connection layer to obtain a judgment probability of the lung nodule region, and determining the first lung nodule region and the second lung nodule region according to the obtained judgment probability.
8. A pulmonary nodule detection apparatus, comprising:
a preliminary detection unit configured to perform preliminary nodule detection on the acquired CT image and PET image of the target lung based on a fast R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image;
an identification unit configured to perform false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule region in the first lung nodule candidate region and a second lung nodule region in the second lung nodule candidate region;
a determination unit configured to obtain a shape feature of the first lung nodule region and a metabolic feature of the second lung nodule region, and classify the first lung nodule region and the second lung nodule region according to the shape feature and the metabolic feature by using an SVM classifier to determine a final lung nodule region in the CT image and the PET image.
9. The pulmonary nodule detection apparatus of claim 8, wherein the preliminary detection unit comprises:
a feature extraction subunit configured to extract lung nodule features in the CT image and the PET image using a feature extraction network;
a determining subunit configured to determine a plurality of potential lung nodule regions in the CT image and the PET image from the extracted lung nodule features using a region suggestion network;
a classification regression subunit configured to classify the determined plurality of potential lung nodule regions using a classification regression network and perform bounding box regression using a loss function to determine the first and second candidate lung nodule regions of the plurality of potential lung nodule regions.
10. A computer storage medium having stored thereon program instructions that, when executed, perform the functions of:
performing preliminary nodule detection on the acquired CT image and PET image of the target lung based on a Faster R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image;
performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule candidate region in the first lung nodule candidate region and a second lung nodule candidate region in the second lung nodule candidate region;
and acquiring the shape feature of the first lung nodule region and the metabolic feature of the second lung nodule region, and classifying the first lung nodule region and the second lung nodule region according to the shape feature and the metabolic feature by utilizing an SVM (support vector machine) classifier so as to determine a final lung nodule region in the CT image and the PET image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911095432.4A CN110838114B (en) | 2019-11-11 | 2019-11-11 | Pulmonary nodule detection method, device and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911095432.4A CN110838114B (en) | 2019-11-11 | 2019-11-11 | Pulmonary nodule detection method, device and computer storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110838114A true CN110838114A (en) | 2020-02-25 |
| CN110838114B CN110838114B (en) | 2022-07-22 |
Family
ID=69574785
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911095432.4A Active CN110838114B (en) | 2019-11-11 | 2019-11-11 | Pulmonary nodule detection method, device and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110838114B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110969622A (en) * | 2020-02-28 | 2020-04-07 | 南京安科医疗科技有限公司 | Image processing method and system for assisting pneumonia diagnosis |
| CN111768845A (en) * | 2020-06-30 | 2020-10-13 | 重庆大学 | Pulmonary nodule auxiliary detection method based on optimal multi-scale perception |
| CN113077434A (en) * | 2021-03-30 | 2021-07-06 | 零氪智慧医疗科技(天津)有限公司 | Method, device and storage medium for lung cancer identification based on multi-modal information |
| CN113506289A (en) * | 2021-07-28 | 2021-10-15 | 中山仰视科技有限公司 | Method for classifying false positive of pulmonary nodule by using double-flow network |
| CN113822861A (en) * | 2021-09-07 | 2021-12-21 | 中山大学中山眼科中心 | Method and device for judging eye surface swelling |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106530296A (en) * | 2016-11-07 | 2017-03-22 | 首都医科大学 | Lung detection method and device based on PET/CT image features |
| CN107230206A (en) * | 2017-06-02 | 2017-10-03 | 太原理工大学 | A kind of 3D Lung neoplasm dividing methods of the super voxel sequence lung images based on multi-modal data |
| CN108010021A (en) * | 2017-11-30 | 2018-05-08 | 上海联影医疗科技有限公司 | A kind of magic magiscan and method |
| CN108986073A (en) * | 2018-06-04 | 2018-12-11 | 东南大学 | A kind of CT image pulmonary nodule detection method based on improved Faster R-CNN frame |
| CN110223352A (en) * | 2019-06-14 | 2019-09-10 | 浙江明峰智能医疗科技有限公司 | A kind of medical image scanning automatic positioning method based on deep learning |
-
2019
- 2019-11-11 CN CN201911095432.4A patent/CN110838114B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106530296A (en) * | 2016-11-07 | 2017-03-22 | 首都医科大学 | Lung detection method and device based on PET/CT image features |
| CN107230206A (en) * | 2017-06-02 | 2017-10-03 | 太原理工大学 | A kind of 3D Lung neoplasm dividing methods of the super voxel sequence lung images based on multi-modal data |
| CN108010021A (en) * | 2017-11-30 | 2018-05-08 | 上海联影医疗科技有限公司 | A kind of magic magiscan and method |
| CN108986073A (en) * | 2018-06-04 | 2018-12-11 | 东南大学 | A kind of CT image pulmonary nodule detection method based on improved Faster R-CNN frame |
| CN110223352A (en) * | 2019-06-14 | 2019-09-10 | 浙江明峰智能医疗科技有限公司 | A kind of medical image scanning automatic positioning method based on deep learning |
Non-Patent Citations (1)
| Title |
|---|
| 姚红革 等: "多模态融合的深度学习脑肿瘤检测方法", 《光子学报》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110969622A (en) * | 2020-02-28 | 2020-04-07 | 南京安科医疗科技有限公司 | Image processing method and system for assisting pneumonia diagnosis |
| CN111768845A (en) * | 2020-06-30 | 2020-10-13 | 重庆大学 | Pulmonary nodule auxiliary detection method based on optimal multi-scale perception |
| CN111768845B (en) * | 2020-06-30 | 2023-08-11 | 重庆大学 | Pulmonary nodule auxiliary detection method based on optimal multi-scale perception |
| CN113077434A (en) * | 2021-03-30 | 2021-07-06 | 零氪智慧医疗科技(天津)有限公司 | Method, device and storage medium for lung cancer identification based on multi-modal information |
| CN113077434B (en) * | 2021-03-30 | 2023-01-24 | 零氪智慧医疗科技(天津)有限公司 | Method, device and storage medium for lung cancer identification based on multi-modal information |
| CN113506289A (en) * | 2021-07-28 | 2021-10-15 | 中山仰视科技有限公司 | Method for classifying false positive of pulmonary nodule by using double-flow network |
| CN113506289B (en) * | 2021-07-28 | 2024-03-29 | 中山仰视科技有限公司 | Method for classifying false positives of lung nodules by using double-flow network |
| CN113822861A (en) * | 2021-09-07 | 2021-12-21 | 中山大学中山眼科中心 | Method and device for judging eye surface swelling |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110838114B (en) | 2022-07-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Haque et al. | Deep learning approaches to biomedical image segmentation | |
| CN110838114B (en) | Pulmonary nodule detection method, device and computer storage medium | |
| Halder et al. | Lung nodule detection from feature engineering to deep learning in thoracic CT images: a comprehensive review | |
| Yousef et al. | A holistic overview of deep learning approach in medical imaging | |
| Xu et al. | DeepLN: a framework for automatic lung nodule detection using multi-resolution CT screening images | |
| JP2006034585A (en) | Picture display device and picture display method, and program thereof | |
| JP2018175217A (en) | Image processing apparatus, method and program | |
| EP4118617A1 (en) | Automated detection of tumors based on image processing | |
| CN112862756A (en) | Method for identifying pathological change type and gene mutation in thyroid tumor pathological image | |
| Alaskar et al. | Deep learning approaches for automatic localization in medical images | |
| Yar et al. | Lung nodule detection and classification using 2D and 3D convolution neural networks (CNNs) | |
| CN113643279A (en) | Pulmonary nodule detection equipment and device based on CT (computed tomography) image and storage medium | |
| Peña-Solórzano et al. | Findings from machine learning in clinical medical imaging applications–Lessons for translation to the forensic setting | |
| Song et al. | A content-based image retrieval framework for multi-modality lung images | |
| CN113205111B (en) | Identification method and device suitable for liver tumor and electronic equipment | |
| Almutairi et al. | An Efficient USE‐Net Deep Learning Model for Cancer Detection | |
| Anshad et al. | Recent methods for the detection of tumor using computer aided diagnosis—A review | |
| Wei et al. | Multi-feature fusion for ultrasound breast image classification of benign and malignant | |
| EP3381010A1 (en) | Process for processing medical images of a face for recognition of facial dysmorphisms | |
| Chen et al. | Ischemic stroke subtyping method combining convolutional neural network and radiomics | |
| Al-Saeed et al. | Liver segmentation using fast-generalized fuzzy C-means (FG-FCM) from CT scans | |
| Amritha et al. | Liver tumor segmentation and classification using deep learning | |
| Hasan et al. | Lung Segmentation from Chest X-Ray Images Using Deeplabv3plus-Based CNN Model | |
| Nguyen et al. | Anomalies Detection in Chest X-Rays Images Using Faster R-CNN and YOLO. | |
| EP4356837A1 (en) | Medical image diagnosis system, medical image diagnosis system evaluation method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |