CN110838114B - Pulmonary nodule detection method, device and computer storage medium - Google Patents
Pulmonary nodule detection method, device and computer storage medium Download PDFInfo
- Publication number
- CN110838114B CN110838114B CN201911095432.4A CN201911095432A CN110838114B CN 110838114 B CN110838114 B CN 110838114B CN 201911095432 A CN201911095432 A CN 201911095432A CN 110838114 B CN110838114 B CN 110838114B
- Authority
- CN
- China
- Prior art keywords
- lung nodule
- region
- lung
- image
- candidate region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10104—Positron emission tomography [PET]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30061—Lung
- G06T2207/30064—Lung nodule
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/03—Recognition of patterns in medical or anatomical images
- G06V2201/032—Recognition of patterns in medical or anatomical images of protuberances, polyps nodules, etc.
Abstract
The application discloses a pulmonary nodule detection method, a pulmonary nodule detection device and a computer storage medium. The method comprises the following steps: performing primary nodule detection on the acquired CT image and PET image of the target lung based on a Faster R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image; performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule region in the first lung nodule candidate region and a second lung nodule region in the second lung nodule candidate region; and acquiring the shape feature of the first lung nodule region and the metabolic feature of the second lung nodule region, and classifying the first lung nodule region and the second lung nodule region according to the shape feature and the metabolic feature by using an SVM (support vector machine) classifier to determine a final lung nodule region in the CT image and the PET image. By utilizing the technical scheme disclosed by the application, the accuracy of the diagnosis result of a doctor can be improved.
Description
Technical Field
The present application relates to the field of medical image data processing technologies, and in particular, to a method and an apparatus for detecting lung nodules, and a computer storage medium.
Background
The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.
Lung cancer is one of the most rapidly growing malignancies and has posed a great threat to human health and life. Compared with other cancers, the biological characteristics of lung cancer are very complex, no obvious symptoms exist in early stage, most of lung cancer is found to be in middle and late stage, and the treatment cost is high and the effect is poor. Therefore, early screening appears to be of paramount importance. Early lung cancer generally manifests as malignant lung nodules in the lungs, and thus, early screening generally begins with the detection of lung nodules.
Early detection of lung nodules was dominated by electron Computed Tomography (CT) scanning, and in recent years Positron Emission Tomography (PET) has also been used as a screening tool for cancer. PET is an imaging device that reflects the genetic, molecular, metabolic and functional status of lesions. The radioactive nuclide with positron is used for marking human body compounds or metabolites, such as glucose, protein, nucleic acid, fatty acid, receptor ligand and water and other human body metabolites, after the radioactive nuclide is injected into the body of a detected person, the positive nuclide is associated with the marked object to participate in the cell metabolic activity of human body tissues and organs in the physiological and pathological processes, and is aggregated and distributed again, and then the external detector PET is used for collecting, so that the information of the molecular level of the human body metabolic activity is obtained, and the biological metabolic information of diseases is provided for clinic.
PET/CT integrated machine, namely positron emission tomography and electron computer tomography system, organically integrates PET and CT, and complements the advantages of the two. The system is used for scanning the whole body of a patient quickly once, a CT anatomical image and a PET functional metabolic image can be obtained simultaneously, biological metabolic information of a focus and accurate anatomical positioning and structure can be presented to a doctor simultaneously through the two images, and therefore the doctor can judge diseases earlier and faster. Therefore, the combination of PET and CT can provide great help for doctors to discover lung cancer in early stage.
However, this is accompanied by a large number of PET/CT images, which greatly increases the workload of the physician. Moreover, the pulmonary nodules vary widely and vary in size and shape, and are more prone to missed and misdiagnosis if observed only by the naked eye of a doctor.
Disclosure of Invention
An embodiment of the present application provides a pulmonary nodule detection method, an apparatus and a computer storage medium, so as to solve at least one problem in the prior art.
In order to solve the above technical problem, an embodiment of the present application provides a pulmonary nodule detection method, which may include:
performing preliminary nodule detection on the acquired CT image and PET image of the target lung based on a Faster R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image;
performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule candidate region in the first lung nodule candidate region and a second lung nodule candidate region in the second lung nodule candidate region;
and acquiring the shape feature of the first lung nodule region and the metabolic feature of the second lung nodule region, and classifying the first lung nodule region and the second lung nodule region according to the shape feature and the metabolic feature by utilizing an SVM (support vector machine) classifier so as to determine a final lung nodule region in the CT image and the PET image.
Optionally, the step of performing preliminary nodule detection on the CT image and the PET image of the acquired target lung based on the Faster R-CNN model comprises:
extracting lung nodule characteristics in the CT image and the PET image by using a characteristic extraction network;
determining a plurality of potential lung nodule regions in the CT image and the PET image according to the extracted lung nodule characteristics by using a region suggestion network;
classifying the determined plurality of potential lung nodule regions using a classification regression network and performing bounding box regression using a loss function to determine the first and second candidate lung nodule regions of the plurality of potential lung nodule regions.
Optionally, the step of extracting the lung nodule features in the CT image and the PET image by using a feature extraction network includes:
performing convolution processing on the CT image and the PET image by using VGG-16 containing a plurality of convolution layers to obtain a convolution characteristic map;
and performing deconvolution processing on the obtained convolution feature map by using a deconvolution layer to extract the lung nodule feature.
Optionally, the lung nodule features comprise shape features in the CT image and metabolic features in the PET image.
Optionally, before the classifying the determined plurality of potential lung nodule regions by using a classification regression network and performing bounding box regression by using a loss function, the method further comprises:
pooling the plurality of potential lung nodule regions with a pooling layer of interest to equalize the size of the plurality of potential lung nodule regions.
Optionally, the first lung nodule region candidate includes a lung nodule region of a CT axial image and a lung nodule region of a CT sagittal image, and the second lung nodule region candidate includes an initial lung nodule region of the PET image.
Optionally, the step of performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model comprises:
performing feature extraction on the first lung nodule candidate region and the second lung nodule candidate region by using a convolutional layer;
performing pooling processing on the first lung nodule candidate region and the second lung nodule candidate region after feature extraction by using a pooling layer;
and performing full connection processing on the first lung nodule candidate region and the second lung nodule candidate region which are processed by the pooling layer by using a full connection layer to obtain the judgment probability of the lung nodule region, and determining the first lung nodule region and the second lung nodule region according to the obtained judgment probability.
An embodiment of the present application further provides a pulmonary nodule detection apparatus, which may include:
a preliminary detection unit configured to perform preliminary nodule detection on the acquired CT image and PET image of the target lung based on a fast R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image;
an identification unit configured to perform false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule region in the first lung nodule candidate region and a second lung nodule region in the second lung nodule candidate region;
a determination unit configured to obtain a shape feature of the first lung nodule region and a metabolic feature of the second lung nodule region, and classify the first lung nodule region and the second lung nodule region according to the shape feature and the metabolic feature using an SVM classifier to determine a final lung nodule region in the CT image and the PET image.
Optionally, the preliminary detection unit includes:
a feature extraction subunit configured to extract lung nodule features in the CT image and the PET image using a feature extraction network;
a determining subunit configured to determine a plurality of potential lung nodule regions in the CT image and the PET image from the extracted lung nodule features using a region suggestion network;
a classification regression subunit configured to classify the determined plurality of potential lung nodule regions using a classification regression network and perform bounding box regression using a loss function to determine the first and second lung nodule candidate regions of the plurality of potential lung nodule regions.
Embodiments of the present application further provide a computer storage medium having program instructions stored thereon, where the program instructions, when executed, implement the following functions:
performing preliminary nodule detection on the acquired CT image and PET image of the target lung based on a Faster R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image;
performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule region of the first lung nodule candidate region and a second lung nodule region of the second lung nodule candidate region;
and acquiring the shape feature of the first lung nodule region and the metabolic feature of the second lung nodule region, and classifying the first lung nodule region and the second lung nodule region according to the shape feature and the metabolic feature by utilizing an SVM (support vector machine) classifier to determine a final lung nodule region in the CT image and the PET image.
According to the technical scheme provided by the embodiment of the application, preliminary nodule detection is carried out on the CT image and the PET image of the target lung based on the Faster R-CNN model so as to determine a first candidate lung nodule region in the CT image and a second candidate lung nodule region in the PET image; performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule region in the first lung nodule candidate region and a second lung nodule region in the second lung nodule candidate region; and classifying the first lung nodule region and the second lung nodule region by using an SVM classifier according to the acquired shape characteristics of the first lung nodule region and the acquired metabolic characteristics of the second lung nodule region to determine the final lung nodule region in the CT image and the PET image, so that the film reading time of a doctor can be greatly shortened, and the accuracy of a diagnosis result can be improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the description below are only some embodiments described in the present application, and for those skilled in the art, other drawings may be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart of a pulmonary nodule detection method provided in an embodiment of the present application;
FIG. 2 is a schematic structural diagram of the Faster R-CNN model;
FIG. 3 is a graph illustrating the effect of an example of the application of the lung nodule detection method provided in the embodiment of the present application;
fig. 4 is a schematic structural diagram of a pulmonary nodule detection apparatus according to an embodiment of the present disclosure.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only used for explaining a part of the embodiments of the present application, but not all embodiments, and are not intended to limit the scope of the present application or the claims. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making creative efforts shall fall within the protection scope of the present application.
It will be understood that when an element is referred to as being "disposed on" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected/coupled" to another element, it can be directly connected/coupled to the other element or intervening elements may also be present. The term "connected/coupled" as used herein may include electrical and/or mechanical physical connections/couplings. The term "comprises/comprising" as used herein refers to the presence of features, steps or elements, but does not preclude the presence or addition of one or more other features, steps or elements. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.
In addition, in the description of the present application, the terms "first", "second", "third", and the like are used for descriptive purposes only and to distinguish similar objects, and there is no order of precedence between the two, and no indication or implication of relative importance is to be inferred.
The lung nodule detection method, apparatus and computer storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.
As shown in fig. 1, an embodiment of the present application provides a pulmonary nodule detection method, which may include the following steps:
s1: and performing primary lung nodule detection on the acquired CT image and the PET image of the target lung based on a Faster R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image.
The CT image and the PET image may be images obtained by scanning lungs of a living body (e.g., a human) using a PET/CT scanner. Among them, the CT image mainly represents the anatomical features of the subject, and the PET image mainly represents the functional metabolic features of the subject.
As shown in FIG. 2, the Faster R-CNN model is a neural network model for target detection, which may generally include a feature extraction network, a region suggestion network (RPN), and a classification regression network. The feature extraction network may be used to extract lung nodule features in CT images and PET images, and may include multiple convolutional layers, e.g., VGG-16 (i.e., a network with a depth of 16) containing multiple convolutional layers, among others. Each convolutional layer generates abstract description by using network information output by the convolutional layer of the previous layer, for example, the convolutional layer of the first layer can be used for learning edge information of an input image, and the convolutional layer of the second layer can be used for learning pattern information in the edge so as to learn more complex shape and other information. In addition, the feature extraction network may further include a deconvolution layer connected to the last convolution layer, which may improve the accuracy of the lung nodule detection result. The kernel size, stride size, pad size, and number of kernels of the deconvolution layer may be 4, and 512, respectively. The region suggestion network may be used to determine a plurality of potential lung nodule regions in the CT image and the PET image based on the shape features and the metabolic features extracted by the feature extraction network. The classification regression network may be configured to classify a plurality of potential lung nodule regions determined by the region suggestion network and perform frame regression by using a loss function, so as to determine candidate lung nodule regions in the potential lung nodule regions, and meanwhile, the classification regression network may further perform fine adjustment on positions of the candidate lung nodule regions. Additionally, the Faster R-CNN model may also include a pooling of interest (RoI) layer, which may be used to pool multiple potential lung nodule regions to make them the same size prior to performing classification regression on the potential lung nodule regions.
After acquiring the CT image and the PET image of the target lung, preliminary lung nodule detection may be performed on the acquired two images based on the fast R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image. Specifically, the method comprises the following steps:
first, after receiving the CT image and the PET image, the feature extraction network may be used to extract lung nodule features in the two images, such as shape features in the CT image (e.g., a volume, a surface region, and cross-sectional regions on different dimensional planes (e.g., XY, YZ, and XZ), contour pixels, and CT values, etc.) and metabolic features in the PET image (e.g., Standardized Uptake Values (SUV) of the lung nodule, a volume, a surface region, cross-sectional regions on different dimensional planes, etc.). Specifically, the CT image and the PET image may be subjected to convolution calculation using VGG-16 including 5 convolution layers to obtain a convolution feature map deeper but smaller in spatial resolution than the received CT image and PET image, and then the obtained convolution feature map may be subjected to deconvolution calculation using a deconvolution layer to extract lung nodule features in the CT image and the PET image. By mapping the convolution characteristic map to the original image space by using the deconvolution layer, the influence of background factors on the extraction of candidate lung nodules can be reduced, and the accuracy of the detection result can be improved.
Then, after extracting the lung nodule features in the CT image and the PET image, a plurality of potential lung nodule regions in the CT image and the PET image can be determined according to the extracted lung nodule features by using the region suggestion network. Specifically, the RPN may determine a series of good target candidate regions (nanoposes) using all bounding boxes or Anchors (Anchors), and then may determine a plurality of potential lung nodule regions in the CT image and the PET image, which may include regions without confidence scores and suspected lung nodule regions, according to the determined coordinates of the target candidate regions and their corresponding target confidence scores. The target confidence score may be derived by utilizing a fully connected layer.
Finally, the determined plurality of potential lung nodule regions may be classified by a classification regression network and bounding box regression may be performed by a loss function to determine a first lung nodule candidate region and a second lung nodule candidate region of the potential lung nodule regions. Specifically, first, the convolution feature map of multiple potential lung nodule regions may be cropped, and then each cropped region may be scaled to a fixed size (e.g., 14x14) using a bilinear interpolation algorithm; maximum pooling may then be performed using a convolution kernel (e.g., of size 2x2) to obtain a feature map for each potential lung nodule region; next, features may be extracted from the obtained feature map to classify a plurality of potential lung nodule regions, where the classification may include a lung nodule target class and a background class, and finally, a frame regression prediction may be performed by using a loss function to obtain center coordinates, widths, and heights of regions where the two classes are located, so as to determine a first lung nodule candidate region and a second lung nodule candidate region. The first lung nodule candidate region may include a lung nodule region of a CT axial image (i.e., a CT image obtained by scanning a subject in a transverse direction parallel to a ground plane, which may also be referred to as a CT transverse scan image) and a lung nodule region of a CT sagittal image (i.e., a CT image obtained by scanning a subject in an anterior-posterior direction), and the second lung nodule candidate region may include an initial lung nodule region in which a lung nodule may be present in a PET image.
In addition, before determining the first lung nodule candidate region and the second lung nodule candidate region of the plurality of potential lung nodule regions, the potential lung nodule regions may be pooled by using the RoI pooling layer to make the potential lung nodule regions have the same size, if necessary.
S2: and performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on the convolutional neural network model to determine a first lung nodule candidate region in the first lung nodule candidate region and a second lung nodule candidate region in the second lung nodule candidate region.
False positive regions may occur in CT and PET images taken due to environmental, operational, and/or patient-specific factors. For example, for lung nodules, false positive areas occur primarily at the stenotic bronchi and blood vessels in the lung. The occurrence of false positive regions may affect the final diagnosis result, so in order to improve the accuracy of the diagnosis result, the false positive regions in the first lung nodule candidate region and the second lung nodule candidate region need to be removed.
The convolutional neural network model may generally include convolutional layers, pooled layers, and fully-connected layers that are interconnected, e.g., may include three convolutional layers, three pooled layers, and two fully-connected layers. Wherein the convolutional layer may include a plurality of filters and convolutional kernels, and the fully-connected layer may include a plurality of layers of perceptrons.
After determining the first lung nodule candidate region in the CT image and the second lung nodule candidate region in the PET image, false positive region identification may be performed on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine the first lung nodule candidate region in the first lung nodule candidate region and the second lung nodule candidate region in the second lung nodule candidate region. Specifically, feature extraction may be performed on the first lung nodule candidate region and the second lung nodule candidate region by using the convolutional layer; then, overlapping sampling can be performed on the first lung nodule candidate region and the second lung nodule candidate region after feature extraction by using a pooling layer, so that feature selection and information filtering of the first lung nodule candidate region and the second lung nodule candidate region are realized; then, the first lung nodule candidate region and the second lung nodule candidate region after the pooling layer processing may be subjected to full-connection processing by using a full-connection layer to obtain a determination probability of the lung nodule candidate region, so that a first lung nodule candidate region in the first lung nodule candidate region and a second lung nodule candidate region in the second lung nodule candidate region may be determined according to the obtained determination probability, that is, non-lung nodule candidate regions in the first lung nodule candidate region and the second lung nodule candidate region are removed.
For example, for a 3-channel color image with 32x32 pixels (width x height) of the input CT image and PET image, feature extraction can be performed on the convolutional layer by using 32 filters and a kernel of 5x5x3, and finally a feature mapping matrix of 32x32x32 pixels can be obtained; the pooling layer performs pooling on the feature mapping matrix, reduces the feature mapping matrix to 16x16x32, and finally outputs a matrix with the size of 8 x 64; the fully connected layer may classify the matrix output by the pooling layer using softmax regression, i.e., classify the first lung nodule candidate region and the second lung nodule candidate region into a lung nodule region and a non-lung nodule region, and output the determination probabilities of the two regions, so that the first lung nodule candidate region in the first lung nodule candidate region and the second lung nodule candidate region in the second lung nodule candidate region may be determined according to the output determination probabilities.
S3: shape features of the first lung nodule region and metabolic features of the second lung nodule region are acquired, and the first lung nodule region and the second lung nodule region are classified according to the acquired shape features and metabolic features by an SVM classifier to determine a final lung nodule region in the CT image and the PET image.
A Support Vector Machine (SVM) classifier is a two-class classification model, and a basic model of the classifier is defined as a linear classifier with the largest interval on a feature space.
After determining the first lung nodule region of the first lung nodule region candidate and the second lung nodule region of the second lung nodule region candidate, shape features of the first lung nodule region candidate, such as a volume, a surface region, a cross-sectional region on three planes (e.g., XY, XZ, YZ), fusion, contour pixels, and CT values (a maximum value, a center value, and a standard deviation of the first lung nodule region candidate) may be obtained from the CT image, and metabolic features of the second lung nodule region candidate, such as a normalized uptake (SUV) of a center of the second lung nodule region candidate, a maximum value and an average value of SUV in the second lung nodule region candidate, a volume, a surface region, a cross-sectional region on three planes, and the like may also be obtained from the PET image. The SVM classifier may then classify the first lung nodule region and the second lung nodule region based on the acquired shape features and metabolic features. In particular, the SVM classifier may train shape features of a first lung nodule region and metabolic features in a second lung nodule region, and in conjunction with changes in some features (e.g., for the second lung nodule region in a PET image, changes in its shape may typically be observed in the corresponding CT image, while for the first lung nodule region in a CT image, many features in the second lung nodule region may be set to 0 if there are no high-absorption regions in the corresponding PET image), implement classification of the first lung nodule region and the second lung nodule region to determine the final lung nodule region.
The specific process of how the SVM classifier classifies the inputted features can refer to the related description in the prior art, and is not described in detail herein.
By observing the final lung nodule region in both the PET and CT images, the physician can quickly make his diagnosis.
As can be seen from the above description, in the embodiment of the present application, preliminary nodule detection is performed on the acquired CT image and PET image of the target lung based on the fast R-CNN model, so as to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image; performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule candidate region in the first lung nodule candidate region and a second lung nodule candidate region in the second lung nodule candidate region; and acquiring the shape feature of the first lung nodule region and the metabolic feature of the second lung nodule region, and classifying the first lung nodule region and the second lung nodule region according to the acquired shape feature and metabolic feature by using an SVM classifier to determine the final lung nodule region in the CT image and the PET image, so that the film reading time of a doctor can be greatly shortened, and the accuracy of a diagnosis result of the doctor is improved.
The beneficial effects of the lung nodule detection method provided by the embodiment of the present application are illustrated in the following specific examples. As shown in fig. 3(a) -3(d), wherein fig. 3(a) and 3(b) are the original PET image and CT image, respectively; fig. 3(c) and 3(d) are a PET image and a CT image respectively output after being processed by the lung nodule detection method provided by the embodiment of the present application, wherein the regions a and B are lung nodule regions. As can be seen from these figures, the lung nodule region that is not detected in the CT image is detected in the PET image, and the lung nodule region that is not apparent in the PET image is clearly detected in the CT image. Therefore, the lung nodule detection method provided by the embodiment of the application can achieve the effect of assisting a doctor in diagnosing lung nodules, reduces missed diagnosis or misdiagnosis and improves diagnosis efficiency.
An embodiment of the present application further provides a pulmonary nodule detection apparatus, as shown in fig. 4, the pulmonary nodule detection apparatus may include:
a preliminary detection unit 310, which may be configured to perform preliminary nodule detection on the CT image and the PET image of the acquired target lung based on the Faster R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image;
an identifying unit 320, which may be configured to perform false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule region in the first lung nodule candidate region and a second lung nodule region in the second lung nodule candidate region; and
a determination unit 330, which may be configured to obtain a shape feature of the first lung nodule region and a metabolic feature of the second lung nodule region, and classify the first lung nodule region and the second lung nodule region according to the obtained shape feature and metabolic feature using an SVM classifier to determine a final lung nodule region in the CT image and the PET image.
In an embodiment, the preliminary detection unit 310 may include (not shown in the figure): a feature extraction subunit, which can be configured to extract lung nodule features in the CT image and the PET image by using a feature extraction network; a determining subunit, which may be configured to determine a plurality of potential lung nodule regions in the CT image and the PET image from the extracted lung nodule features using a region suggestion network; a classification regression subunit configured to classify the determined plurality of potential lung nodule regions using a classification regression network and perform bounding box regression using a loss function to determine a first lung nodule candidate region and the second lung nodule candidate region of the plurality of potential lung nodule regions.
Regarding the detailed description of the preliminary detection unit 310, the identification unit 320 and the classification unit 330, reference may be made to the description of steps S1 to S3 in the above method embodiment, which is not repeated here.
The device may be a server, an electronic device, or the like, or may be any device capable of performing data processing based on a neural network, and is not limited thereto. In addition, the functions realized by the units in the above device may be executed by a processor in the computer according to program instructions stored in the memory.
Embodiments of the present application further provide a computer storage medium having program instructions stored thereon, where the program instructions, when executed, may implement the following functions: performing preliminary nodule detection on the acquired CT image and PET image of the target lung based on a Faster R-CNN model to determine a first candidate lung nodule region in the CT image and a second candidate lung nodule region in the PET image; performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule candidate region in the first lung nodule candidate region and a second lung nodule candidate region in the second lung nodule candidate region; shape features of the first lung nodule region and metabolic features of the second lung nodule region are acquired, and the first lung nodule region and the second lung nodule region are classified according to the acquired shape features and metabolic features by an SVM classifier to determine a final lung nodule region in the CT image and the PET image.
The program instructions may be executed by a processor or other processing device.
The apparatuses, units and the like described in the foregoing embodiments may be specifically implemented by computer chips and/or entities, or implemented by products with certain functions. For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functions of the units may be implemented in the same or multiple computer chips when implementing the present application.
Although the present application provides method steps as described in the above embodiments or flowcharts, additional or fewer steps may be included in the methods, based on routine or non-inventive efforts. In steps where there is no logically necessary causal relationship, the order of execution of these steps is not limited to the order of execution provided by the embodiments of the present application.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments.
The embodiments described above are described in order to enable those skilled in the art to understand and use the present application. It will be readily apparent to those skilled in the art that various modifications to these embodiments may be made, and the generic principles described herein may be applied to other embodiments without the use of the inventive faculty. Therefore, the present application is not limited to the above embodiments, and those skilled in the art should make improvements and modifications within the scope of the present application without departing from the scope of the present application.
Claims (9)
1. A pulmonary nodule detection method, comprising:
performing preliminary nodule detection on the acquired CT image and PET image of the target lung based on a Faster R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image;
performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule region in the first lung nodule candidate region and a second lung nodule region in the second lung nodule candidate region, which specifically includes: performing feature extraction on the first lung nodule candidate region and the second lung nodule candidate region by using a convolutional layer; performing pooling processing on the first lung nodule candidate region and the second lung nodule candidate region after feature extraction by using a pooling layer; performing full-connection processing on the first lung nodule candidate region and the second lung nodule candidate region processed by the pooling layer by using a full-connection layer to obtain a judgment probability of the lung nodule region, and determining the first lung nodule region and the second lung nodule region according to the obtained judgment probability;
and obtaining shape features of the first lung nodule region and metabolic features of the second lung nodule region, and classifying the first lung nodule region and the second lung nodule region according to the shape features and the metabolic features by using an SVM (support vector machine) classifier to determine a final lung nodule region in the CT image and the PET image, wherein the metabolic features comprise standardized uptake values, volumes, surface regions or cross-sectional regions on different dimensional planes of lung nodules.
2. The method according to claim 1, wherein the step of performing preliminary nodule detection on the acquired CT and PET images of the target lung based on the Faster R-CNN model comprises:
extracting lung nodule characteristics in the CT image and the PET image by using a characteristic extraction network;
determining a plurality of potential lung nodule regions in the CT image and the PET image according to the extracted lung nodule characteristics by using a region suggestion network;
classifying the determined plurality of potential lung nodule regions using a classification regression network and performing bounding box regression using a loss function to determine the first and second candidate lung nodule regions of the plurality of potential lung nodule regions.
3. The method according to claim 2, wherein the step of extracting lung nodule features in the CT image and the PET image by using a feature extraction network comprises:
performing convolution processing on the CT image and the PET image by using VGG-16 containing a plurality of convolution layers to obtain a convolution characteristic map;
and performing deconvolution processing on the obtained convolution feature map by using a deconvolution layer to extract the lung nodule feature.
4. The method of claim 3, wherein the lung nodule features include shape features in the CT image and metabolic features in the PET image.
5. The method of claim 2, wherein prior to classifying the determined plurality of potential lung nodule regions using a classification regression network and performing bounding box regression using a loss function, the method further comprises:
pooling the plurality of potential lung nodule regions with a pooling layer of interest to equalize the size of the plurality of potential lung nodule regions.
6. The method of claim 2 or 5, wherein the first lung nodule region candidate comprises a lung nodule region of a CT axial image and a lung nodule region of a CT sagittal image, and the second lung nodule region candidate comprises an initial lung nodule region of the PET image.
7. A pulmonary nodule detection apparatus, comprising:
a preliminary detection unit configured to perform preliminary nodule detection on the acquired CT image and PET image of the target lung based on the Faster R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image;
an identifying unit configured to perform false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule candidate region in the first lung nodule candidate region and a second lung nodule candidate region in the second lung nodule candidate region, which specifically includes: performing feature extraction on the first lung nodule candidate region and the second lung nodule candidate region by using a convolutional layer; performing pooling processing on the first lung nodule candidate region and the second lung nodule candidate region after feature extraction by using a pooling layer; performing full-connection processing on the first lung nodule candidate region and the second lung nodule candidate region processed by the pooling layer by using a full-connection layer to obtain a judgment probability of the lung nodule region, and determining the first lung nodule region and the second lung nodule region according to the obtained judgment probability;
a determination unit configured to obtain a shape feature of the first lung nodule region and a metabolic feature of the second lung nodule region, and classify the first lung nodule region and the second lung nodule region according to the shape feature and the metabolic feature by using an SVM classifier to determine a final lung nodule region in the CT image and the PET image, wherein the metabolic feature comprises a standardized uptake value, a volume, a surface region or a cross-sectional region on different dimensional planes of a lung nodule.
8. The pulmonary nodule detection apparatus of claim 7, wherein the preliminary detection unit comprises:
a feature extraction subunit configured to extract lung nodule features in the CT image and the PET image using a feature extraction network;
a determining subunit configured to determine a plurality of potential lung nodule regions in the CT image and the PET image from the extracted lung nodule features using a region suggestion network;
a classification regression subunit configured to classify the determined plurality of potential lung nodule regions using a classification regression network and perform bounding box regression using a loss function to determine the first and second lung nodule candidate regions of the plurality of potential lung nodule regions.
9. A computer storage medium having stored thereon program instructions that, when executed, perform the functions of:
performing preliminary nodule detection on the acquired CT image and PET image of the target lung based on a Faster R-CNN model to determine a first lung nodule candidate region in the CT image and a second lung nodule candidate region in the PET image;
performing false positive region identification on the first lung nodule candidate region and the second lung nodule candidate region based on a convolutional neural network model to determine a first lung nodule region in the first lung nodule candidate region and a second lung nodule region in the second lung nodule candidate region, which specifically includes: performing feature extraction on the first lung nodule candidate region and the second lung nodule candidate region by using a convolutional layer; performing pooling processing on the first candidate lung nodule area and the second candidate lung nodule area after feature extraction by using a pooling layer; performing full-connection processing on the first lung nodule candidate region and the second lung nodule candidate region processed by the pooling layer by using a full-connection layer to obtain a judgment probability of the lung nodule region, and determining the first lung nodule region and the second lung nodule region according to the obtained judgment probability;
and obtaining shape features of the first lung nodule region and metabolic features of the second lung nodule region, and classifying the first lung nodule region and the second lung nodule region according to the shape features and the metabolic features by using an SVM (support vector machine) classifier to determine a final lung nodule region in the CT image and the PET image, wherein the metabolic features comprise standardized uptake values, volumes, surface regions or cross-sectional regions on different dimensional planes of lung nodules.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911095432.4A CN110838114B (en) | 2019-11-11 | 2019-11-11 | Pulmonary nodule detection method, device and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911095432.4A CN110838114B (en) | 2019-11-11 | 2019-11-11 | Pulmonary nodule detection method, device and computer storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110838114A CN110838114A (en) | 2020-02-25 |
| CN110838114B true CN110838114B (en) | 2022-07-22 |
Family
ID=69574785
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911095432.4A Active CN110838114B (en) | 2019-11-11 | 2019-11-11 | Pulmonary nodule detection method, device and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110838114B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110969622B (en) * | 2020-02-28 | 2020-07-24 | 南京安科医疗科技有限公司 | Image processing method and system for assisting pneumonia diagnosis |
| CN111768845B (en) * | 2020-06-30 | 2023-08-11 | 重庆大学 | Pulmonary nodule auxiliary detection method based on optimal multi-scale perception |
| CN113077434B (en) * | 2021-03-30 | 2023-01-24 | 零氪智慧医疗科技(天津)有限公司 | Method, device and storage medium for lung cancer identification based on multi-modal information |
| CN113506289B (en) * | 2021-07-28 | 2024-03-29 | 中山仰视科技有限公司 | Method for classifying false positives of lung nodules by using double-flow network |
| CN113822861A (en) * | 2021-09-07 | 2021-12-21 | 中山大学中山眼科中心 | Method and device for judging eye surface swelling |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106530296A (en) * | 2016-11-07 | 2017-03-22 | 首都医科大学 | Lung detection method and device based on PET/CT image features |
| CN107230206A (en) * | 2017-06-02 | 2017-10-03 | 太原理工大学 | A kind of 3D Lung neoplasm dividing methods of the super voxel sequence lung images based on multi-modal data |
| CN108010021A (en) * | 2017-11-30 | 2018-05-08 | 上海联影医疗科技有限公司 | A kind of magic magiscan and method |
| CN108986073A (en) * | 2018-06-04 | 2018-12-11 | 东南大学 | A kind of CT image pulmonary nodule detection method based on improved Faster R-CNN frame |
| CN110223352A (en) * | 2019-06-14 | 2019-09-10 | 浙江明峰智能医疗科技有限公司 | A kind of medical image scanning automatic positioning method based on deep learning |
-
2019
- 2019-11-11 CN CN201911095432.4A patent/CN110838114B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106530296A (en) * | 2016-11-07 | 2017-03-22 | 首都医科大学 | Lung detection method and device based on PET/CT image features |
| CN107230206A (en) * | 2017-06-02 | 2017-10-03 | 太原理工大学 | A kind of 3D Lung neoplasm dividing methods of the super voxel sequence lung images based on multi-modal data |
| CN108010021A (en) * | 2017-11-30 | 2018-05-08 | 上海联影医疗科技有限公司 | A kind of magic magiscan and method |
| CN108986073A (en) * | 2018-06-04 | 2018-12-11 | 东南大学 | A kind of CT image pulmonary nodule detection method based on improved Faster R-CNN frame |
| CN110223352A (en) * | 2019-06-14 | 2019-09-10 | 浙江明峰智能医疗科技有限公司 | A kind of medical image scanning automatic positioning method based on deep learning |
Non-Patent Citations (1)
| Title |
|---|
| 多模态融合的深度学习脑肿瘤检测方法;姚红革 等;《光子学报》;20190731;第48卷(第7期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110838114A (en) | 2020-02-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110838114B (en) | Pulmonary nodule detection method, device and computer storage medium | |
| Halder et al. | Lung nodule detection from feature engineering to deep learning in thoracic CT images: a comprehensive review | |
| CN106682435B (en) | System and method for automatically detecting lesion in medical image through multi-model fusion | |
| JP5864542B2 (en) | Image data processing method, system, and program for detecting image abnormality | |
| EP1719080B1 (en) | A system and method for toboggan based object segmentation using divergent gradient field response in images | |
| CN109493325A (en) | Tumor Heterogeneity analysis system based on CT images | |
| Hussain et al. | Cascaded regression neural nets for kidney localization and segmentation-free volume estimation | |
| JP2006034585A (en) | Picture display device and picture display method, and program thereof | |
| Dutande et al. | Deep residual separable convolutional neural network for lung tumor segmentation | |
| JP2018175217A (en) | Image processing apparatus, method and program | |
| WO2022110525A1 (en) | Comprehensive detection apparatus and method for cancerous region | |
| CN112862756A (en) | Method for identifying pathological change type and gene mutation in thyroid tumor pathological image | |
| US20230005140A1 (en) | Automated detection of tumors based on image processing | |
| Alaskar et al. | Deep learning approaches for automatic localization in medical images | |
| Xiao et al. | A cascade and heterogeneous neural network for CT pulmonary nodule detection and its evaluation on both phantom and patient data | |
| Peña-Solórzano et al. | Findings from machine learning in clinical medical imaging applications–Lessons for translation to the forensic setting | |
| Nurmaini et al. | An improved semantic segmentation with region proposal network for cardiac defect interpretation | |
| Almutairi et al. | An efficient USE-Net deep learning model for cancer detection | |
| CN116129184A (en) | Multi-phase focus classification method, device, equipment and readable storage medium | |
| Kanawade et al. | A Deep Learning Approach for Pneumonia Detection from X− ray Images | |
| Dulhare et al. | Detection and classification of rheumatoid nodule using deep learning models | |
| WO2017089953A1 (en) | Process for processing medical images of a face for recognition of facial dysmorphisms | |
| Al-Saeed et al. | Liver segmentation using fast-generalized fuzzy C-means (FG-FCM) from CT scans | |
| CN112884759B (en) | Method and related device for detecting metastasis state of axillary lymph nodes of breast cancer | |
| Xu et al. | Computer-aided detection of acinar shadows in chest radiographs |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |