CN115222651A - Pulmonary nodule detection system based on improved Mask R-CNN - Google Patents

Pulmonary nodule detection system based on improved Mask R-CNN Download PDF

Info

Publication number
CN115222651A
CN115222651A CN202210462990.5A CN202210462990A CN115222651A CN 115222651 A CN115222651 A CN 115222651A CN 202210462990 A CN202210462990 A CN 202210462990A CN 115222651 A CN115222651 A CN 115222651A
Authority
CN
China
Prior art keywords
lung
image
detection
region
nodule
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210462990.5A
Other languages
Chinese (zh)
Inventor
孟庆松
刘野鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harbin University of Science and Technology
Original Assignee
Harbin University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harbin University of Science and Technology filed Critical Harbin University of Science and Technology
Priority to CN202210462990.5A priority Critical patent/CN115222651A/en
Publication of CN115222651A publication Critical patent/CN115222651A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/20Image enhancement or restoration by the use of local operators
    • G06T5/30Erosion or dilatation, e.g. thinning
    • G06T5/90
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/11Region-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/12Edge-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/187Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10072Tomographic images
    • G06T2207/10081Computed x-ray tomography [CT]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20036Morphological image processing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30061Lung
    • G06T2207/30064Lung nodule

Abstract

The invention discloses a pulmonary nodule detection system based on improved Mask R-CNN, relating to the technical field of pulmonary nodule detection; the method comprises the following steps: the method comprises the following steps: segmenting lung parenchyma; step two: a candidate nodule segmentation and detection algorithm; step three: improving the network for small targets; step four: a false positive reduction algorithm; step five: unifying the frames; the invention improves the detection sensitivity, has good detection effect and can realize the segmentation and detection of acquisition; the accuracy of the detection result is improved, and the doctor can visually see the result.

Description

Pulmonary nodule detection system based on improved Mask R-CNN
Technical Field
The invention belongs to the technical field of pulmonary nodule detection, and particularly relates to a pulmonary nodule detection system based on improved Mask R-CNN.
Background
The main manifestation of lung cancer is the rapid increase in the rate of growth of abnormal cells, which form a mass in the lung called a lung nodule. Lung nodules, an early manifestation of lung cancer, are associated with cancer in a more complex manner. The presence of lung nodules does not necessarily mean cancer, but rather, accurate and careful analysis of each suspect nodule is required to better provide an effective method for early diagnosis of lung cancer.
In a conventional lung diagnosis process, a radiologist observes whether a nodule, a size of the nodule, a shape of the nodule, and the like exist in a lung of a patient by means of medical images such as CT or MRI, which are shot by a professional instrument, and provides a basis for diagnosis of an illness state of the patient based on obtained information. However, with the development of medical imaging technology, the increase of the scanning speed and the imaging quantity of lung CT images makes the manual detection time-consuming and serious, many of the images are similar, and the monotonous work is repeated all the time, which is easy to cause the fatigue of doctors. Meanwhile, the pulmonary nodules in the CT image may have the characteristics of unobvious performance or various changes and the like, so that the detection has certain difficulty, doctors are required to have rich experience knowledge, and the detection result and the later treatment are influenced by missed diagnosis and misdiagnosis easily only through the visual observation of the doctors, so that the pulmonary nodule detection system is required to realize the detection.
Disclosure of Invention
To solve the problems in the background art; the invention aims to provide a pulmonary nodule detection system based on improved Mask R-CNN.
The invention relates to a pulmonary nodule detection system based on improved Mask R-CNN, which comprises the following steps:
the method comprises the following steps: and (3) lung parenchyma segmentation:
firstly, binaryzation is carried out on an original CT image, image pixels are marked, the whole image is converted into a gray image, and the image is separated by utilizing the brightness distribution of the image pixels to obtain a lung contour region; then, further processing the gray level image by adopting image morphology operation to eliminate the small cavity area of the lung, specifically adopting corrosion and expansion operation, wherein the corrosion operation reduces the high brightness area, and the expansion operation increases the high brightness area; then marking a lung communication region to obtain a lung mask region, and finally performing and operation on the lung mask region and the original CT image to segment a lung parenchyma part;
step two: candidate nodule segmentation and detection algorithm:
the spatial attention mechanism adopts two methods of global average pooling and maximum pooling to extract image spatial features, the extracted features are spliced, and spatial attention weight coefficients are obtained through convolutional layers with convolution kernels of 1 x 1 and sigmoid activation function operation; multiplying the obtained weight by the input feature map to finally generate a spatial attention feature; the weight calculation formula of the spatial attention is shown as (1):
Figure RE-GDA0003853153910000021
in the formula, M S (F) The weights of the spatial attention module are represented,
Figure RE-GDA0003853153910000022
representing a global average pooling operation of features over space S,
Figure RE-GDA0003853153910000023
the maximum pooling operation is performed on the features in the space S, f is the convolution operation after the two features are spliced, and a function sigma represents a sigmoid activation function;
step three: improving the network for small targets:
adding a module for calculating a loss function on the basis of the FPN, wherein the module is used for monitoring the parameter adjustment of the backbone network; simultaneously referring to UNet, calculating a loss function and outputting a semantic segmentation result related to a detection target, wherein the result is used for adjusting a feature map output by the FPN;
step four: false positive reduction algorithm:
adding a three-dimensional convolutional neural network into a Mask R-CNN network for accurately extracting lung nodule characteristics to achieve the purpose of reducing false positive lung nodules;
step five: unifying the frame:
the improvement is integrated into a unified model, then experimental verification is carried out on a pulmonary nodule data set, the influence of each module on an experimental result is studied in detail, and parameters are adjusted to achieve an expected effect.
Compared with the prior art, the invention has the beneficial effects that:
1. the sensitivity of detection is improved, the detection effect is good, and meanwhile, the segmentation and detection of collection can be realized.
2. The accuracy of the detection result is improved, and the doctor can visually see the result.
Drawings
For ease of illustration, the invention is described in detail by the following detailed description and the accompanying drawings.
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a flow chart of the spatial attention mechanism of the present invention;
Detailed Description
In order that the objects, aspects and advantages of the invention will become more apparent, the invention will be described by way of example only, and in connection with the accompanying drawings. It is to be understood that such description is merely illustrative and not intended to limit the scope of the present invention. The structure, proportion, size and the like shown in the drawings are only used for matching with the content disclosed in the specification, so that the person skilled in the art can understand and read the description, and the description is not used for limiting the limit condition of the implementation of the invention, so the method has no technical essence, and any structural modification, proportion relation change or size adjustment still falls within the range covered by the technical content disclosed by the invention without affecting the effect and the achievable purpose of the invention. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present invention.
It should be noted that, in order to avoid obscuring the present invention with unnecessary details, only the structures and/or processing steps closely related to the scheme according to the present invention are shown in the drawings, and other details not so relevant to the present invention are omitted.
The technical scheme adopted by the specific implementation mode is as follows: including small lung nodules occupying less than 32 x 32 pixels in the image. The main flow of lung nodule detection is shown in fig. 1. Aiming at the process, the main research contents of the design are as follows:
(1) Data set preprocessing and lung parenchyma segmentation: firstly, binaryzation is carried out on an original lung image, an image region is separated through distribution of pixel brightness, then a small lung cavity region is eliminated by adopting an image morphology operation method, then a connected region is marked to obtain a lung mask image, and then AND operation is carried out on the lung mask image and the original image, and finally normalization processing is carried out to obtain a segmented lung parenchyma image.
(2) And candidate nodule segmentation and detection algorithm research: and (3) taking the prior knowledge of the medical image into consideration, and finding a method for improving the Mask R-CNN so that the Mask R-CNN can better detect the lung nodule. Most medical image detection tasks have a common point: the same type of lesions all appear on a specific organ or location, i.e. they are mostly concentrated in a small area on the image. Based on this a priori knowledge, when extracting candidate frames, more and more dense candidate frames can be extracted at the positions where the probability of the occurrence of lung nodules is high, and a small amount of candidate frames, even no candidate frames, are extracted at the positions where the probability of the occurrence of the target is low. An RPN module in the Mask R-CNN is improved, so that a candidate frame is densely generated in a high probability area, and the segmentation effect of the model is improved.
(3) And small target detection algorithm research: although most small lung nodules are benign, they cannot be ignored. Due to the problems of insufficient small target detection information amount, too small area and the like, the detection effect is poor, and the FPN module in the Mask R-CNN is improved, so that the detection efficiency of the network on the small target is improved.
(4) And a false positive reduction algorithm research:
the network has higher sensitivity and can detect false positive pulmonary nodules to a certain extent. Considering tomographic properties based on lung CT images (i.e., lung CT images have three-dimensional properties), a network is found that can input three-dimensional features to reduce false positive lung nodules.
The embodiment is as follows: the protocol of this example is as follows:
(1) And (3) lung parenchyma segmentation:
the CT image read in the data set is a whole lung image, mainly including a lung and its surrounding tissues and organs, wherein tissues such as blood vessels, bronchi and bones may have a certain influence on the subsequent detection of lung nodules, so that the lung parenchyma needs to be segmented to remove other tissues and organs except the lung. Firstly, binaryzation is carried out on an original CT image, image pixels are marked, the whole image is converted into a gray image, the image is separated by utilizing the brightness distribution of the image pixels, and a lung contour region is obtained. And then further processing the gray level image by adopting image morphology operation to eliminate the small cavity region of the lung, specifically adopting corrosion and expansion operation, wherein the high brightness region is reduced by the corrosion operation, and the high brightness region is increased by the expansion operation. And marking the lung communication region to obtain a lung mask region, and finally performing AND operation on the lung mask region and the original CT image to segment the lung parenchyma part.
(2) Candidate nodule segmentation and detection algorithm research:
in lung CT, the occurrence position of a lung nodule is divided into high and low probabilities, attention is focused on the lung nodule region, interference of other irrelevant regions is inhibited, and the accuracy of the network in the aspect of feature extraction can be improved. And (4) considering adding a spatial attention mechanism, wherein the spatial attention mechanism is mainly used for establishing a model aiming at spatial information in an image, and a compression channel obtains spatial attention characteristics. The feature information of the target area on the space is highlighted, and the representation capability of local features is enhanced. The spatial attention mechanism is shown in fig. 2.
The spatial attention mechanism adopts two methods of global average pooling and maximum pooling to extract image spatial features, the extracted features are spliced, and spatial attention weight coefficients are obtained through convolutional layer with convolution kernel of 1 multiplied by 1 and sigmoid activation function operation. And multiplying the obtained weight by the input feature map to finally generate the spatial attention feature. The weight calculation formula of the spatial attention is shown as (1):
Figure RE-GDA0003853153910000061
in the formula, M S (F) The weights of the spatial attention module are represented,
Figure RE-GDA0003853153910000062
representing a global average pooling operation of features over space S,
Figure RE-GDA0003853153910000063
the maximum pooling operation is performed on the features in the space S, f is the convolution operation after the two features are spliced, and the function sigma is a sigmoid activation function.
(3) Improving the network for small targets:
by taking the idea of UNet as a reference, the FPN is considered as a coding and decoding structure. The bottom-up portion of the FPN may be referred to as the encoding portion of the codec structure, while the top-down portion may be referred to as the decoding portion. According to the design, a module for calculating a loss function is added on the basis of FPN (field programmable gate array) according to UNet, and is used for monitoring adjustment of parameters of a backbone network; and simultaneously referring to UNet, outputting a semantic segmentation result about the detection target while calculating the loss function, wherein the result is used for adjusting the feature map output by the FPN.
(4) Study of false positive reduction algorithm:
in the nodule segmentation and detection stage, all the nodule candidates need to be detected as much as possible with high sensitivity, which tends to result in high false positives. In order to detect true nodules therefrom, it is necessary to further remove false positive nodules. The design considers that a three-dimensional convolution neural network is added into a Mask R-CNN network and is used for accurately extracting lung nodule characteristics so as to achieve the purpose of reducing false positive lung nodules.
(5) Unifying the frame:
the improvement is integrated into a unified model, experimental verification is carried out on a pulmonary nodule data set, the influence of each module on an experimental result is studied in detail, and parameters are adjusted to achieve an expected effect.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.
Furthermore, it should be understood that although the present description refers to embodiments, not every embodiment may contain only a single embodiment, and such description is for clarity only, and those skilled in the art should integrate the description, and the embodiments may be combined as appropriate to form other embodiments understood by those skilled in the art.

Claims (1)

1. A pulmonary nodule detection system based on improved Mask R-CNN is characterized in that: the method comprises the following steps:
the method comprises the following steps: lung parenchyma segmentation:
firstly, binaryzation is carried out on an original CT image, image pixels are marked, the whole image is converted into a gray image, and the image is separated by utilizing the brightness distribution of the image pixels to obtain a lung contour region; then, further processing the gray level image by adopting image morphological operation to eliminate a small lung cavity region, specifically adopting corrosion and expansion operation, wherein the high brightness region is reduced by the corrosion operation, and the high brightness region is increased by the expansion operation; then marking a lung communication region to obtain a lung mask region, and finally performing and operation on the lung mask region and the original CT image to segment a lung parenchyma part;
step two: candidate nodule segmentation and detection algorithm:
the spatial attention mechanism adopts two methods of global average pooling and maximum pooling to extract image spatial features, the extracted features are spliced, and spatial attention weight coefficients are obtained through convolutional layer with convolution kernel of 1 multiplied by 1 and sigmoid activation function operation; multiplying the obtained weight by the input feature map to finally generate a spatial attention feature; the weight calculation formula of the spatial attention is shown as (1):
Figure FDA0003622663830000011
in the formula, M S (F) The weights of the spatial attention module are represented,
Figure FDA0003622663830000012
representing a global average pooling operation of features over space S,
Figure FDA0003622663830000013
the maximum pooling operation is performed on the features in the space S, f is the convolution operation after the two features are spliced, and a function sigma represents a sigmoid activation function;
step three: improving the network for small targets:
adding a module for calculating a loss function on the basis of the FPN, wherein the module is used for monitoring the parameter adjustment of the backbone network; simultaneously referring to UNet, calculating a loss function and outputting a semantic segmentation result related to a detection target, wherein the result is used for adjusting a feature map output by the FPN;
step four: false positive reduction algorithm:
adding a three-dimensional convolutional neural network into a Mask R-CNN network for accurately extracting lung nodule characteristics to achieve the purpose of reducing false positive lung nodules;
step five: unifying the frame:
the improvement is integrated into a unified model, then experimental verification is carried out on a pulmonary nodule data set, the influence of each module on an experimental result is studied in detail, and parameters are adjusted to achieve an expected effect.
CN202210462990.5A 2022-04-28 2022-04-28 Pulmonary nodule detection system based on improved Mask R-CNN Pending CN115222651A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210462990.5A CN115222651A (en) 2022-04-28 2022-04-28 Pulmonary nodule detection system based on improved Mask R-CNN

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210462990.5A CN115222651A (en) 2022-04-28 2022-04-28 Pulmonary nodule detection system based on improved Mask R-CNN

Publications (1)

Publication Number Publication Date
CN115222651A true CN115222651A (en) 2022-10-21

Family

ID=83608560

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210462990.5A Pending CN115222651A (en) 2022-04-28 2022-04-28 Pulmonary nodule detection system based on improved Mask R-CNN

Country Status (1)

Country Link
CN (1) CN115222651A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116310483A (en) * 2022-12-06 2023-06-23 河北玖嘉医药科技有限公司 Lung cancer pathology recognition and classification method based on MobileNet V2 network

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116310483A (en) * 2022-12-06 2023-06-23 河北玖嘉医药科技有限公司 Lung cancer pathology recognition and classification method based on MobileNet V2 network

Similar Documents

Publication Publication Date Title
CN109461495B (en) Medical image recognition method, model training method and server
CN109493308B (en) Medical image synthesis and classification method for generating confrontation network based on condition multi-discrimination
CN110930416B (en) MRI image prostate segmentation method based on U-shaped network
US20220092789A1 (en) Automatic pancreas ct segmentation method based on a saliency-aware densely connected dilated convolutional neural network
CN111784721B (en) Ultrasonic endoscopic image intelligent segmentation and quantification method and system based on deep learning
CN110889853A (en) Tumor segmentation method based on residual error-attention deep neural network
CN115359052B (en) Medical image enhancement method based on clustering algorithm
Tong et al. Improved U-net MALF model for lesion segmentation in breast ultrasound images
Tan et al. Analysis of segmentation of lung parenchyma based on deep learning methods
CN112001895B (en) Thyroid calcification detection device
Lan et al. Run: Residual u-net for computer-aided detection of pulmonary nodules without candidate selection
CN112258514A (en) Segmentation method of pulmonary blood vessels of CT (computed tomography) image
CN112071418B (en) Gastric cancer peritoneal metastasis prediction system and method based on enhanced CT image histology
CN114529505A (en) Breast lesion risk assessment system based on deep learning
CN115222651A (en) Pulmonary nodule detection system based on improved Mask R-CNN
Honghan et al. Rms-se-unet: A segmentation method for tumors in breast ultrasound images
CN113538363A (en) Lung medical image segmentation method and device based on improved U-Net
CN112967254A (en) Lung disease identification and detection method based on chest CT image
CN113052857A (en) Lung lesion image segmentation method based on CovSegNet
CN112862783A (en) Thyroid CT image nodule automatic diagnosis system based on neural network
CN112634308A (en) Nasopharyngeal carcinoma target area and endangered organ delineation method based on different receptive fields
CN109948706B (en) Micro-calcification cluster detection method combining deep learning and feature multi-scale fusion
CN111507918B (en) Medical image enhancement method based on deep learning technology
CN110648333B (en) Real-time segmentation system of mammary gland ultrasonic video image based on middle-intelligence theory
Gao et al. Classification of pulmonary nodules by using improved convolutional neural networks

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination