CN109523525B - Image fusion malignant lung nodule identification method, device, equipment and storage medium - Google Patents
Image fusion malignant lung nodule identification method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN109523525B CN109523525B CN201811323026.4A CN201811323026A CN109523525B CN 109523525 B CN109523525 B CN 109523525B CN 201811323026 A CN201811323026 A CN 201811323026A CN 109523525 B CN109523525 B CN 109523525B
- Authority
- CN
- China
- Prior art keywords
- layer
- image
- lung
- convolution
- size
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000004927 fusion Effects 0.000 title claims abstract description 67
- 206010056342 Pulmonary mass Diseases 0.000 title claims abstract description 59
- 230000003211 malignant effect Effects 0.000 title claims abstract description 46
- 238000000034 method Methods 0.000 title claims abstract description 30
- 210000004072 lung Anatomy 0.000 claims abstract description 101
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 31
- 238000012549 training Methods 0.000 claims abstract description 17
- 238000007781 pre-processing Methods 0.000 claims abstract description 16
- 238000002372 labelling Methods 0.000 claims abstract description 9
- 238000007499 fusion processing Methods 0.000 claims abstract description 8
- 238000011176 pooling Methods 0.000 claims description 45
- 238000004590 computer program Methods 0.000 claims description 22
- 230000005484 gravity Effects 0.000 claims description 15
- 238000001514 detection method Methods 0.000 claims description 13
- 230000006870 function Effects 0.000 claims description 9
- 238000001914 filtration Methods 0.000 claims description 8
- 230000004913 activation Effects 0.000 claims description 4
- 230000002685 pulmonary effect Effects 0.000 description 13
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 206010035664 Pneumonia Diseases 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 208000032376 Lung infection Diseases 0.000 description 1
- 206010058467 Lung neoplasm malignant Diseases 0.000 description 1
- 238000010521 absorption reaction Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 201000005202 lung cancer Diseases 0.000 description 1
- 208000020816 lung neoplasm Diseases 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000391 smoking effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000008733 trauma Effects 0.000 description 1
- 201000008827 tuberculosis Diseases 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30061—Lung
- G06T2207/30064—Lung nodule
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Radiology & Medical Imaging (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Medical Informatics (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Quality & Reliability (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Apparatus For Radiation Diagnosis (AREA)
Abstract
The invention discloses a method, a device, equipment and a storage medium for identifying malignant lung nodules by image fusion, wherein the method comprises the following steps: preprocessing at least one acquired lung CT image; performing fusion processing on each group of preprocessed lung CT images to obtain a fusion image; labeling the nodule part in the fusion image according to the label of the nodule position of each group of lung CT images; and training the constructed convolutional neural network by using the labeled fusion image so as to identify the position of the malignant lung nodule in the CT image by using the trained convolutional neural network. By using the method and the device, the malignant lung nodules in the lung CT image can be automatically and accurately identified, and the misdiagnosis probability is reduced.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for identifying malignant lung nodules by image fusion.
Background
According to the survey, there are roughly four possibilities for the appearance of malignant lung nodules: first, acute pneumonia is difficult to heal because of pulmonary nodules caused by pneumonia, so that certain nodules appear in the lung; second, pulmonary nodules due to lung infection, such as the appearance of tuberculosis, may result in pulmonary nodules; third, pulmonary nodules that result from severe trauma to the lungs, such as a puncture in the lungs, can cause pulmonary nodules; fourth, pulmonary nodules due to long-term smoking, or long-term absorption of contaminated air.
The pulmonary nodules have high possibility of being degraded into lung cancer, and because the hospital equipment in the two-three city in China is deficient, the experience of doctors is insufficient, and the judgment of the pulmonary nodules also depends on the professional level of the doctors, the possibility of misdiagnosis of the pulmonary nodules is increased. Therefore, the accuracy of pulmonary nodule detection is improved, and the automated equipment for nodule detection based on the CT image is very important for assisting the diagnosis of doctors.
Disclosure of Invention
In view of the foregoing problems, an object of the embodiments of the present invention is to provide a method, an apparatus, a device and a storage medium for identifying malignant lung nodules by image fusion, which can automatically and accurately detect lung nodules according to CT images, thereby reducing the probability of misdiagnosis.
The embodiment of the invention provides a malignant lung nodule identification method based on image fusion, which comprises the following steps:
preprocessing at least one acquired lung CT image;
performing fusion processing on each group of preprocessed lung CT images to obtain a fusion image;
labeling the nodule part in the fusion image according to the label of the nodule position of each group of lung CT images; and
and training the constructed convolutional neural network by using the labeled fusion image so as to identify the position of the malignant lung nodule in the CT image by using the trained convolutional neural network.
Preferably, the preprocessing includes applying gaussian filtering to make the lung CT image linear and smooth overall, and improve contrast, so as to more clearly display the lung nodule characteristics.
Preferably, the performing the fusion processing on the preprocessed CT images of each group of lungs to obtain the fusion image specifically includes:
assigning a predetermined specific gravity to each lung CT image in the set of lung CT images;
and performing fusion superposition on all images in the group of lung CT images according to the specific gravity of each lung CT image to generate a fusion image.
Preferably, each set of lung CT images includes a first lung CT image, a second lung CT image, and a third lung CT image; wherein the first lung CT image and the third lung CT image are edge part slice images of a nodule; the second lung CT image is a nodule center slice image; the specific gravity of the first lung CT image and the third lung CT image is 0.25; the specific gravities of the second lung CT images are respectively 0.5.
Preferably, the convolutional neural network has a specific structure as follows:
the first layer, convolution kernel size is that using 64 convolution kernels with size of 3 × 3 × 3, making convolution with step length of 1 for the input fusion image;
the second layer, convolution layer, the convolution kernel size is that using 64 convolution kernels with size of 3 × 3 × 64, making convolution with step size of 1 for the input data;
the third layer, the maximum pooling layer, performs pooling operation with a pooling interval of 2 x 2 and a step length of 2 on the data input from the second layer to the third layer;
a fourth layer, convolution layer, the convolution kernel size is that the convolution with step size 1 is made to the input data by using 128 convolution kernels with size of 3 × 3 × 64;
the fifth layer, convolution kernel size is that using 128 convolution kernels with size of 3 × 3 × 128, making convolution with step size of 1 for the input data;
the sixth layer, the maximum pooling layer, perform pooling operation with pooling interval of 2 x 2 and step length of 2 on the data input from the fifth layer to the sixth layer;
a seventh layer, convolution layer, the convolution kernel size is that 256 convolution kernels with the size of 3 × 3 × 128 are used, and convolution with the step length of 1 is performed on the input data;
the eighth layer, convolution kernel size is to use 256 convolution kernels with size of 3 × 3 × 256, to make convolution with step length of 1 for the input data;
the ninth layer is a maximum pooling layer, and pooling operation with a pooling interval of 2 multiplied by 2 and a step length of 2 is performed on the data input to the ninth layer from the eighth layer;
the tenth layer, convolution layer, the convolution kernel size is that using 512 convolution kernels with size of 3 × 3 × 256, making convolution with step length of 1 for the input data;
the eleventh layer, convolution kernel size is that using 512 convolution kernels with size of 3 × 3 × 512, making convolution with step length of 1 for the input data;
a twelfth layer, convolution layer, the convolution kernel size is that 512 convolution kernels with the size of 3 × 3 × 512 are used, and convolution with the step length of 1 is performed on the input data;
a thirteenth layer, namely a maximum pooling layer, wherein pooling operation with a pooling interval of 2 x 2 and a step length of 2 is performed on data input from the twelfth layer to the twelfth layer;
a fourteenth layer, convolutional layer, the convolutional kernel size is that 512 convolutional kernels with the size of 3 × 3 × 512 are used, and the convolution with the step length of 1 is performed on the input data;
a fifteenth layer, convolutional layer, where the convolutional kernel size is that 512 convolutional kernels with size of 3 × 3 × 512 are used, and the input data is convolved with step length of 1;
a sixteenth layer, convolutional layer, where the convolutional kernel size is that 512 convolutional kernels with size of 3 × 3 × 512 are used, and the input data is convolved with step length of 1;
a seventeenth layer, a maximum pooling layer, wherein pooling operation is performed on data input to the seventeenth layer by the sixteenth layer with a pooling interval of 2 × 2 and a step length of 2;
eighteenth layer, full connection layer, convolution kernel size 7 × 7 × 512 × 4096, step size 1;
nineteenth layer, full connection layer, convolution kernel size 1 × 1 × 512 × 4096, step size 1.
Preferably, the convolutional neural network uses a leak ReLu as an activation function, the leak ReLu assigns all negative values to a non-zero slope, and the mathematical expression is as follows:
aiis a fixed parameter in the interval (1, + ∞).
Preferably, the training set of the convolutional neural network model is a Tianchi data set in a Tianchi medical AI tournament.
The embodiment of the invention also provides a malignant lung nodule recognition device for image fusion, which comprises:
the preprocessing unit is used for preprocessing the acquired at least one group of lung CT images;
the fusion unit is used for carrying out fusion processing on each group of preprocessed lung CT images so as to obtain a fusion image;
the labeling unit is used for labeling the nodule part in the fusion image according to the label of the nodule position of each group of lung CT images; and
and the training unit is used for training the constructed convolutional neural network by using the labeled fusion image so as to identify the position of a malignant lung nodule in the CT image by using the trained convolutional neural network.
The embodiment of the invention also provides image fusion malignant lung nodule identification equipment, which comprises: a processor, a display, a memory, and a computer program stored in the memory and configured to be executed by the processor, the processor when executing the computer program implementing the above-described image fusion malignant lung nodule detection method.
An embodiment of the present invention further provides a computer-readable storage medium, which includes a stored computer program, where the computer program, when running, controls a device on which the computer-readable storage medium is located to execute the method for identifying malignant lung nodules in image fusion as described above.
In the above embodiment, a group of lung CT images are subjected to image fusion and processed by proportion to generate a two-dimensional fusion image, and the characteristics of nodules in a plurality of lung CT images are combined to one image, so that the detection effect on the lung nodules is improved; in addition, the accuracy rate of lung nodule detection is greatly improved by combining a convolutional neural network, so that the accuracy rate of doctor on lung nodule detection judgment is improved, and the misdiagnosis probability is reduced.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart illustrating a malignant lung nodule identification method by image fusion according to a first embodiment of the present invention;
FIG. 2 is a diagram illustrating the effect of image fusion of three CT images of the lung according to the first embodiment of the present invention;
fig. 3 is a schematic structural diagram of an image-fused malignant lung nodule detection apparatus according to a second embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The embodiments of the present invention, and all other embodiments obtained by a person of ordinary skill in the art without any inventive work, belong to the scope of protection of the present invention.
Referring to fig. 1, a first embodiment of the present invention provides an image fusion malignant lung nodule identification method, which can be performed by an image fusion malignant lung nodule identification device (hereinafter referred to as an identification device), and at least includes the following steps:
s101, preprocessing at least one group of acquired lung CT images.
In this embodiment, the identification device may be a computer or other devices with computing processing capability, and the present invention is not limited in particular.
In the present embodiment, three images are generally used to describe the nodule characteristics of the lung, so the present embodiment classifies the three lung CT images into a set of lung CT images.
In this embodiment, after at least one set of lung CT images is acquired, each set of images in each set of lung CT images needs to be preprocessed to improve the detection accuracy. The purpose of preprocessing mainly comprises improving the image and enhancing the image characteristics, and obtaining an image which has higher contrast and can more clearly display the lung nodule characteristics. For example, a frequency domain method (such as gaussian filtering) can be used to make the whole image linear and smooth, and improve the contrast.
It should be noted that, of course, other frequency domain methods may also be adopted, such as a butterworth high-pass filter, a gaussian high-pass filter, an exponential filter, and the like, and the present invention is not limited in particular.
In addition, a spatial domain method, such as a mean filtering method, a median filtering method, a spatial domain filtering method, a gradient operator method, a second derivative operator method, a high-pass filtering method, a mask matching method, and the like, may also be used for the preprocessing, and may be specifically set according to actual needs.
And S102, carrying out fusion processing on the preprocessed CT images of each group of lungs to obtain a fusion image.
In this embodiment, after the preprocessing, the three CT images in each group of lung CT images may be fused, so that the fine features of the lung nodules in the three lung CT images are fused in one image, which is more beneficial for the neural network to identify and detect the nodules.
Specifically, each set of lung CT images comprises a first lung CT image, a second lung CT image and a third lung CT image; wherein the first lung CT image and the third lung CT image are edge part slice images of a nodule; the second lung CT image is a nodule center slice image. During the fusion, the fusion is performed according to the weights configured for each group of lung CT images in advance to obtain a fused image.
It should be noted that, in a preferred embodiment, the second lung CT image is a nodule center slice image; the specific gravity of the first lung CT image and the third lung CT image is 0.25; the specific gravities of the second lung CT images are respectively 0.5.
Wherein the first and third pulmonary CT images are considered to be edge portion slice images of a nodule; the second pulmonary CT image is a nodule center slice image, and therefore, a larger weight is set for the second pulmonary CT image to be beneficial to enhance the main feature of the nodule.
Of course, it should be understood that in other embodiments of the present invention, the specific gravity of the second lung CT image may be other values, for example, 0.6, 0.7 or other values, and it is only necessary to ensure that the specific gravity of the three is greater, and the present invention is not limited in particular.
As shown in fig. 2, the fusion of the three lung CT images can be realized by opencv, and of course, other image processing software may be adopted, which is not limited in the present invention.
And S103, labeling the nodule part in the fusion image according to the label of the nodule position of each lung CT image group.
In this embodiment, the nodule position of each lung CT image is provided with a label, and after fusion, a specific nodule position needs to be marked according to the nodule coordinates in the label of the nodule position of the lung CT image, so as to perform subsequent training.
And S104, training the constructed convolutional neural network by using the marked fusion image so as to identify the position of the malignant lung nodule in the CT image by using the trained convolutional neural network.
In this embodiment, the convolutional neural network may be a two-dimensional convolutional neural network, which specifically includes a nineteen-layer structure, and sequentially includes:
the first layer, convolution kernel size is that using 64 convolution kernels with size of 3 × 3 × 3, making convolution with step length of 1 for the input fusion image;
the second layer, convolution layer, the convolution kernel size is that using 64 convolution kernels with size of 3 × 3 × 64, making convolution with step size of 1 for the input data;
the third layer, the maximum pooling layer, performs pooling operation with a pooling interval of 2 x 2 and a step length of 2 on the data input from the second layer to the third layer;
a fourth layer, convolution layer, the convolution kernel size is that the convolution with step size 1 is made to the input data by using 128 convolution kernels with size of 3 × 3 × 64;
the fifth layer, convolution kernel size is that using 128 convolution kernels with size of 3 × 3 × 128, making convolution with step size of 1 for the input data;
the sixth layer, the maximum pooling layer, perform pooling operation with pooling interval of 2 x 2 and step length of 2 on the data input from the fifth layer to the sixth layer;
a seventh layer, convolution layer, the convolution kernel size is that 256 convolution kernels with the size of 3 × 3 × 128 are used, and convolution with the step length of 1 is performed on the input data;
the eighth layer, convolution kernel size is to use 256 convolution kernels with size of 3 × 3 × 256, to make convolution with step length of 1 for the input data;
the ninth layer is a maximum pooling layer, and pooling operation with a pooling interval of 2 multiplied by 2 and a step length of 2 is performed on the data input to the ninth layer from the eighth layer;
the tenth layer, convolution layer, the convolution kernel size is that using 512 convolution kernels with size of 3 × 3 × 256, making convolution with step length of 1 for the input data;
the eleventh layer, convolution kernel size is that using 512 convolution kernels with size of 3 × 3 × 512, making convolution with step length of 1 for the input data;
a twelfth layer, convolution layer, the convolution kernel size is that 512 convolution kernels with the size of 3 × 3 × 512 are used, and convolution with the step length of 1 is performed on the input data;
a thirteenth layer, namely a maximum pooling layer, wherein pooling operation with a pooling interval of 2 x 2 and a step length of 2 is performed on data input from the twelfth layer to the twelfth layer;
a fourteenth layer, convolutional layer, the convolutional kernel size is that 512 convolutional kernels with the size of 3 × 3 × 512 are used, and the convolution with the step length of 1 is performed on the input data;
a fifteenth layer, convolutional layer, where the convolutional kernel size is that 512 convolutional kernels with size of 3 × 3 × 512 are used, and the input data is convolved with step length of 1;
a sixteenth layer, convolutional layer, where the convolutional kernel size is that 512 convolutional kernels with size of 3 × 3 × 512 are used, and the input data is convolved with step length of 1;
a seventeenth layer, a maximum pooling layer, wherein pooling operation is performed on data input to the seventeenth layer by the sixteenth layer with a pooling interval of 2 × 2 and a step length of 2;
eighteenth layer, full connection layer, convolution kernel size 7 × 7 × 512 × 4096, step size 1;
nineteenth layer, full connection layer, convolution kernel size 1 × 1 × 512 × 4096, step size 1.
The convolutional neural network adopts a leak ReLu as an activation function, the leak ReLu endows all negative values with a non-zero slope, and the mathematical expression is as follows:
aiis a fixed parameter in the interval (1, + ∞).
It should be noted that, in this embodiment, in order to obtain an accurate convolutional neural network model, a large amount of training data is required, the embodiment uses a Tianchi data set in the Tianchi medical AI tournament, and of course, other data sets may also be used as the training set, which is not specifically limited herein.
It should be noted that in other embodiments of the present invention, other modified or optimized convolutional neural network models may be used, and these schemes and modifications are within the scope of the present invention.
In this embodiment, after the convolutional neural network is obtained through training, the lung CT image to be identified may be input into the convolutional neural network, and the convolutional neural network may identify the position of a malignant lung nodule in the image.
In summary, in the embodiment, a group of lung CT images are subjected to image fusion and processed in proportion to generate a two-dimensional fusion image, and the characteristics of nodules in a plurality of lung CT images are combined to one image, so that the detection effect of lung nodules is improved; in addition, the accuracy rate of lung nodule detection is greatly improved by combining a convolutional neural network, so that the accuracy rate of doctor on lung nodule detection judgment is improved, and the misdiagnosis probability is reduced.
Referring to fig. 3, a second embodiment of the present invention further provides an image fusion malignant lung nodule recognition apparatus, including:
the preprocessing unit 10 is used for preprocessing at least one group of acquired lung CT images;
a fusion unit 20, configured to perform fusion processing on each preprocessed lung CT image group to obtain a fusion image;
the labeling unit 30 is configured to label a nodule portion in the fused image according to a label of a nodule position in each group of lung CT images; and
and the training unit 40 is used for training the constructed convolutional neural network by using the labeled fusion image so as to identify the position of the malignant lung nodule in the CT image by using the trained convolutional neural network.
Preferably, the preprocessing includes applying gaussian filtering to make the lung CT image linear and smooth overall, and improve contrast, so as to more clearly display the lung nodule characteristics.
Preferably, the fusion unit 20 specifically includes:
a specific gravity setting module for assigning a preset specific gravity to each lung CT image in the group of lung CT images;
and the fusion image generation module is used for performing fusion superposition on all images in the group of lung CT images according to the proportion of each lung CT image so as to generate a fusion image.
Preferably, each set of lung CT images includes a first lung CT image, a second lung CT image, and a third lung CT image; wherein the first lung CT image and the third lung CT image are edge part slice images of a nodule; the second lung CT image is a nodule center slice image; the specific gravity of the first lung CT image and the third lung CT image is 0.25; the specific gravities of the second lung CT images are respectively 0.5.
Preferably, the convolutional neural network uses a leak ReLu as an activation function, the leak ReLu assigns all negative values to a non-zero slope, and the mathematical expression is as follows:
aiis a fixed parameter in the interval (1, + ∞).
Preferably, the lung nodule training set is a Tianchi dataset in a Tianchi medical AI tournament.
The third embodiment of the present invention also provides an image-fused malignant lung nodule identifying apparatus, including: a processor, a display, a memory, and a computer program stored in the memory and configured to be executed by the processor, the processor when executing the computer program implementing the above-described image fusion malignant lung nodule detection method.
The fourth embodiment of the present invention further provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program runs, the apparatus on which the computer-readable storage medium is located is controlled to execute the method for identifying malignant lung nodules by image fusion as described above.
Illustratively, the computer program may be divided into one or more units, which are stored in the memory and executed by the processor to accomplish the present invention. The one or more elements may be a series of computer program instruction segments capable of performing certain functions for describing the execution of the computer program in an image fusion malignant lung nodule identification apparatus.
The malignant lung nodule identification device for image fusion can be a desktop computer, a notebook, a palm computer, a cloud server cluster and other computing devices. The image fused malignant lung nodule identification device may include, but is not limited to, a processor, a memory. It will be appreciated by those skilled in the art that the schematic diagram is merely an example of an image-fused malignant lung nodule recognition device, and does not constitute a limitation of an image-fused malignant lung nodule recognition device, and may include more or less components than those shown, or combine some components, or different components, for example, the image-fused malignant lung nodule recognition device may further include an input-output device, a network access device, a bus, etc.
The Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. The general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, the control center of the image fusion malignant lung nodule identification apparatus, various interfaces and lines connecting the various parts of the entire image fusion malignant lung nodule identification apparatus.
The memory may be used to store the computer programs and/or modules, and the processor may implement the various functions of the image-fused malignant lung nodule identification apparatus by running or executing the computer programs and/or modules stored in the memory and invoking the data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.
Wherein the image-fused malignant lung nodule identification apparatus integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. It is understood that all or part of the processes in the method according to the embodiments of the present invention may be implemented by executing a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps in the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
It should be noted that the above-described device embodiments are merely illustrative, where the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. In addition, in the drawings of the embodiment of the apparatus provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, and may be specifically implemented as one or more communication buses or signal lines. One of ordinary skill in the art can understand and implement it without inventive effort.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention.
Claims (7)
1. An image fusion malignant lung nodule identification method, comprising:
preprocessing at least one acquired lung CT image; the preprocessing comprises the steps of adopting Gaussian filtering to enable the whole lung CT image to be linear and smooth, and improving contrast, so that the lung nodule characteristics can be displayed more clearly;
performing fusion processing on each group of preprocessed lung CT images to obtain a fusion image; each group of lung CT images comprises a first lung CT image, a second lung CT image and a third lung CT image; wherein the first lung CT image and the third lung CT image are edge part slice images of a nodule; the second lung CT image is a nodule center slice image;
the fusing the preprocessed CT images of each group of lungs to obtain fused images comprises the following steps:
assigning a predetermined specific gravity to each lung CT image in the set of lung CT images;
according to the proportion of each lung CT image, performing fusion superposition on all images in a group of lung CT images to generate a fusion image;
labeling the nodule part in the fusion image according to the label of the nodule position of each group of lung CT images; and
training the constructed convolutional neural network by using the labeled fusion image so as to identify the position of a malignant lung nodule in the CT image by using the trained convolutional neural network; the convolutional neural network has the specific structure as follows:
the first layer, convolution kernel size is that using 64 convolution kernels with size of 3 × 3 × 3, making convolution with step length of 1 for the input fusion image;
the second layer, convolution layer, the convolution kernel size is that using 64 convolution kernels with size of 3 × 3 × 64, making convolution with step size of 1 for the input data;
the third layer, the maximum pooling layer, performs pooling operation with a pooling interval of 2 x 2 and a step length of 2 on the data input from the second layer to the third layer;
a fourth layer, convolution layer, the convolution kernel size is that the convolution with step size 1 is made to the input data by using 128 convolution kernels with size of 3 × 3 × 64;
the fifth layer, convolution kernel size is that using 128 convolution kernels with size of 3 × 3 × 128, making convolution with step size of 1 for the input data;
the sixth layer, the maximum pooling layer, perform pooling operation with pooling interval of 2 x 2 and step length of 2 on the data input from the fifth layer to the sixth layer;
a seventh layer, convolution layer, the convolution kernel size is that 256 convolution kernels with the size of 3 × 3 × 128 are used, and convolution with the step length of 1 is performed on the input data;
the eighth layer, convolution kernel size is to use 256 convolution kernels with size of 3 × 3 × 256, to make convolution with step length of 1 for the input data;
the ninth layer is a maximum pooling layer, and pooling operation with a pooling interval of 2 multiplied by 2 and a step length of 2 is performed on the data input to the ninth layer from the eighth layer;
the tenth layer, convolution layer, the convolution kernel size is that using 512 convolution kernels with size of 3 × 3 × 256, making convolution with step length of 1 for the input data;
the eleventh layer, convolution kernel size is that using 512 convolution kernels with size of 3 × 3 × 512, making convolution with step length of 1 for the input data;
a twelfth layer, convolution layer, the convolution kernel size is that 512 convolution kernels with the size of 3 × 3 × 512 are used, and convolution with the step length of 1 is performed on the input data;
a thirteenth layer, namely a maximum pooling layer, wherein pooling operation with a pooling interval of 2 x 2 and a step length of 2 is performed on data input from the twelfth layer to the twelfth layer;
a fourteenth layer, convolutional layer, the convolutional kernel size is that 512 convolutional kernels with the size of 3 × 3 × 512 are used, and the convolution with the step length of 1 is performed on the input data;
a fifteenth layer, convolutional layer, where the convolutional kernel size is that 512 convolutional kernels with size of 3 × 3 × 512 are used, and the input data is convolved with step length of 1;
a sixteenth layer, convolutional layer, where the convolutional kernel size is that 512 convolutional kernels with size of 3 × 3 × 512 are used, and the input data is convolved with step length of 1;
a seventeenth layer, a maximum pooling layer, wherein pooling operation is performed on data input to the seventeenth layer by the sixteenth layer with a pooling interval of 2 × 2 and a step length of 2;
eighteenth layer, full connection layer, convolution kernel size 7 × 7 × 512 × 4096, step size 1;
nineteenth layer, full connection layer, convolution kernel size 1 × 1 × 512 × 4096, step size 1.
2. The image-fused malignant lung nodule identification method of claim 1, wherein the first lung CT image and the third lung CT image have a specific gravity of 0.25; the specific gravities of the second lung CT images are respectively 0.5.
3. The image-fused malignant lung nodule identification method of claim 1, wherein the convolutional neural network uses a leak ReLu as an activation function, the leak ReLu assigns all negative values a non-zero slope, and the mathematical expression is:
aiis a fixed parameter in the interval (1, + ∞).
4. The image-fused malignant lung nodule recognition method of claim 1, wherein the training set of convolutional neural network models is a Tianchi dataset in a Tianchi medical AI tournament.
5. An image-fused malignant lung nodule recognition apparatus, comprising:
the preprocessing unit is used for preprocessing the acquired at least one group of lung CT images;
the fusion unit is used for carrying out fusion processing on each group of preprocessed lung CT images so as to obtain a fusion image;
the labeling unit is used for labeling the nodule part in the fusion image according to the label of the nodule position of each group of lung CT images; and
and the training unit is used for training the constructed convolutional neural network by using the labeled fusion image so as to identify the position of a malignant lung nodule in the CT image by using the trained convolutional neural network.
6. An image fused malignant lung nodule identifying apparatus, comprising: a processor, a display, a memory, and a computer program stored in the memory and configured to be executed by the processor, the processor when executing the computer program implementing the image fusion malignant lung nodule detection method of any one of claims 1 to 4.
7. A computer-readable storage medium, comprising a stored computer program, wherein the computer program, when executed, controls an apparatus in which the computer-readable storage medium is located to perform the method for image fusion malignant lung nodule identification according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811323026.4A CN109523525B (en) | 2018-11-07 | 2018-11-07 | Image fusion malignant lung nodule identification method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811323026.4A CN109523525B (en) | 2018-11-07 | 2018-11-07 | Image fusion malignant lung nodule identification method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109523525A CN109523525A (en) | 2019-03-26 |
| CN109523525B true CN109523525B (en) | 2021-04-16 |
Family
ID=65776033
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811323026.4A Active CN109523525B (en) | 2018-11-07 | 2018-11-07 | Image fusion malignant lung nodule identification method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109523525B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110309860B (en) * | 2019-06-06 | 2022-11-08 | 昆明理工大学 | Method for classifying malignancy degree of lung nodule based on convolutional neural network |
| CN111583179B (en) * | 2020-04-02 | 2022-09-16 | 天津大学 | Lung nodule deep learning classification method based on floating cutting |
| CN111739004B (en) * | 2020-06-22 | 2024-01-23 | 联想(北京)有限公司 | Image processing method, device and storage medium |
| CN112668527B (en) * | 2020-12-31 | 2023-04-25 | 华南理工大学 | Ultrasonic guided wave semi-supervised imaging detection method |
| CN113506289B (en) * | 2021-07-28 | 2024-03-29 | 中山仰视科技有限公司 | Method for classifying false positives of lung nodules by using double-flow network |
| CN113610739A (en) * | 2021-08-10 | 2021-11-05 | 平安国际智慧城市科技股份有限公司 | Image data enhancement method, device, equipment and storage medium |
| CN113724177B (en) * | 2021-09-07 | 2023-12-15 | 北京大学深圳医院 | Lung nodule information fusion method, device, equipment and storage medium thereof |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106097340A (en) * | 2016-06-12 | 2016-11-09 | 山东大学 | A kind of method automatically detecting and delineating Lung neoplasm position based on convolution grader |
| US20180033144A1 (en) * | 2016-09-21 | 2018-02-01 | Realize, Inc. | Anomaly detection in volumetric images |
| CN107945168A (en) * | 2017-11-30 | 2018-04-20 | 上海联影医疗科技有限公司 | The processing method and magic magiscan of a kind of medical image |
| CN108197580A (en) * | 2018-01-09 | 2018-06-22 | 吉林大学 | A kind of gesture identification method based on 3d convolutional neural networks |
| CN108389190A (en) * | 2018-02-08 | 2018-08-10 | 贵州联科卫信科技有限公司 | A kind of Lung neoplasm automatic testing method based on deep learning method |
-
2018
- 2018-11-07 CN CN201811323026.4A patent/CN109523525B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106097340A (en) * | 2016-06-12 | 2016-11-09 | 山东大学 | A kind of method automatically detecting and delineating Lung neoplasm position based on convolution grader |
| US20180033144A1 (en) * | 2016-09-21 | 2018-02-01 | Realize, Inc. | Anomaly detection in volumetric images |
| CN107945168A (en) * | 2017-11-30 | 2018-04-20 | 上海联影医疗科技有限公司 | The processing method and magic magiscan of a kind of medical image |
| CN108197580A (en) * | 2018-01-09 | 2018-06-22 | 吉林大学 | A kind of gesture identification method based on 3d convolutional neural networks |
| CN108389190A (en) * | 2018-02-08 | 2018-08-10 | 贵州联科卫信科技有限公司 | A kind of Lung neoplasm automatic testing method based on deep learning method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109523525A (en) | 2019-03-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109523525B (en) | Image fusion malignant lung nodule identification method, device, equipment and storage medium | |
| CN110874594B (en) | Human body appearance damage detection method and related equipment based on semantic segmentation network | |
| CN109214238B (en) | Multi-target tracking method, device, equipment and storage medium | |
| WO2019201204A1 (en) | Method and device for detecting nodule in image | |
| WO2021088849A1 (en) | Ultrasonic imaging method and apparatus, readable storage medium, and terminal device | |
| CN108537292A (en) | Semantic segmentation network training method, image, semantic dividing method and device | |
| CN110378966B (en) | Method, device and equipment for calibrating external parameters of vehicle-road coordination phase machine and storage medium | |
| WO2019223147A1 (en) | Liver canceration locating method and apparatus, and storage medium | |
| CN110381369A (en) | Determination method, apparatus, equipment and the storage medium of recommendation information implantation position | |
| CN110930414A (en) | Lung region shadow marking method and device of medical image, server and storage medium | |
| CN111667474A (en) | Fracture identification method, apparatus, device and computer readable storage medium | |
| CN109948666A (en) | Image similarity recognition methods, device, equipment and storage medium | |
| WO2022160202A1 (en) | Method and apparatus for inspecting mask wearing, terminal device and readable storage medium | |
| CN113449538A (en) | Visual model training method, device, equipment and storage medium | |
| CN109241930B (en) | Method and apparatus for processing eyebrow image | |
| JP2020522056A (en) | Surface-based object identification | |
| CN112465050B (en) | Image template selection method, device, equipment and storage medium | |
| Skirnevskiy et al. | Digital image processing using parallel computing based on CUDA technology | |
| CN116227573B (en) | Segmentation model training method, image segmentation device and related media | |
| CN111281355B (en) | Method and equipment for determining pulse acquisition position | |
| WO2021120577A1 (en) | Method for data computation in neural network model, image processing method, and device | |
| CN108875899A (en) | Data processing method, device and system and storage medium for neural network | |
| CN115294426B (en) | Method, device and equipment for tracking interventional medical equipment and storage medium | |
| CN116503608A (en) | Data distillation method based on artificial intelligence and related equipment | |
| CN111382831A (en) | Method and device for accelerating forward reasoning of convolutional neural network model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |