CN110084810B - Pulmonary nodule image detection method, model training method, device and storage medium - Google Patents
Pulmonary nodule image detection method, model training method, device and storage medium Download PDFInfo
- Publication number
- CN110084810B CN110084810B CN201910374504.2A CN201910374504A CN110084810B CN 110084810 B CN110084810 B CN 110084810B CN 201910374504 A CN201910374504 A CN 201910374504A CN 110084810 B CN110084810 B CN 110084810B
- Authority
- CN
- China
- Prior art keywords
- dimensional feature
- dimensional
- lung nodule
- image
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30061—Lung
- G06T2207/30064—Lung nodule
Abstract
The invention discloses a lung nodule image detection method, a model training method, a device and a storage medium, and relates to the field of medical image processing. The model training method for detecting the lung nodule image comprises the following steps: preprocessing CT pulmonary nodule data through a convolution network to obtain a pulmonary nodule characteristic image; acquiring three-dimensional characteristic data of a lung nodule characteristic image through an Xscene network structure; stacking three-dimensional feature data of the lung nodule feature image to obtain a first four-dimensional feature; carrying out three-dimensional convolution kernel processing on the first four-dimensional feature to obtain a second four-dimensional feature; calculating a class probability according to the second four-dimensional feature; the category probability is the probability that each pixel point is a pulmonary nodule; and when the class probability meets the convergence condition of the model, obtaining the trained lung nodule image detection model. And a two-dimensional convolution and three-dimensional convolution kernel processing architecture is adopted, so that the calculation efficiency is improved, and the requirement on hardware resources is reduced.
Description
Technical Field
The application relates to the field of medical image processing, in particular to a lung nodule image detection method, a lung nodule image model training method, a lung nodule image detection device and a lung nodule image model training storage medium.
Background
Pulmonary nodules are granulomatous diseases of multiple systems and multiple organs with unknown causes, often invade organs such as lung, bilateral pulmonary lymph nodes, eyes, skin and the like, and the invasion rate of the chest is as high as 80-90%. CT is an effective tool for lung nodule detection, but the contradiction between the number of CT images and the number of image physicians makes an automatic lung nodule detection algorithm an urgent need. A great deal of research has been conducted on the automatic detection technology of pulmonary nodules, which focuses mainly on the field of traditional machine learning and deep learning.
The prior art generally uses two approaches: firstly, extracting features by two-dimensional convolution, and performing correlation analysis on data of each layer by using a circulation network; second, the CT data is processed using a three-dimensional convolutional network. The first scheme can only obtain one classification result, the serial structure of the circulation network can reduce the calculation efficiency, and meanwhile, the calculation of the circulation network between any layers is not suitable for disease monitoring of lung nodules which are locally related. The second solution requires a large number of samples and a highly configured graphics processor, and requires a long training time.
There is a need for a computationally efficient lung nodule image detection model training method to solve the above-mentioned problems.
Disclosure of Invention
The embodiment of the invention aims to provide a lung nodule image detection method, a model training method, a device and a storage medium, which solve the problem of low calculation efficiency by using a model architecture of two-dimensional convolution and three-dimensional convolution kernel processing.
In order to achieve the above purpose, the embodiment of the present invention adopts the following technical solutions:
in a first aspect, an embodiment of the present invention provides a method for training a lung nodule image detection model, where the method includes: preprocessing CT pulmonary nodule data through a convolution network to obtain a pulmonary nodule characteristic image; acquiring three-dimensional characteristic data of a lung nodule characteristic image through an Xscene network structure; stacking three-dimensional feature data of the lung nodule feature image to obtain a first four-dimensional feature; carrying out three-dimensional convolution kernel processing on the first four-dimensional feature to obtain a second four-dimensional feature; calculating a class probability according to the second four-dimensional feature; the category probability is the probability that each pixel point is a pulmonary nodule; and when the class probability meets the convergence condition of the model, obtaining the trained lung nodule image detection model.
Optionally, before the three-dimensional feature data of the lung nodule feature image is acquired through an Xception network structure, the method further includes: and pre-training the Xception network structure according to the open-source two-dimensional image data.
Optionally, the method further comprises: and when the category probability does not meet the convergence condition of the model, acquiring CT lung nodule data again, and executing migration training on the Xcenter network structure.
Specifically, the three-dimensional convolution kernel processing is performed on the first four-dimensional feature to obtain a second four-dimensional feature of which the output channel number is the corresponding classification category number, and the method comprises the following steps:
processing the first four-dimensional feature by using a three-dimensional convolution kernel of 1x1x1 to obtain a third four-dimensional feature with the output channel number of 256; the number of output channels of the first four-dimensional feature is 2048. Processing the third four-dimensional feature by sequentially using three-dimensional convolution kernels of 5x1x3, 5x3x1, 7x1x3 and 7x3x1 to obtain a fourth four-dimensional feature with the output channel number of 256; and correcting the fourth four-dimensional feature by using a correction linear unit. And performing upsampling on the corrected fourth four-dimensional feature to obtain a fifth four-dimensional feature with the output channel number of 256. And performing matrix addition on the fifth four-dimensional feature and the attribute mapping layer with the corresponding size in the Xconvergence network structure to obtain a sixth four-dimensional feature with the output channel number of 256. Processing the sixth four-dimensional feature by using two three-dimensional convolution kernels of 3x3x3 to obtain a seventh four-dimensional feature with the output channel number of 256; the seventh four-dimensional feature is modified using a linear revision unit. And processing the corrected seventh four-dimensional feature by using a three-dimensional convolution kernel of 1x1x1 to obtain a second four-dimensional feature of which the output channel number is the corresponding classification category number.
In a second aspect, an embodiment of the present invention further provides a lung nodule image detection method, which is applied to a trained lung nodule image detection model obtained by the above lung nodule image detection model training method. The lung nodule image detection method comprises the following steps: and sequentially extracting adjacent N layers of CT image data according to the interval of N/2 through a convolution network until all CT image data are extracted, and preprocessing the CT image data to obtain a lung nodule characteristic image. Acquiring three-dimensional characteristic data of a lung nodule characteristic image through an Xscene network structure; stacking three-dimensional feature data of the lung nodule feature image to obtain a first four-dimensional feature; and carrying out three-dimensional convolution kernel processing on the first four-dimensional feature to obtain a second four-dimensional feature. Calculating a class probability according to the second four-dimensional feature; the class probability is the probability that each pixel point is a lung nodule.
Optionally, the method further comprises: predicting the N layers of CT image data according to the class probability to obtain a prediction result of the pulmonary nodule detection; and taking the average value of the prediction results of the two adjacent N layers of CT image data as the final prediction result of the lung nodule detection.
In a third aspect, an embodiment of the present invention further provides a model training device for lung nodule image detection, including: the device comprises a first processing module and a first obtaining module; the first acquisition module is used for acquiring CT pulmonary nodule data through a convolutional network; the first processing module is used for preprocessing CT pulmonary nodule data; the first acquisition module acquires a lung nodule feature image. The first acquisition module is further used for acquiring three-dimensional characteristic data of the lung nodule characteristic image through an Xscene network structure; the first processing module is further configured to stack three-dimensional feature data of the lung nodule feature image, and the first obtaining module is further configured to obtain a first four-dimensional feature. The first processing module is further used for performing three-dimensional convolution kernel processing on the first four-dimensional feature, and the first obtaining module is further used for obtaining a second four-dimensional feature; the first processing module is also used for calculating the class probability according to the second four-dimensional feature; the category probability is the probability that each pixel point is a pulmonary nodule; the first obtaining module is further configured to obtain the trained lung nodule image detection model when the category probability satisfies a convergence condition of the model.
Optionally, the first processing module is further configured to pre-train an Xception network structure according to the open-source two-dimensional image data.
Optionally, when the category probability does not satisfy the convergence condition of the model, the first obtaining module is further configured to re-obtain the CT lung nodule data; the first processing module is further configured to perform migration training on the Xception network structure.
In a fourth aspect, an embodiment of the present invention further provides a pulmonary nodule image detection apparatus, including: the device comprises a second processing module and a second acquiring module.
The second acquisition module is used for sequentially extracting adjacent N layers of CT image data through a convolution network according to the interval of N/2 until all CT image data are extracted; the second processing module is used for preprocessing the CT image data; the second acquisition module is further configured to acquire a lung nodule feature image.
The second processing module is further configured to obtain three-dimensional feature data of the lung nodule feature image according to the lung nodule feature image and the Xception network structure, and stack the three-dimensional feature data of the lung nodule feature image, and the second obtaining module is further configured to obtain a first four-dimensional feature. The second processing module is further configured to perform three-dimensional convolution kernel processing on the first four-dimensional feature, and the second obtaining module is further configured to obtain a second four-dimensional feature. The second processing module is further used for calculating the class probability according to the second four-dimensional feature; the class probability is the probability that each pixel point is a lung nodule.
Optionally, the second processing module is further configured to predict the N-layer image data according to the category probability to obtain a prediction result of lung nodule detection; the second processing module is further configured to take an average of the prediction results of the two adjacent N-layer CT image data as a final prediction result of the lung nodule detection.
In a fifth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is read and executed by a processor, the lung nodule image detection method and the model training method for lung nodule image detection are implemented.
The invention discloses a lung nodule image detection method, a model training method, a device and a storage medium, and relates to the field of medical image processing. The model training method for detecting the lung nodule image comprises the following steps: preprocessing CT pulmonary nodule data through a convolution network to obtain a pulmonary nodule characteristic image; acquiring three-dimensional characteristic data of a lung nodule characteristic image through an Xscene network structure; stacking three-dimensional feature data of the lung nodule feature image to obtain a first four-dimensional feature; carrying out three-dimensional convolution kernel processing on the first four-dimensional feature to obtain a second four-dimensional feature; calculating a class probability according to the second four-dimensional feature; the category probability is the probability that each pixel point is a pulmonary nodule; and when the class probability meets the convergence condition of the model, obtaining the trained lung nodule image detection model. And a two-dimensional convolution and three-dimensional convolution kernel architecture is adopted, so that the calculation efficiency is improved, and the requirement on hardware resources is reduced.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the embodiments of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a schematic diagram illustrating a model training method for lung nodule image detection according to an embodiment of the present invention.
Fig. 2 is a schematic diagram illustrating a pre-training Xception network structure according to an embodiment of the present invention.
Fig. 3 is a schematic diagram illustrating a three-dimensional convolution process according to an embodiment of the present invention.
Fig. 4 is a schematic diagram illustrating a lung nodule detection method according to an embodiment of the present invention.
Fig. 5 is a schematic diagram illustrating an optimized prediction result according to an embodiment of the present invention.
Fig. 6 is a schematic diagram illustrating a model training apparatus for lung nodule image detection according to an embodiment of the present invention.
Fig. 7 shows a lung nodule image detection apparatus according to an embodiment of the present invention.
Icon: 300-a model training device for lung nodule image detection, 301-a first acquisition module 301, 302-a first processing module, 400-a lung nodule image detection device, 401-a second acquisition module, 402-a second processing module.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present disclosure, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.
In order to improve the model training efficiency of pulmonary nodule image detection, the present invention provides a model training method for pulmonary nodule image detection, referring to fig. 1, fig. 1 is a schematic diagram of a model training method for pulmonary nodule image detection provided by an embodiment of the present invention, and the method includes:
and step 100, preprocessing the CT lung nodule data through a convolution network to obtain a lung nodule characteristic image.
The CT lung nodule data acquired through the convolutional network may be data extracted at random by a certain amount, and the specific extracted data amount may be set according to actual hardware resources: in the actual use process, 32 layers of randomly extracted image data can realize better training. The size of the acquired lung nodule feature image may be 496x496x 3.
And 102, stacking three-dimensional feature data of the lung nodule feature image to obtain a first four-dimensional feature.
In the process of processing the lung nodule characteristic image to the first four-dimensional characteristic, the change of the output channel number of the data can be realized.
And 103, performing three-dimensional convolution kernel processing on the first four-dimensional feature to obtain a second four-dimensional feature.
And 104, calculating the class probability according to the second four-dimensional feature.
The above category probability is the probability that each pixel point is a lung nodule.
And 105, judging whether the class probability meets the convergence condition of the model.
The convergence condition of the model may be the number of training times, or the accuracy of the lung nodule may be measured according to the class probability, or a statistical F1 score may be used, or an average Intersection over Union (MIoU for short).
When the class probability meets the convergence condition of the model, executing step 106;
and step 106, acquiring a trained lung nodule image detection model.
When the category probability does not meet the convergence condition of the model, the CT lung nodule data is obtained again, and migration training is carried out on the Xscene network structure; and re-executing the steps 100, 101, 102, 103, 104 and 105 until the model converges.
By using the model training architecture processed by the two-dimensional convolution and the three-dimensional convolution kernel, most of feature extraction work is completed by the two-dimensional convolution network, the calculated amount is small, and the calculation efficiency of model training is improved. Meanwhile, the method can also carry out model training in a lower-configuration hardware environment. When the network is used for model training, one-time training can be completed only by using the image data of 32 CT faults, the calculation efficiency of the model is improved, and the requirement on hardware configuration is further reduced.
Optionally, in order to obtain a better initial template in the training process, before step 101, a pre-training process is added, as shown in fig. 2, where fig. 2 is a schematic diagram of a pre-training Xception network structure provided in the embodiment of the present invention. The above method for training a lung nodule image detection model may further include:
and step 107, pre-training an Xconvergence network structure according to the open-source two-dimensional image data.
In particular, the pre-trained Xception network structure may be part of its parameters.
Optionally, in order to obtain a better result of the three-dimensional convolution kernel, a possible implementation manner of the step 10 is described, as shown in fig. 3, where fig. 3 is a schematic diagram of a three-dimensional convolution process provided by an embodiment of the present invention. When the number of output channels of the first four-dimensional feature is 2048, the three-dimensional convolution kernel processing process includes:
and 103-1, processing the first four-dimensional feature by using a three-dimensional convolution kernel of 1x1x1 to obtain a third four-dimensional feature with the output channel number of 256.
103-2, processing the third four-dimensional feature by sequentially using three-dimensional convolution kernels of 5x1x3, 5x3x1, 7x1x3 and 7x3x1 to obtain a fourth four-dimensional feature with the output channel number of 256; and correcting the fourth four-dimensional feature by using a correction linear unit.
And 103-3, performing up-sampling on the corrected fourth four-dimensional feature to obtain a fifth four-dimensional feature with the output channel number of 256.
Upsampling may be achieved by bilinear interpolation in order to double the spatial resolution of the data.
And 103-4, performing matrix addition on the fifth four-dimensional feature and the attribute mapping layer with the corresponding size in the Xconvergence network structure to obtain a sixth four-dimensional feature with the output channel number of 256.
Since the above step 101 only performs two-dimensional convolution, the above matrix is mainly for performing convolution in three dimensions.
103-5, processing the sixth four-dimensional feature by using two three-dimensional convolution kernels of 3x3x3 to obtain a seventh four-dimensional feature with the output channel number being 256; the seventh four-dimensional feature is modified using a linear revision unit.
And 103-6, processing the corrected seventh four-dimensional feature by using a three-dimensional convolution kernel of 1x1x1 to obtain a second four-dimensional feature of which the output channel number is the corresponding classification category number.
Step 103-1 to step 103-6 realize the conversion and processing from two-dimensional to three-dimensional, and obtain the second four-dimensional feature with the output channel number as the classification category number, and the category probability, that is, the probability that each pixel point is a lung nodule can be obtained through calculation. From the present point of view, three-dimensional convolution kernels are currently the most suitable way to process CT image data.
The lung nodule image detection model training method improves the calculation efficiency, reduces the requirements on hardware resources, and can reduce the manufacturing cost and shorten the detection time of lung nodules.
In order to implement lung nodule detection, the present invention provides a lung nodule image detection method, as shown in fig. 4, and fig. 4 is a schematic diagram of a lung nodule detection method provided in an embodiment of the present invention. The lung nodule image detection method comprises the following steps:
and 200, sequentially extracting adjacent N layers of CT image data through a convolution network according to the interval of N/2 until all CT image data are extracted, and preprocessing the CT image data to obtain a lung nodule characteristic image.
And 203, performing three-dimensional convolution kernel processing on the first four-dimensional feature to obtain a second four-dimensional feature.
And step 204, calculating the class probability according to the second four-dimensional feature.
By acquiring each CT image, the whole CT image is detected, and the CT images to be detected are sequentially extracted at intervals of N/2 for detection and judgment, so that the detection mode improves the calculation efficiency and reduces the requirement on hardware.
Optionally, in order to make the result of lung nodule detection more accurate, the process of optimizing the result on the basis of fig. 1 is shown in fig. 5, and fig. 5 is a schematic diagram of an optimized predicted result provided by the embodiment of the present invention. The lung nodule image detection method further comprises the following steps:
and step 205, predicting the CT image data of each N layers according to the category probability to obtain a prediction result of the lung nodule.
And step 206, taking the average value of the prediction results of the two adjacent N layers of CT image data as the final prediction result of the lung nodule detection.
For example, when N is 32, 32 adjacent samples (specific data can be set according to actual hardware resources, the larger the data is, the better the data is) are sequentially extracted for detection, an overlapping region including 16 samples is used for the intermediate samples, that is, the extracted sample numbers are [0,.. multidot.. 31], [16,. multidot.. 47], [32,. multidot.. 63], and the like, and the intermediate portion samples have an average value of two predicted values (e.g., [16,. multidot.. 31] fault is extracted at the first time and is also extracted at the second time) and is used twice as a final result.
The accuracy of lung nodule detection can be further improved by detecting adjacent CT image data and combining with average processing on a prediction result.
In order to implement the above model training method for detecting lung nodule images, an embodiment of the present invention further provides a model training device for detecting lung nodule images, as shown in fig. 6, where fig. 6 is a schematic view of the model training device for detecting lung nodule images provided in the embodiment of the present invention. The model training apparatus 300 for lung nodule image detection includes: a first acquisition module 301 and a first processing module 302.
A first obtaining module 301, configured to obtain CT lung nodule data through a convolutional network.
A first processing module 302 for preprocessing CT lung nodule data; the first acquisition module 301 acquires a lung nodule feature image.
The first obtaining module 301 is further configured to obtain three-dimensional feature data of a lung nodule feature image through an Xception network structure; the first processing module 302 is further configured to stack three-dimensional feature data of the lung nodule feature image, and the first obtaining module 301 is further configured to obtain a first four-dimensional feature. The first processing module 302 is further configured to perform three-dimensional convolution kernel processing on the first four-dimensional feature, and the first obtaining module 301 is further configured to obtain a second four-dimensional feature. The first processing module 302 is further configured to calculate a class probability according to the second four-dimensional feature; the category probability is the probability that each pixel point is a pulmonary nodule; the first obtaining module 301 is further configured to obtain a trained lung nodule image detection model when the class probability satisfies a convergence condition of the model.
The network structure processed by the two-dimensional convolution and the three-dimensional convolution kernel is used for training the model, so that the requirement of processing CT image data is met, the calculation efficiency of model training can be improved, and the configuration requirement of hardware is reduced.
Optionally, in order to ensure that the model training can obtain a better result, the first processing module 302 is further configured to pre-train the Xception network structure according to the open-source two-dimensional image data.
The pre-training can enable the Xmeeting network structure to obtain more accurate results when processing data.
Optionally, when the category probability does not satisfy the convergence condition of the model, the first obtaining module 301 is further configured to re-obtain the CT lung nodule data; the first processing module 302 is also configured to perform migration training on the Xception network structure.
The migration training is used for retraining the Xcaption network structure, so that the calculation efficiency can be further improved, the requirement on hardware is reduced, and the defect that the number of samples is small in the actual situation is overcome.
In order to implement the lung nodule image detection method, an embodiment of the present invention further provides a lung nodule image detection apparatus, as shown in fig. 7, where fig. 7 is the lung nodule image detection apparatus provided in the embodiment of the present invention, and the lung nodule image detection apparatus 400 includes: a second obtaining module 401 and a second processing module 402;
a second obtaining module 401, configured to sequentially extract adjacent N layers of CT image data according to an interval of N/2 through a convolutional network until all CT image data are extracted; a second processing module 402 for preprocessing the CT image data; the second acquisition module 401 is also used to acquire lung nodule feature images.
The second obtaining module 401 is further configured to obtain three-dimensional feature data of a lung nodule feature image through an Xception network structure; the second processing module 402 is further configured to stack three-dimensional feature data of the lung nodule feature image, and the second obtaining module 401 is further configured to obtain a first four-dimensional feature;
the second processing module 402 is further configured to perform three-dimensional convolution kernel processing on the first four-dimensional feature, and the second obtaining module 401 is further configured to obtain a second four-dimensional feature; the second processing module 402 is further configured to calculate a class probability according to the second four-dimensional feature; the class probability is the probability that each pixel point is a lung nodule.
The lung nodule image detection device uses a model framework of two-dimensional convolution and three-dimensional convolution kernel processing, improves the calculation efficiency of the model, and provides possibility for realizing the same function on low-configuration hardware.
In order to improve the accuracy of the lung nodule image detection, the second processing module 402 is further configured to predict every N layers of image data according to the class probability to obtain a prediction result of the lung nodule detection.
Since the data of each N/2 layer is extracted twice, the second processing module 402 is further configured to take an average of the prediction results of two adjacent N-layer CT image data as a final prediction result of lung nodule detection.
The method and the device have the advantages that the same or even better lung nodule prediction result as that in the prior art can be obtained on lower-configuration hardware, and the calculation efficiency of lung nodule image detection is improved.
The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is read and executed by a processor, the lung nodule image detection method and the model training method are implemented.
In summary, the invention discloses a lung nodule image detection method, a model training method, a device and a storage medium, and relates to the field of medical image processing. The model training method for detecting the lung nodule image comprises the following steps: preprocessing CT pulmonary nodule data through a convolution network to obtain a pulmonary nodule characteristic image; acquiring three-dimensional characteristic data of a lung nodule characteristic image through an Xscene network structure; stacking three-dimensional feature data of the lung nodule feature image to obtain a first four-dimensional feature; carrying out three-dimensional convolution kernel processing on the first four-dimensional feature to obtain a second four-dimensional feature; calculating a class probability according to the second four-dimensional feature; the category probability is the probability that each pixel point is a pulmonary nodule; and when the class probability meets the convergence condition of the model, obtaining the trained lung nodule image detection model. And a two-dimensional convolution and three-dimensional convolution kernel architecture is adopted, so that the calculation efficiency is improved, and the requirement on hardware resources is reduced.
The above description is only an alternative embodiment of the present invention and is not intended to limit the present invention, and various modifications and variations of the present invention may occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (9)
1. A method of model training for lung nodule image detection, the method comprising:
preprocessing CT pulmonary nodule data through a convolution network to obtain a pulmonary nodule characteristic image;
acquiring three-dimensional characteristic data of the lung nodule characteristic image through an Xscene network structure;
stacking the three-dimensional feature data of the lung nodule feature image to obtain a first four-dimensional feature;
performing three-dimensional convolution kernel processing on the first four-dimensional feature to obtain a second four-dimensional feature;
calculating a class probability according to the second four-dimensional feature; the category probability is the probability that each pixel point is a pulmonary nodule;
when the class probability meets the convergence condition of the model, obtaining a trained lung nodule image detection model;
performing three-dimensional convolution kernel processing on the first four-dimensional feature to obtain a second four-dimensional feature, wherein the three-dimensional convolution kernel processing comprises the following steps:
processing the first four-dimensional feature by using a three-dimensional convolution kernel of 1x1x1 to obtain a third four-dimensional feature with the output channel number of 256; the number of output channels of the first four-dimensional feature is 2048;
processing the third four-dimensional feature by sequentially using three-dimensional convolution kernels of 5x1x3, 5x3x1, 7x1x3 and 7x3x1 to obtain a fourth four-dimensional feature with the output channel number of 256; correcting the fourth four-dimensional feature by using a correction linear unit;
performing up-sampling on the corrected fourth four-dimensional feature to obtain a fifth four-dimensional feature with the output channel number of 256;
matrix addition is carried out on the fifth four-dimensional feature and an attribute mapping layer with the corresponding size in the Xmeeting network structure, and a sixth four-dimensional feature with 256 output channels is obtained;
processing the sixth four-dimensional feature by using two three-dimensional convolution kernels of 3x3x3 to obtain a seventh four-dimensional feature with the output channel number of 256; modifying the seventh fourth-dimensional feature using the linear revision unit;
and processing the corrected seventh four-dimensional feature by using a three-dimensional convolution kernel of 1x1x1 to obtain a second four-dimensional feature of which the output channel number is the corresponding classification category number.
2. The method of model training for lung nodule image detection as claimed in claim 1, wherein before the obtaining three-dimensional feature data of the lung nodule feature image through an Xception network structure, the method further comprises:
and pre-training the Xcaption network structure according to the open-source two-dimensional image data.
3. The method of model training for lung nodule image detection as recited in claim 2, further comprising:
and when the category probability does not meet the convergence condition of the model, the CT lung nodule data is obtained again, and the migration training is carried out on the Xscene network structure.
4. A lung nodule image detection method applied to a trained lung nodule image detection model obtained by the method of any one of claims 1-3, the method comprising:
sequentially extracting adjacent N layers of CT image data according to the interval of N/2 through a convolution network until all CT image data are extracted, and preprocessing the CT image data to obtain a lung nodule characteristic image;
acquiring three-dimensional characteristic data of the lung nodule characteristic image through an Xscene network structure;
stacking the three-dimensional feature data of the lung nodule feature image to obtain a first four-dimensional feature;
performing three-dimensional convolution kernel processing on the first four-dimensional feature to obtain a second four-dimensional feature;
calculating a class probability according to the second four-dimensional feature; the category probability is the probability that each pixel point is a lung nodule.
5. The lung nodule image detection method of claim 4, further comprising:
predicting the N-layer CT image data according to the category probability to obtain a prediction result of pulmonary nodule detection;
and taking the average value of the prediction results of the N adjacent layers of CT image data as the final prediction result of the lung nodule detection.
6. A model training device for lung nodule image detection is characterized by comprising: the device comprises a first processing module and a first obtaining module;
the first acquisition module is used for acquiring CT pulmonary nodule data through a convolutional network;
the first processing module is used for preprocessing the CT pulmonary nodule data; the first acquisition module acquires a lung nodule characteristic image;
the first acquisition module is further used for acquiring three-dimensional feature data of the lung nodule feature image through an Xscene network structure;
the first processing module is further used for stacking three-dimensional feature data of the lung nodule feature image, and the first acquiring module is further used for acquiring a first four-dimensional feature;
the first processing module is further configured to perform three-dimensional convolution kernel processing on the first four-dimensional feature, and the first obtaining module is further configured to obtain a second four-dimensional feature;
the first processing module is further used for calculating a class probability according to the second four-dimensional feature; the category probability is the probability that each pixel point is a pulmonary nodule;
when the category probability meets the convergence condition of the model, the first obtaining module is further used for obtaining a trained lung nodule image detection model;
the first obtaining module is further configured to obtain a second four-dimensional feature by:
processing the first four-dimensional feature by using a three-dimensional convolution kernel of 1x1x1 to obtain a third four-dimensional feature with the output channel number of 256; the number of output channels of the first four-dimensional feature is 2048;
processing the third four-dimensional feature by sequentially using three-dimensional convolution kernels of 5x1x3, 5x3x1, 7x1x3 and 7x3x1 to obtain a fourth four-dimensional feature with the output channel number of 256; correcting the fourth four-dimensional feature by using a correction linear unit;
performing up-sampling on the corrected fourth four-dimensional feature to obtain a fifth four-dimensional feature with the output channel number of 256;
matrix addition is carried out on the fifth four-dimensional feature and an attribute mapping layer with the corresponding size in the Xmeeting network structure, and a sixth four-dimensional feature with 256 output channels is obtained;
processing the sixth four-dimensional feature by using two three-dimensional convolution kernels of 3x3x3 to obtain a seventh four-dimensional feature with the output channel number of 256; modifying the seventh fourth-dimensional feature using the linear revision unit;
and processing the corrected seventh four-dimensional feature by using a three-dimensional convolution kernel of 1x1x1 to obtain a second four-dimensional feature of which the output channel number is the corresponding classification category number.
7. A lung nodule image detecting apparatus applied to a trained lung nodule image detection model obtained by the model training apparatus according to claim 6, the apparatus comprising: the second processing module and the second acquiring module;
the second acquisition module is used for sequentially extracting adjacent N layers of CT image data through a convolution network according to the interval of N/2 until all CT image data are extracted;
the second processing module is used for preprocessing the CT image data; the second acquisition module is further used for acquiring a lung nodule characteristic image;
the second obtaining module is further configured to obtain three-dimensional feature data of the lung nodule feature image through an Xception network structure;
the second processing module is further configured to stack three-dimensional feature data of the lung nodule feature image, and the second obtaining module is further configured to obtain a first four-dimensional feature;
the second processing module is further configured to perform three-dimensional convolution kernel processing on the first four-dimensional feature, and the second obtaining module is further configured to obtain a second four-dimensional feature;
the second processing module is further used for calculating a class probability according to the second four-dimensional feature; the category probability is the probability that each pixel point is a lung nodule.
8. The lung nodule image detecting apparatus according to claim 7, wherein the second processing module is further configured to predict the N-layer CT image data according to the category probability to obtain a prediction result of lung nodule detection; the second processing module is further configured to take an average of the prediction results of the two adjacent N-layer CT image data as a final prediction result of lung nodule detection.
9. A computer-readable storage medium, on which a computer program is stored which, when read and executed by a processor, implements the method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910374504.2A CN110084810B (en) | 2019-05-06 | 2019-05-06 | Pulmonary nodule image detection method, model training method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910374504.2A CN110084810B (en) | 2019-05-06 | 2019-05-06 | Pulmonary nodule image detection method, model training method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110084810A CN110084810A (en) | 2019-08-02 |
| CN110084810B true CN110084810B (en) | 2021-11-05 |
Family
ID=67418839
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910374504.2A Active CN110084810B (en) | 2019-05-06 | 2019-05-06 | Pulmonary nodule image detection method, model training method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110084810B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110751183A (en) * | 2019-09-24 | 2020-02-04 | 东软集团股份有限公司 | Image data classification model generation method, image data classification method and device |
| CN112529894B (en) * | 2020-12-22 | 2022-02-15 | 徐州医科大学 | Thyroid nodule diagnosis method based on deep learning network |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101515365A (en) * | 2009-03-25 | 2009-08-26 | 沈阳东软医疗系统有限公司 | Method for automatically separating adherent hyaline-vascular type lung nodule in CT image |
| CN109003260A (en) * | 2018-06-28 | 2018-12-14 | 深圳视见医疗科技有限公司 | CT image pulmonary nodule detection method, device, equipment and readable storage medium storing program for executing |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107016665B (en) * | 2017-02-16 | 2021-05-04 | 浙江大学 | CT pulmonary nodule detection method based on deep convolutional neural network |
| CN107563383A (en) * | 2017-08-24 | 2018-01-09 | 杭州健培科技有限公司 | A kind of medical image auxiliary diagnosis and semi-supervised sample generation system |
| CN108537784B (en) * | 2018-03-30 | 2021-08-24 | 四川元匠科技有限公司 | CT image pulmonary nodule detection method based on deep learning |
| CN108921195A (en) * | 2018-05-31 | 2018-11-30 | 沈阳东软医疗系统有限公司 | A kind of Lung neoplasm image-recognizing method neural network based and device |
| CN109492547B (en) * | 2018-10-24 | 2022-03-08 | 腾讯科技（深圳）有限公司 | Nodule identification method and device and storage medium |
| CN109685768B (en) * | 2018-11-28 | 2020-11-20 | 心医国际数字医疗系统(大连)有限公司 | Pulmonary nodule automatic detection method and system based on pulmonary CT sequence |
-
2019
- 2019-05-06 CN CN201910374504.2A patent/CN110084810B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101515365A (en) * | 2009-03-25 | 2009-08-26 | 沈阳东软医疗系统有限公司 | Method for automatically separating adherent hyaline-vascular type lung nodule in CT image |
| CN109003260A (en) * | 2018-06-28 | 2018-12-14 | 深圳视见医疗科技有限公司 | CT image pulmonary nodule detection method, device, equipment and readable storage medium storing program for executing |
Non-Patent Citations (1)
| Title |
|---|
| 基于深度卷积神经网络算法的肺结节检测模型;杨晶晶等;《数学建模及其应用》;20171215(第04期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110084810A (en) | 2019-08-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110232383B (en) | Focus image recognition method and focus image recognition system based on deep learning model | |
| WO2020259582A1 (en) | Neural network model training method and apparatus, and electronic device | |
| CN110428432B (en) | Deep neural network algorithm for automatically segmenting colon gland image | |
| CN109118473B (en) | Angular point detection method based on neural network, storage medium and image processing system | |
| US11967181B2 (en) | Method and device for retinal image recognition, electronic equipment, and storage medium | |
| CN109712128B (en) | Feature point detection method, feature point detection device, computer equipment and storage medium | |
| CN110188802B (en) | SSD target detection algorithm based on multi-layer feature map fusion | |
| CN113284149B (en) | COVID-19 chest CT image identification method and device and electronic equipment | |
| CN110084810B (en) | Pulmonary nodule image detection method, model training method, device and storage medium | |
| CN111754453A (en) | Pulmonary tuberculosis detection method and system based on chest radiography image and storage medium | |
| JP6107531B2 (en) | Feature extraction program and information processing apparatus | |
| CN111967464A (en) | Weak supervision target positioning method based on deep learning | |
| CN112102259A (en) | Image segmentation algorithm based on boundary guide depth learning | |
| CN110570394A (en) | medical image segmentation method, device, equipment and storage medium | |
| CN114758137A (en) | Ultrasonic image segmentation method and device and computer readable storage medium | |
| Kumar et al. | Lung parenchyma segmentation: fully automated and accurate approach for thoracic CT scan images | |
| CN113487610B (en) | Herpes image recognition method and device, computer equipment and storage medium | |
| CN116309459A (en) | Improved network-based lung nodule detection method, apparatus, device and storage medium | |
| CN111738975A (en) | Image identification method and image identification device | |
| Basiri et al. | Domain-Specific Deep Learning Feature Extractor for Diabetic Foot Ulcer Detection | |
| CN113160199B (en) | Image recognition method and device, computer equipment and storage medium | |
| CN112634224B (en) | Focus detection method and device based on target image | |
| CN110705570B (en) | Image feature identification method | |
| CN113012189A (en) | Image recognition method and device, computer equipment and storage medium | |
| CN110852377A (en) | Method for identifying pneumonia according to lung X-ray film |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |