CN110084813B - Lung nodule benign and malignant prediction method based on three-dimensional deep learning network - Google Patents

Lung nodule benign and malignant prediction method based on three-dimensional deep learning network Download PDF

Info

Publication number
CN110084813B
CN110084813B CN201910392053.5A CN201910392053A CN110084813B CN 110084813 B CN110084813 B CN 110084813B CN 201910392053 A CN201910392053 A CN 201910392053A CN 110084813 B CN110084813 B CN 110084813B
Authority
CN
China
Prior art keywords
network
resolution
lung
channel
lung nodule
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910392053.5A
Other languages
Chinese (zh)
Other versions
CN110084813A (en
Inventor
董恩清
熊文硕
纪惠中
金叶
倪天骄
薛鹏
韩贺
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Zhiwei Zhicheng Electronic Technology Co ltd
Original Assignee
Shandong Zhiwei Zhicheng Electronic Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Zhiwei Zhicheng Electronic Technology Co ltd filed Critical Shandong Zhiwei Zhicheng Electronic Technology Co ltd
Priority to CN201910392053.5A priority Critical patent/CN110084813B/en
Publication of CN110084813A publication Critical patent/CN110084813A/en
Application granted granted Critical
Publication of CN110084813B publication Critical patent/CN110084813B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30061—Lung
    • G06T2207/30064—Lung nodule

Landscapes

  • Engineering & Computer Science (AREA)
  • Quality & Reliability (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

A lung nodule benign and malignant prediction method based on a three-dimensional deep learning network belongs to the field of image processing. The method comprises the following steps: lung CT image preprocessing and generation of multi-resolution input data; constructing a multi-resolution 3D dual-channel compression excitation deep learning network model; and (5) iteratively solving the model parameters by using a gradient descent method. The invention adopts a deep learning network model combining the ideas of a dual-channel network and a compression excitation network, can not only repeatedly utilize the low-order characteristics of the lung nodule image and continuously generate new high-order combined characteristics, but also can recalibrate the weight of the characteristic channel and effectively describe the importance degree of different characteristic channels to network output. The 3D multi-resolution data processing method adopted by the invention can effectively solve the problems of boundary loss, noise and the like caused by the inconsistent diameters of lung nodules.

Description

Lung nodule benign and malignant prediction method based on three-dimensional deep learning network
Technical Field
The invention belongs to the field of medical image processing, and particularly relates to lung nodule benign and malignant prediction based on a multi-resolution 3D dual-channel compression excitation network.
Background
The accurate diagnosis and prediction of benign and malignant lung nodules is an important research direction for accurate medical treatment, and has very important significance in lung CT image analysis and practical clinical application. The traditional method for diagnosing the benign and malignant pulmonary nodules through manual interpretation not only excessively depends on the diagnosis level of doctors and has strong subjectivity, but also brings huge workload. The deep learning method can deeply mine complex implicit representation of image data, and can further analyze and process deeper features, so that classification or prediction is more intelligent. Deep learning has many representative algorithms: the Dual Path Network (DPN) is proposed by the pigment group, which merges core ideas of Deep Residual Networks (reset) and dense Connected Networks (densneet) of two symbolic Deep learning Networks, and the densneet can extract new features from previous levels, while the rent is essentially a reuse of extracted features in previous levels. The compression-and-Excitation Networks (features) is a new network structure proposed by the kingdom team of Momenta corporation, which starts from another point of view, considers explicit modeling of the relationship between feature channels in the network, analyzes the dependency relationship and interaction between feature channels emphatically, adopts a "feature recalibration" mode to automatically learn the importance degree of each feature channel in the current task, and appropriately strengthens or inhibits the influence of the current channel on the final result according to the degree value. However, these methods are not fully applied to the field of 3D medical image processing, which is limited to the problems of data set and complexity, and these problems are also the key problems to be faced and urgently solved in the current research.
Disclosure of Invention
The method aims at the problems that the traditional manual method for diagnosing benign and malignant pulmonary nodules excessively depends on the diagnosis level of doctors, has strong subjectivity, brings huge workload and the like, and the traditional algorithm cannot effectively process the pulmonary nodules with different morphological changes and the like.
The invention provides a lung nodule benign and malignant prediction method based on a three-dimensional deep learning network, which comprises the following steps of:
s1, carrying out gray level reconstruction and layer thickness reconstruction on the lung CT image;
s2, extracting and storing the 3D lung nodule data according to the size of the center O and the diameter D of the lung nodule area, and screening out images with boundaries exceeding the image range or marked with non-standard images;
s3, generating data of three corresponding resolutions through a B spline interpolation algorithm;
s4, standardizing and sample expanding the data in the step S3 to be used as training network input;
s5, constructing a multi-resolution 3D dual-channel compression excitation network;
and S6, training and iteratively solving the data in the step S4 corresponding to the input network by adopting a random gradient descent method.
And S7, performing classification accuracy verification.
The invention has the beneficial effects that: a doctor for diagnosing the benign and malignant lung nodules only needs to confirm the center and the size position of the lung nodules in the lung CT image and can obtain a prediction result of the network model after inputting the lung nodules into the network. Compared with the manual diagnosis, the method has higher real-time performance, accuracy and robustness, can be well suitable for CT images with different specifications and standards, and has good expandability.
Drawings
Fig. 1 is a schematic diagram of a multi-resolution 3D dual-channel compressed excitation network.
Fig. 2 is a schematic flow chart of lung CT image preprocessing.
Fig. 3 is a schematic diagram of multi-resolution 3D data processing.
Fig. 4 is a schematic structural diagram of a two-path compression excitation unit (DPSE unit).
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and specific embodiments. The invention discloses a lung nodule benign and malignant prediction method based on a three-dimensional deep learning network, which mainly comprises the following steps:
s1, carrying out gray level reconstruction and layer thickness reconstruction on the lung CT image;
read-in DICOM format lung CT image training sample T1And test specimen T2And converted into a uint16 type, and its gray scale is reconstructed to [ -1000,400 ]]HU (Hounsfield Unit), the layer thickness is reconstructed to 1mm using nearest neighbor interpolation, as shown in FIG. 2.
S2, extracting and storing the 3D lung nodule data according to the size of the center O and the diameter D of the lung nodule area, and screening out images with boundaries exceeding the image range or labeling irregular images.
Artificially determining training sample T1And test specimen T2And D is equal to or larger than 3mm and equal to or smaller than 14mm, and the corresponding pixel extraction window is 20 x 20. 14mm<D is less than or equal to 21mm, the resolution is medium, and the corresponding pixel extraction window is 30 × 30. 21mm<D is less than or equal to 30mm, the resolution is high, and the corresponding pixel extraction window is 45 x 45. Lung nodules with diameters outside the three grades were all screened out.
S3, generating data of three corresponding resolutions through a B spline interpolation algorithm;
centering on O, training sample T1And test specimen T2And 3D image extraction is carried out on the center of the lung nodule according to the pixel extraction window, and 3D images with the other two resolutions are generated through B-spline interpolation, as shown in figure 3, and are correspondingly stored.
S4, standardizing and sample expanding the data in the step S3 to be used as training network input;
the stored data were normalized to 0-1. Will T1The data in the method adopts X-axis, Y-axis and Z-axis rotation and 2-pixel random translation to perform data expansion so as to solve the problem of sample imbalance, and the rotation angle is [90 degrees, 180 degrees and 270 degrees °]Three special angles. The augmented 3D data is retained as training network input.
S5, constructing a multi-resolution 3D dual-channel compression excitation network;
and constructing a multi-resolution 3D dual-channel compression excitation network according to the architecture of the whole network in FIG. 4 and the dual-channel compression excitation (DPSE) unit in FIG. 1. In fig. 1, GN is group normalization, leakyreu is leakrelu activation function, Conv denotes convolution operation, ReLU is ReLU activation function, and Sigmoid is Sigmoid activation function. X1lAnd X2lNetwork inputs of a residual connecting path and a dense connecting path of the l layer are respectively shown, wherein the basic channel of the dense connecting path is 32, the increment is 16, and the dimensionality reduction ratio in the compression excitation module is set to be 4. In fig. 4, 64-3 × 3 above the arrow indicates that the current layer is convolved with 64 convolution kernels having dimensions of 3 × 3, 10 × 10@ (256+64) below the feature cube indicates that the current layer output is 10 × 10, the number of feature channels of the residual connecting channels is 256, the number of channels of the dense connecting channels is 64, and the rest is similar. Maxpooling3D indicates maximum pooling of 3D data, and Softmax is the Softmax classifier. The whole network adopts three resolution networks, the network input pixel dimensions are respectively a low resolution network 20 × 20 × 20, a middle resolution network 30 × 30 × 30 and a high resolution network 45 × 45 × 45, the main body frame of each resolution network is formed by stacking dual-channel compression excitation (DPSE) units in the graph 1, and the whole network construction parameters are given in a table 1.
In FIG. 4, the tail-end classification scaling factor is λ1＝0.5,λ2＝0.3,λ30.2. Probability P of each tail end class1、P2、P3Given by:
Figure BDA0002056835820000041
wherein N is given a correspondence of input sample and true tag value { (D)1,y1),K,(DN,yN)}，DjFor the jth input sample, yjIs DjThe corresponding real label is marked with a real label,
Figure BDA0002056835820000042
then the result of the network prediction, k is the number of classes, ηiIs a parameter of the i-th resolution network. Has calculated PiThen, at the tail end, by a scaling factor lambda1、λ2、λ3Performing probability fusion to obtain fusion probability PfAs follows:
Figure BDA0002056835820000043
TABLE 1 detailed structure of multi-resolution 3D dual-channel compression excitation network
Figure BDA0002056835820000051
In table 1, C represents a convolutional layer, DPSE-C represents a convolutional layer in a dual-channel compression excitation module, FC represents a full link layer, the former of the summation terms of the channel numbers is the channel number of residual link channels, and the latter is the channel number of dense link channels.
And S6, training and iteratively solving the data in the step S4 corresponding to the input network by adopting a random gradient descent method.
Step 3, using a Stochastic Gradient Descent (SGD) method to reduce T1Is trained in response to the input networkAnd iteratively solving. The network parameter initialization is performed by using He, the learning rate is set to be 0.005, the learning rate is reduced by 30% after each 300 iteration, the size of the network batch (batch size) is 8, and the loss function is selected from cross entropy loss functions. And judging whether the algorithm is converged according to the accuracy rate convergence trend and the number of iteration rounds, and if not, storing the network parameters until the algorithm is converged.
And S7, performing classification accuracy verification.
After the network is converged, the stored parameters are loaded into the network and T is input2And (5) carrying out classification accuracy verification on the data.
Finally, it should be noted that the above embodiments are only used for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, the protection scope of the present invention is not limited thereto, and any modifications or equivalent substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention disclosed herein are all covered within the protection scope of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims (6)

1. A lung nodule benign and malignant prediction method based on a three-dimensional deep learning network is characterized by comprising the following steps:
s1, carrying out gray level reconstruction and layer thickness reconstruction on the lung CT image;
s2, extracting and storing the 3D lung nodule data according to the size of the center O and the diameter D of the lung nodule area, and screening out images with boundaries exceeding the image range or marked with non-standard images;
s3, generating data of three corresponding resolutions through a B spline interpolation algorithm;
s4, standardizing and sample expanding the data in the step S3 to be used as training network input;
s5, constructing a multi-resolution 3D dual-channel compression excitation network;
s6, training and iteratively solving the data in the step S4 corresponding to the input network by adopting a random gradient descent method;
s7, verifying the classification accuracy;
multi-resolution 3D dual channel pressureThe reduced excitation network is constructed by stacking two-channel compressed excitation units, the structure and parameters of the reduced excitation network are shown in table 1, wherein the basic channels of the dense connecting channels are 32, the basic channels are increased by 16, the dimensionality reduction ratio in the compressed excitation module is set to be 4, and the tail end classification proportionality coefficient is lambda1＝0.5,λ2＝0.3,λ30.2, probability P of each tail end class1、P2、P3Given by:
Figure FDA0003531105520000011
wherein N is given a correspondence of input sample and true tag value { (D)1,y1),...,(DN,yN)}，DjFor the jth input sample, yjIs DjThe corresponding real label is marked with a real label,
Figure FDA0003531105520000012
then the result of the network prediction, k is the number of classes, ηiFor the parameter of the ith resolution network, the respective prediction probabilities P of the three resolution networks are calculatediThen, at the tail end, by a scaling factor lambda1、λ2、λ3Performing probability fusion to obtain fusion probability PfAs follows:
Figure FDA0003531105520000013
TABLE 1 detailed structure of multi-resolution 3D dual-channel compression excitation network
Figure FDA0003531105520000021
In table 1, C represents a convolutional layer, DPSE-C represents a convolutional layer in a dual-channel compression excitation module, FC represents a full link layer, the former of the summation terms of the channel numbers is the channel number of residual link channels, and the latter is the channel number of dense link channels.
2. The method for predicting benign and malignant lung nodules according to claim 1, wherein said step S1 comprises using a read-in DICOM-format lung CT image training sample T1And test specimen T2And converted into a uint16 type, and its gray scale is reconstructed to [ -1000,400 ]]HU, layer thickness was reconstructed to 1mm using nearest neighbor interpolation.
3. The method for predicting benign and malignant pulmonary nodules according to claim 1, wherein said step S2 comprises manually determining a training sample T1And test specimen T2The center O and the diameter D of the middle lung nodule are large, the lung nodule is divided into three resolutions according to the diameter D, if the D is more than or equal to 3mm and less than or equal to 14mm, the resolution is low, and the corresponding pixel extraction window is 20 × 20; d is more than 14mm and less than or equal to 21mm, the resolution is medium, and the corresponding pixel extraction window is 30 × 30; d is more than 21mm and less than or equal to 30mm, the high resolution is obtained, and the corresponding pixel extraction window is 45 × 45.
4. The method for predicting lung nodule malignancy and well, according to claim 1, wherein the step S3 is to train a sample T around O1And test specimen T2And 3D image extraction is carried out on the center of the lung nodule according to the pixel extraction window, and 3D images with the other two resolutions are generated through B spline interpolation and are correspondingly stored to form input data.
5. The method for predicting lung nodule malignancy and well, according to claim 1, wherein in step S4, the input data is normalized by 0-1 to obtain a training sample T1The data in the system are subjected to data expansion by adopting X-axis, Y-axis and Z-axis rotation and 2-pixel random translation, and the rotation angle is [90 degrees, 180 degrees and 270 degrees °]Three special angles.
6. The method for predicting the benign and malignant pulmonary nodule of claim 1, wherein in S6, He is used for initializing network parameters, the learning rate is set to 0.005, 30% reduction is performed every 300 iterations, the size of the network batch batchsize is 8, the loss function is a cross entropy loss function, whether the algorithm converges or not is judged according to the convergence trend of the accuracy rate and the number of iterations, and if not, the network parameters are stored until the convergence.
CN201910392053.5A 2019-05-13 2019-05-13 Lung nodule benign and malignant prediction method based on three-dimensional deep learning network Active CN110084813B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910392053.5A CN110084813B (en) 2019-05-13 2019-05-13 Lung nodule benign and malignant prediction method based on three-dimensional deep learning network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910392053.5A CN110084813B (en) 2019-05-13 2019-05-13 Lung nodule benign and malignant prediction method based on three-dimensional deep learning network

Publications (2)

Publication Number Publication Date
CN110084813A CN110084813A (en) 2019-08-02
CN110084813B true CN110084813B (en) 2022-05-13

Family

ID=67419861

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910392053.5A Active CN110084813B (en) 2019-05-13 2019-05-13 Lung nodule benign and malignant prediction method based on three-dimensional deep learning network

Country Status (1)

Country Link
CN (1) CN110084813B (en)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110517253B (en) * 2019-08-29 2023-04-07 电子科技大学 Method for classifying benign and malignant pulmonary nodules based on 3D multi-target feature learning
CN110766051A (en) * 2019-09-20 2020-02-07 四川大学华西医院 Lung nodule morphological classification method based on neural network
CN111008570B (en) * 2019-11-11 2022-05-03 电子科技大学 Video understanding method based on compression-excitation pseudo-three-dimensional network
CN111369546B (en) * 2020-03-10 2023-07-18 北京邮电大学 Cervical lymph node image classification and identification device and method
CN111584073B (en) * 2020-05-13 2023-05-09 山东大学 Method for constructing diagnosis models of benign and malignant lung nodules in various pathological types
CN112232433B (en) * 2020-10-27 2022-08-26 河北工业大学 Lung nodule benign and malignant classification method based on dual-channel network
CN116012355B (en) * 2023-02-07 2023-11-21 重庆大学 Adaptive false positive lung nodule removing method based on deep learning
CN116739890B (en) * 2023-06-26 2024-06-14 强联智创（北京）科技有限公司 Method and equipment for training generation model for generating healthy blood vessel image

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106127725B (en) * 2016-05-16 2019-01-22 北京工业大学 A kind of millimetre-wave radar cloud atlas dividing method based on multiresolution CNN
CN107016665B (en) * 2017-02-16 2021-05-04 浙江大学 CT pulmonary nodule detection method based on deep convolutional neural network
CN107154043B (en) * 2017-06-05 2020-03-24 杭州健培科技有限公司 Pulmonary nodule false positive sample inhibition method based on 3DCNN
US10692211B2 (en) * 2017-06-20 2020-06-23 Case Western Reserve University Intra-perinodular textural transition (IPRIS): a three dimenisonal (3D) descriptor for nodule diagnosis on lung computed tomography (CT) images
CN108288271A (en) * 2018-02-06 2018-07-17 上海交通大学 Image detecting system and method based on three-dimensional residual error network
CN109544510B (en) * 2018-10-24 2021-10-26 广州大学 Three-dimensional lung nodule identification method based on convolutional neural network

Also Published As

Publication number Publication date
CN110084813A (en) 2019-08-02

Similar Documents

Publication Publication Date Title
CN110084813B (en) Lung nodule benign and malignant prediction method based on three-dimensional deep learning network
CN110189334B (en) Medical image segmentation method of residual error type full convolution neural network based on attention mechanism
CN113779675B (en) Physical-data driven intelligent shear wall building structure design method and device
CN109711426B (en) Pathological image classification device and method based on GAN and transfer learning
CN111584073B (en) Method for constructing diagnosis models of benign and malignant lung nodules in various pathological types
CN110992252B (en) Image multi-grid conversion method based on latent variable feature generation
CN111369563A (en) Semantic segmentation method based on pyramid void convolutional network
CN110657984B (en) Planetary gearbox fault diagnosis method based on reinforced capsule network
CN110533683B (en) Image omics analysis method fusing traditional features and depth features
CN111899168B (en) Remote sensing image super-resolution reconstruction method and system based on feature enhancement
CN110930378A (en) Emphysema image processing method and system based on low data demand
CN116091490A (en) Lung nodule detection method based on YOLOv4-CA-CBAM-K-means++ -SIOU
CN116628592A (en) Dynamic equipment fault diagnosis method based on improved generation type countering network
Tang et al. Classification of benign and malignant pulmonary nodules based on the multiresolution 3D DPSECN model and semisupervised clustering
CN113496481A (en) Auxiliary detection method for chest X-Ray image with few samples
CN115661029A (en) Pulmonary nodule detection and identification system based on YOLOv5
CN113344933B (en) Glandular cell segmentation method based on multi-level feature fusion network
CN114187506A (en) Remote sensing image scene classification method of viewpoint-aware dynamic routing capsule network
CN113208641A (en) Pulmonary nodule auxiliary diagnosis method based on three-dimensional multi-resolution attention capsule network
Zhang et al. Multicontext 3D residual CNN for false positive reduction of pulmonary nodule detection
CN117095199A (en) Industrial visual anomaly detection system based on simplex diffusion model
CN115631847B (en) Early lung cancer diagnosis system, storage medium and equipment based on multiple groups of chemical characteristics
CN116432053A (en) Multi-mode data representation method based on modal interaction deep hypergraph neural network
CN114913164A (en) Two-stage weak supervision new crown lesion segmentation method based on super pixels
CN110852167A (en) Remote sensing image classification method based on optimal activation model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right

Effective date of registration: 20220425

Address after: Room 913, building a, innovation and entrepreneurship base, 213 Torch Road, gaoqu District, Weihai City, Shandong Province, 264200

Applicant after: Shandong Zhiwei Zhicheng Electronic Technology Co.,Ltd.

Address before: 264209 No. 180 Wenhua West Road, gaoqu, Huancui District, Weihai City, Shandong Province

Applicant before: SHANDONG University

TA01 Transfer of patent application right
GR01 Patent grant
GR01 Patent grant