CN114373096A - Pulmonary nodule benign and malignant prediction system and method based on multi-feature fusion - Google Patents

Pulmonary nodule benign and malignant prediction system and method based on multi-feature fusion Download PDF

Info

Publication number
CN114373096A
CN114373096A CN202111502379.2A CN202111502379A CN114373096A CN 114373096 A CN114373096 A CN 114373096A CN 202111502379 A CN202111502379 A CN 202111502379A CN 114373096 A CN114373096 A CN 114373096A
Authority
CN
China
Prior art keywords
lung
fusion
benign
image
malignant
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111502379.2A
Other languages
Chinese (zh)
Inventor
乔建苹
范燕玲
颉丽华
杨晓双
姚文龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Normal University
Original Assignee
Shandong Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Normal University filed Critical Shandong Normal University
Priority to CN202111502379.2A priority Critical patent/CN114373096A/en
Publication of CN114373096A publication Critical patent/CN114373096A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30061—Lung
    • G06T2207/30064—Lung nodule

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biomedical Technology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

The invention provides a system and a method for predicting benign and malignant pulmonary nodules based on multi-feature fusion, which belong to the technical field of medical image processing and comprise the following steps: the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a lung CT image of a patient; the second acquisition module is used for acquiring the physical structure attribute characteristics of lung nodules contained in the lung CT image; the extraction module is used for extracting the depth characteristics of the lung CT image; the fusion module is used for fusing the depth characteristic and the physical structure attribute characteristic to obtain a fusion characteristic; and the classification module is used for processing the fusion characteristics to obtain the classification result of the benign and malignant grade of the lung nodule contained in the lung CT image. Aiming at the identification of the benign and malignant pulmonary nodules, the attribute characteristics of the pulmonary nodules are combined with the deep learning characteristics, and the importance of the attribute characteristics in the classification of the benign and malignant pulmonary nodules is determined; has higher classification accuracy, specificity, sensitivity and area under the curve, and provides reliable data for assisting doctors in lung cancer diagnosis and treatment.

Description

Pulmonary nodule benign and malignant prediction system and method based on multi-feature fusion
Technical Field
The invention relates to the technical field of medical image processing, in particular to a system and a method for predicting benign and malignant pulmonary nodules based on multi-feature fusion.
Background
The asymptomatic growth of lung cancer, which results in its inability to be detected and treated early, is a major cause of death in patients. In most cases, the patient is treated before the first symptoms appear, and the extent of the disease has reached an uncontrolled stage. Relevant research data show that if lung cancer symptoms are found at an early stage, the survival rate can be improved to 47%. Therefore, early judgment of the benign and malignant lung nodules is of great significance for successful treatment of lung cancer.
Computed Tomography (CT) is the most effective method of lung nodule detection because it enables the formation of three-dimensional (3D) images of the breast, thereby improving the resolution of nodules and tumor pathology. Generally, a physician will judge the malignancy or benign of a nodule based on the clinical features of the nodule in a CT image. However, it takes a lot of time for the doctor, and sometimes the doctor's experience and subjective factors also affect the accuracy of the judgment. Therefore, how to quickly and accurately identify the lesion in the lung CT image to assist the radiologist in diagnosing the lung disease plays an important role.
At present, there are two main categories of lung nodule identification and classification, one is that manual features are made by using various traditional image processing algorithms based on traditional machine learning methods and then put into a classifier for classification. For example, De Carvalho et al classifies nodes using a Support Vector Machine (SVM) using 5 systematic developmental diversity indicators of dense quadratic entropy, broad quadratic entropy, average classification significance, total classification significance, and pure diversity index. Costa et al used the mean phylogenetic distance gap and the classification diversity index as texture description factors in the study, and then combined the genetic algorithm with SVM to achieve classification. And the other is a classification algorithm based on deep learning, which mainly extracts and classifies the features through various neural networks. For example, Tong et al propose a deep automated pulmonary nodule diagnosis system based on a Three-dimensional Convolutional Neural Network (3D-CNN), a Support Vector Machine (SVM) and a Multi-Kernel Learning algorithm (MKL). Liu et al, propose a new end-to-end deep learning architecture dense convolutional binary tree network (DenseBTNet). The DenseBTNet reserves a tight connection mechanism for extracting lung nodule characteristics of different levels by DenseNet, further strengthens the mechanism to the level of dense blocking, and enriches multi-scale characteristics.
Although the above lung nodule classification methods all achieve good classification effects, the role of attribute features such as appearance of lung nodules on the judgment of benign and malignant lung nodules is not considered, so that the classification result cannot reflect the benign and malignant lung nodules more accurately and objectively.
Disclosure of Invention
The invention aims to provide a multi-feature fusion-based lung nodule benign and malignant degree prediction system and method which extract depth features and attribute features of lung nodules for fusion and more accurately identify the benign and malignant grades of the lung nodules, so as to solve at least one technical problem in the background art.
In order to achieve the purpose, the invention adopts the following technical scheme:
in one aspect, the present invention provides a system for predicting benign and malignant pulmonary nodules based on multi-feature fusion, including:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a lung CT image of a patient;
the second acquisition module is used for acquiring the physical structure attribute characteristics of lung nodules contained in the lung CT image;
the extraction module is used for extracting the depth characteristics of the lung CT image;
the fusion module is used for fusing the depth characteristic and the physical structure attribute characteristic to obtain a fusion characteristic;
and the classification module is used for processing the fusion characteristics to obtain the classification result of the benign and malignant grade of the lung nodule contained in the lung CT image.
In a second aspect, the present invention provides a method for predicting benign and malignant pulmonary nodules based on multi-feature fusion, including:
acquiring a lung CT image of a patient;
acquiring physical structure attribute characteristics of lung nodules contained in the lung CT image;
extracting depth features of the lung CT image;
fusing the depth feature and the physical structure attribute feature to obtain a fused feature;
and processing the fusion characteristics to obtain a classification result of the benign and malignant grade of the lung nodules contained in the lung CT image.
Preferably, the physical structure attribute features include: attribute features include fineness, internal structure, degree of calcification, sphericity, edges, lobular features, burrs, and texture of the lung nodules.
Preferably, the physical structure attributes further include patient age, volume, diameter and surface area of the lung nodule.
Preferably, obtaining the fusion signature comprises: and respectively giving different weights to the attribute characteristics, respectively multiplying the attribute characteristics which are re-assigned with the depth characteristics, and obtaining the most relevant characteristics according to the two groups of multiplied characteristic values to serve as the final fusion characteristics.
Preferably, the two multiplied feature values are respectively processed by a hyperbolic tangent module to obtain the correlation between the depth feature and the attribute feature, and the most correlated feature is output by a Sigmoid module to obtain the fusion feature.
Preferably, the deep neural network used for extracting the depth features of the lung CT image is one of VGG16, VGG19, ResNet50, ResNet101 and MobileNet.
In a third aspect, the present invention provides a non-transitory computer-readable storage medium for storing computer instructions which, when executed by a processor, implement the method for lung nodule benign and malignant prediction based on multi-feature fusion as described above.
In a fourth aspect, the present invention provides a computer program product comprising a computer program for implementing a multi-feature fusion based lung nodule benign and malignant prediction method as described above when run on one or more processors.
In a fifth aspect, the present invention provides an electronic device, comprising: a processor, a memory, and a computer program; wherein a processor is connected to the memory, the computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory to make the electronic device execute the instructions to implement the lung nodule benign and malignant prediction method based on multi-feature fusion as described above.
The invention has the beneficial effects that: aiming at the identification of benign and malignant lung nodules, the attribute features of the lung nodules are combined with the deep learning features, and the importance of the attribute features in the classification of the benign and malignant lung nodules is determined; has higher classification accuracy, specificity, sensitivity and area under the curve, and provides reliable data for assisting doctors in lung cancer diagnosis and treatment.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a flowchart of a method for predicting benign and malignant pulmonary nodules based on multi-feature fusion according to an embodiment of the present invention.
Fig. 2 is a functional schematic block diagram of the F-LSTM module according to the embodiment of the present invention.
Fig. 3 is a flowchart of a comparative experiment implementation method according to an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below by way of the drawings are illustrative only and are not to be construed as limiting the invention.
It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
For the purpose of facilitating an understanding of the present invention, the present invention will be further explained by way of specific embodiments with reference to the accompanying drawings, which are not intended to limit the present invention.
It should be understood by those skilled in the art that the drawings are merely schematic representations of embodiments and that the elements shown in the drawings are not necessarily required to practice the invention.
Example 1
This embodiment 1 provides a system for predicting benign and malignant pulmonary nodules based on multi-feature fusion, the system including:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a lung CT image of a patient;
the second acquisition module is used for acquiring the physical structure attribute characteristics of lung nodules contained in the lung CT image;
the extraction module is used for extracting the depth characteristics of the lung CT image;
the fusion module is used for fusing the depth characteristic and the physical structure attribute characteristic to obtain a fusion characteristic;
and the classification module is used for processing the fusion characteristics to obtain the classification result of the benign and malignant grade of the lung nodule contained in the lung CT image.
In this embodiment 1, a method for predicting benign and malignant pulmonary nodules based on multi-feature fusion is implemented by using the above system for predicting benign and malignant pulmonary nodules based on multi-feature fusion, and the method includes:
acquiring a lung CT image of a patient by using a first acquisition module; acquiring physical structure attribute characteristics of lung nodules contained in the lung CT image by using a second acquisition module; extracting the depth feature of the lung CT image by using an extraction module; fusing the depth feature and the physical structure attribute feature by utilizing a fusion module to obtain a fusion feature; and finally, processing the fusion characteristics by using a classification module to obtain a classification result of the benign and malignant grade of the lung nodules contained in the lung CT image.
Wherein the physical structure attribute characteristics include: attribute features include fineness, internal structure, degree of calcification, sphericity, edges, lobular features, burrs, and texture of the lung nodules. The physical structure attributes also include patient age, volume, diameter, and surface area of the lung nodules.
Obtaining the fusion features includes: and respectively giving different weights to the attribute characteristics, respectively multiplying the attribute characteristics which are re-assigned with the depth characteristics, and obtaining the most relevant characteristics according to the two groups of multiplied characteristic values to serve as the final fusion characteristics. And respectively passing the two multiplied characteristic values through a hyperbolic tangent module to obtain the correlation between the depth characteristic and the attribute characteristic, and outputting the most correlated characteristic through a Sigmoid module to obtain the fusion characteristic.
The deep neural network used for extracting the depth features of the lung CT image is one of VGG16, VGG19, ResNet50, ResNet101 or MobileNet.
Specifically, in this embodiment 1, the training construction process of the system is as follows:
the method comprises the following steps: screening of experimental data from Lung nodule Image Database alliance and Image Database Resource Initiative public Database (LIDC-IDRI, The Lung Image Database Consortium and Image Database Resource Initiative). According to the nodule report of the LIDC-IDRI database, nodules with 4 doctor scores (1-5 grades, 1 represents low malignancy and 5 represents high malignancy) are screened out, and the average malignancy score of each nodule is selected as an experimental label. Nodules with an average score below 3 were marked as benign nodules; nodules with an average score above 3, marked as malignant nodules; nodules with an average score of 3 (indeterminate malignancy or benign) were removed.
Step two: and (3) preprocessing data, namely intercepting the ROI in the original image according to contour coordinate information sketched by a radiologist, and taking out three sections with clear forms. And then sorting the attribute information of the selected lung nodule images at the same time. The attribute features include fineness (Subtlety), Internal Structure (Internal Structure), Calcification (calcium), Sphericity (Sphericity), edge (Margin), Lobulation (distribution), burr (distribution), Texture (Texture)8 features, and 12 features of Age (Age), Volume (Volume), Diameter (Diameter), and Surface area (Surface of mesh) are added.
Step three: in this embodiment 1, 5 types of deep learning networks, namely VGG16, VGG19, ResNet50, ResNet101, and MobileNet, are respectively selected, and the depth features of the last layer of convolutional layer are taken out as image features and are respectively put into an F-LSTM module with attribute features for feature fusion.
And step four, classifying the benign and malignant lung nodules, and entering the fused features into a Softmax layer to perform classified prediction of the benign and malignant lung nodules.
In summary, in this embodiment 1, a classification and prediction algorithm for benign and malignant pulmonary nodules based on deep learning and attribute feature fusion is implemented. Extracting depth features by using a deep learning network, fusing the depth features with attribute features of lung nodules in a fusion mode of F-LSTM (Fuse-Long Short Term Memory), and entering the fused features into a Softmax classification layer to perform precise classification of benign and malignant features.
Example 2
In this embodiment 2, a lung nodule benign and malignant prediction method based on deep learning and attribute feature fusion is provided, and a F-LSTM feature fusion mode is adopted, so that a defect that the importance of attribute features is not maximized in a conventional feature splicing mode is avoided.
In order to verify the effectiveness of the F-LSTM feature fusion method, in this embodiment 2, features extracted by 5 deep convolutional neural networks and attribute features are respectively adopted for fusion, the accuracy is respectively calculated, and classification tasks in the 5 methods all achieve good effects.
In this embodiment 2, a lung nodule benign and malignant classification prediction algorithm based on deep learning and attribute feature fusion is proposed based on superiority of F-LSTM fusion features. As shown in fig. 1, the method specifically includes:
and (3) data preprocessing, namely intercepting the ROI in the region of interest in the original image according to contour coordinate information sketched by a radiologist, and taking out three sections with clear shapes.
The preprocessed ROI is modified to a size of 224 x 3 as input to the deep network. And then, sorting the attribute information of the selected lung nodule image.
The attribute features comprise 8 features of fineness (fineness), Internal Structure (Internal Structure), Calcification (calcium), Sphericity (Sphericity), edge (Margin), Lobulation, burr (Spiculation) and Texture (Texture), and in addition, Age (Age), Volume (Volume), Diameter (Diameter) and Surface area (Surface area of mesh) are added to obtain 12 features.
In the embodiment, 5 deep learning networks, namely VGG16, VGG19, ResNet50, ResNet101 and MobileNet, are selected, and the depth features of the last convolutional layer are taken out to be used as image features. The five models used were pre-trained on the ImageNet dataset and the weights of the network pre-training were collected.
And (4) feature fusion, namely respectively putting the extracted 5 depth features and the attribute features into an F-LSTM module for feature fusion.
And the classification module is used for entering the fused features into a Softmax classification layer for classification of benign and malignant features.
The preprocessing module comprises preprocessing of a lung nodule CT image and preprocessing of attribute features. The preprocessing of the CT image is to derive the nodules which are labeled by four experts together and have the number of slices for segmenting the nodules not less than 3 in an IDRI-LIDC database in a DICOM format, take the average score of scores of the 4 experts on the nodules as a label of data, wherein the nodules with the average score larger than 3 are marked as malignant nodules, the nodules with the average score smaller than 3 are marked as benign nodules, and the nodules with the average score of 3 are removed.
The attribute feature preprocessing means that scores of 12 attribute features by 4 experts are respectively taken out, and the average of the scores of the 4 experts is taken as a feature value of the attribute features.
In the depth feature extraction module, in order to verify the effectiveness of the F-LSTM fusion mode, 5 kinds of depth learning networks are used for extracting the depth features of the lung nodule images.
In the feature fusion module, the specific way of F-LSTM fusion is as shown in FIG. 2, firstly, the attribute features are passed through FbAnd gbThe two functions are endowed with different weights, then the attribute characteristics which are re-assigned by the two functions are respectively multiplied by the image characteristics extracted by the deep learning network, then two groups of characteristic values after multiplication are respectively passed through a hyperbolic tangent (tanh) module, and the basic principle behind the tanh module is that the value of each characteristic is modified into a range [ -1,1]To increase or decrease its correlation, where 1 is the most correlated and-1 is the least correlated. And finally, outputting the most relevant characteristics through the two groups of characteristic values of the tanh module and the Sigmoid module. The basic principle of the Sigmoid module is similar to that of the tanh module, scaling the eigenvalues to [0,1 ]]Where 0 represents the least relevant feature and 1 is the opposite. The basic idea of the Sigmoid module is to zero out the uncorrelated features and output the most correlated ones.
Wherein f isbAnd gbThe expressions for both functions are as follows:
Figure BDA0003402259140000091
Figure BDA0003402259140000092
wherein f isbAnd gbWhat is indicated is the weight of the network,
Figure BDA0003402259140000093
and
Figure BDA0003402259140000094
represents a transpose of a matrix;
Figure BDA0003402259140000095
and
Figure BDA0003402259140000096
indicated is the bias of the network;
Figure BDA0003402259140000097
representing the attribute characteristics of the lung nodule. In the iterative process, the weight and bias of the network and thus the weight of the attribute feature may change. After the network is trained, the classification effect is improved, the weight of the attribute features is enhanced, and the model can be helped to be concentrated on more important features.
And the classification module is used for entering the fused features into a Softmax classification layer to perform benign and malignant classification.
In this embodiment, the optimizer used for network training is random Gradient Descent (SGD), the Learning Rate (Learning Rate) is set to 0.001, the Momentum (Momentum) is set to 0.9, the Weight attenuation (Weight _ Decay) is 0.001, and the iteration number (Epoch) is 150 times. In addition, if the model is not optimized in 10 periods, the learning rate is reduced by 0.1; if the model is not optimized within 15 cycles, an Early Stopping (Early Stopping) strategy is adopted. For all experiments in this example, 5-fold cross-validation layered by tag frequency was applied to evaluate the effectiveness of the model. To measure the classification performance, the following evaluation indices Accuracy (ACC), Specificity (SPE), Sensitivity (SEN), area under the curve (AUC) were calculated.
The classification accuracy and various evaluation indexes of the image features extracted by the five depth models and the attribute features in the F-LSTM-based fusion mode are shown in Table 1.
TABLE 1
CNN Model ACC SPE SEN AUC
VGG16 91.2％ 80.4％ 97.8％ 94.5％
VGG19 91.2％ 85.7％ 97.8％ 96.2％
ResNet101 91.2％ 76.7％ 1 89.3％
MobileNet 92.2％ 91.1％ 95.7％ 98.4％
ResNet50 94.1％ 91.2％ 97.8％ 98.1％
It can be seen that when ResNet50 is used for extracting image features, the classification accuracy of the model is highest, and meanwhile, the Specificity (SPE), Sensitivity (SEN) and area under the curve (AUC) indexes all reach good levels.
To verify the validity of the solution of the present embodiment, a comparison is made with other methods.
The comparison method comprises the following steps: and (4) feature fusion based on the traditional splicing mode. The image features extracted by the 5 deep learning models are respectively spliced with the attribute features, the spliced features are sent to a Softmax classification layer for classification of benign and malignant conditions, and a specific flow chart is shown in FIG. 3.
The classification accuracy and various evaluation indexes of the image features extracted by the five depth models and the attribute features in a traditional feature splicing-based fusion mode are shown in table 2.
TABLE 2
CNN Model ACC SPE SEN AUC
VGG16 85.3％ 73.2％ 1 89.9％
VGG19 86.4％ 76.7％ 97.8％ 87.2％
ResNet101 87.3％ 76.8％ 1 89.8％
MobileNet 85.4％ 89.9％ 1 88.3％
ResNet50 88.2％ 83.9％ 95.7％ 91.5％
It can be seen that the classification Accuracy (ACC) and the area under the curve (AUC) based on the traditional feature splicing fusion mode are lower than those of the method provided by the invention. The method of the embodiment has a good effect in the classification task of the benign and malignant lung nodules.
Example 3
Embodiment 3 of the present invention provides a non-transitory computer readable storage medium for storing computer instructions, which when executed by a processor, implement the method for predicting benign and malignant pulmonary nodule based on multi-feature fusion, as described above, the method including:
acquiring a lung CT image of a patient;
acquiring physical structure attribute characteristics of lung nodules contained in the lung CT image;
extracting depth features of the lung CT image;
fusing the depth feature and the physical structure attribute feature to obtain a fused feature;
and processing the fusion characteristics to obtain a classification result of the benign and malignant grade of the lung nodules contained in the lung CT image.
Example 4
Embodiment 4 of the present invention provides a computer program (product) comprising a computer program, which when run on one or more processors, is configured to implement the method for predicting benign and malignant pulmonary nodule based on multi-feature fusion as described above, the method comprising:
acquiring a lung CT image of a patient;
acquiring physical structure attribute characteristics of lung nodules contained in the lung CT image;
extracting depth features of the lung CT image;
fusing the depth feature and the physical structure attribute feature to obtain a fused feature;
and processing the fusion characteristics to obtain a classification result of the benign and malignant grade of the lung nodules contained in the lung CT image.
Example 5
An embodiment 5 of the present invention provides an electronic device, including: a processor, a memory, and a computer program; wherein a processor is connected with the memory, a computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory to make the electronic device execute instructions for implementing the lung nodule benign and malignant prediction method based on multi-feature fusion as described above, the method includes:
acquiring a lung CT image of a patient;
acquiring physical structure attribute characteristics of lung nodules contained in the lung CT image;
extracting depth features of the lung CT image;
fusing the depth feature and the physical structure attribute feature to obtain a fused feature;
and processing the fusion characteristics to obtain a classification result of the benign and malignant grade of the lung nodules contained in the lung CT image.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Although the embodiments of the present invention have been described with reference to the accompanying drawings, it is not intended to limit the scope of the present invention, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive efforts based on the technical solutions disclosed in the present invention.

Claims (10)

1. A system for predicting benign and malignant pulmonary nodules based on multi-feature fusion, comprising:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a lung CT image of a patient;
the second acquisition module is used for acquiring the physical structure attribute characteristics of lung nodules contained in the lung CT image;
the extraction module is used for extracting the depth characteristics of the lung CT image;
the fusion module is used for fusing the depth characteristic and the physical structure attribute characteristic to obtain a fusion characteristic;
and the classification module is used for processing the fusion characteristics to obtain the classification result of the benign and malignant grade of the lung nodule contained in the lung CT image.
2. A lung nodule benign and malignant prediction method based on multi-feature fusion is characterized by comprising the following steps:
acquiring a lung CT image of a patient;
acquiring physical structure attribute characteristics of lung nodules contained in the lung CT image;
extracting depth features of the lung CT image;
fusing the depth feature and the physical structure attribute feature to obtain a fused feature;
and processing the fusion characteristics to obtain a classification result of the benign and malignant grade of the lung nodules contained in the lung CT image.
3. The method for predicting benign and malignant pulmonary nodules based on multi-feature fusion according to claim 2, wherein the physical structure attribute features comprise: attribute features include fineness, internal structure, degree of calcification, sphericity, edges, lobular features, burrs, and texture of the lung nodules.
4. The method of claim 3, wherein the physical structure attributes further include patient age, volume, diameter and surface area of the lung nodule.
5. The method for predicting benign and malignant pulmonary nodules based on multi-feature fusion according to claim 2, wherein obtaining the fusion features comprises: and respectively giving different weights to the attribute characteristics, respectively multiplying the attribute characteristics which are re-assigned with the depth characteristics, and obtaining the most relevant characteristics according to the two groups of multiplied characteristic values to serve as the final fusion characteristics.
6. The method for predicting benign and malignant pulmonary nodules based on multi-feature fusion as claimed in claim 5, wherein the two groups of feature values after multiplication are respectively passed through a hyperbolic tangent module to obtain the correlation between depth features and attribute features, and the most correlated features are output through a Sigmoid module to obtain the fusion features.
7. The method of claim 2, wherein the deep neural network used for extracting the deep features of the lung CT image is one of VGG16, VGG19, ResNet50, ResNet101 and MobileNet.
8. A non-transitory computer readable storage medium storing computer instructions which, when executed by a processor, implement the multi-feature fusion based lung nodule benign and malignant prediction method of any one of claims 2-7.
9. A computer program product comprising a computer program for implementing a multi-feature fusion based lung nodule benign and malignant prediction method as claimed in any one of claims 2 to 7 when run on one or more processors.
10. An electronic device, comprising: a processor, a memory, and a computer program; wherein a processor is connected with the memory, a computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory to make the electronic device execute the instructions to implement the lung nodule benign and malignant prediction method based on multi-feature fusion according to any one of claims 2-7.
CN202111502379.2A 2021-12-09 2021-12-09 Pulmonary nodule benign and malignant prediction system and method based on multi-feature fusion Pending CN114373096A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111502379.2A CN114373096A (en) 2021-12-09 2021-12-09 Pulmonary nodule benign and malignant prediction system and method based on multi-feature fusion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111502379.2A CN114373096A (en) 2021-12-09 2021-12-09 Pulmonary nodule benign and malignant prediction system and method based on multi-feature fusion

Publications (1)

Publication Number Publication Date
CN114373096A true CN114373096A (en) 2022-04-19

Family

ID=81140868

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111502379.2A Pending CN114373096A (en) 2021-12-09 2021-12-09 Pulmonary nodule benign and malignant prediction system and method based on multi-feature fusion

Country Status (1)

Country Link
CN (1) CN114373096A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115578307A (en) * 2022-05-25 2023-01-06 广州市基准医疗有限责任公司 Method for classifying benign and malignant pulmonary nodules and related products

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115578307A (en) * 2022-05-25 2023-01-06 广州市基准医疗有限责任公司 Method for classifying benign and malignant pulmonary nodules and related products
CN115578307B (en) * 2022-05-25 2023-09-15 广州市基准医疗有限责任公司 Lung nodule benign and malignant classification method and related products

Similar Documents

Publication Publication Date Title
Zhang et al. Automatic nodule detection for lung cancer in CT images: A review
Sun et al. Brain tumor segmentation and survival prediction using multimodal MRI scans with deep learning
Sun et al. Deep learning-based classification of liver cancer histopathology images using only global labels
Al-Antari et al. Deep learning computer-aided diagnosis for breast lesion in digital mammogram
Halder et al. Lung nodule detection from feature engineering to deep learning in thoracic CT images: a comprehensive review
Wang et al. A multi-view deep convolutional neural networks for lung nodule segmentation
Gao et al. Classification of CT brain images based on deep learning networks
Javaid et al. A novel approach to CAD system for the detection of lung nodules in CT images
Zhang et al. Classification of lung nodules based on CT images using squeeze-and-excitation network and aggregated residual transformations
Zhang et al. NODULe: Combining constrained multi-scale LoG filters with densely dilated 3D deep convolutional neural network for pulmonary nodule detection
Monkam et al. CNN models discriminating between pulmonary micro-nodules and non-nodules from CT images
ur Rehman et al. An appraisal of nodules detection techniques for lung cancer in CT images
Maurya et al. A review on recent developments in cancer detection using Machine Learning and Deep Learning models
CN111767952B (en) Interpretable lung nodule benign and malignant classification method
Mastouri et al. Deep learning-based CAD schemes for the detection and classification of lung nodules from CT images: A survey
CN104268873A (en) Breast tumor partition method based on nuclear magnetic resonance images
Niyaz et al. Advances in deep learning techniques for medical image analysis
Osadebey et al. Three-stage segmentation of lung region from CT images using deep neural networks
Gao et al. Multi-label deep regression and unordered pooling for holistic interstitial lung disease pattern detection
Tang et al. NSCGCN: A novel deep GCN model to diagnosis COVID-19
Soewu et al. Lung Cancer Detection using Image Processing
CN114373096A (en) Pulmonary nodule benign and malignant prediction system and method based on multi-feature fusion
Zhang et al. A 3D+ 2D CNN approach incorporating boundary loss for stroke lesion segmentation
Kwong et al. A survey on deep learning approaches for breast cancer diagnosis
Shao et al. A segmentation method of airway from chest CT image based on VGG-Unet neural network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination