CN113095382B - Interpretable tuberculosis classification network identification method based on CT image - Google Patents
Interpretable tuberculosis classification network identification method based on CT image Download PDFInfo
- Publication number
- CN113095382B CN113095382B CN202110337007.2A CN202110337007A CN113095382B CN 113095382 B CN113095382 B CN 113095382B CN 202110337007 A CN202110337007 A CN 202110337007A CN 113095382 B CN113095382 B CN 113095382B
- Authority
- CN
- China
- Prior art keywords
- layer
- image
- tuberculosis
- interpretable
- lung
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
- Image Analysis (AREA)
- Apparatus For Radiation Diagnosis (AREA)
Abstract
The invention discloses an interpretable tuberculosis classification network identification method based on CT images, which comprises the following steps: acquiring a Computed Tomography (CT) image of the lung of a person to be detected, and carrying out normalization processing and scaling on the lung parenchyma image; establishing and training an interpretable tuberculosis classification network, wherein the interpretable tuberculosis classification network is established by taking a residual error network as a basic network and adding a Dense thought and an improved attention mechanism; and (4) sending the lung parenchymal image subjected to normalization processing and scaled to 512x512 size into a trained interpretable tuberculosis classification network, and obtaining and outputting a classification result of the tuberculosis of the person to be detected and a corresponding class activation thermodynamic diagram in an upper computer. The invention can realize the classification result of the pulmonary tuberculosis of the person to be detected and output the corresponding class activation thermodynamic diagram according to the CT image of the person to be detected.
Description
Technical Field
The invention relates to the field of computer-aided diagnosis and treatment and image identification, in particular to an interpretable tuberculosis classification network identification method based on a CT image.
Background
Tuberculosis, which is one of the major health-threatening infectious diseases, is currently represented as pulmonary tuberculosis. Tuberculosis is a chronic infectious disease which harms human health and has a long history, even causes death of millions of people all over the world in the early 20 th century with the most serious epidemic, and at present, China is one of high-load countries of pulmonary tuberculosis epidemic situations. The tuberculosis has the characteristics of high infectivity and lethality, and the early auxiliary diagnosis of the tuberculosis can help doctors to find early tuberculosis patients, perform early diagnosis and early treatment and reduce the infection rate and the death rate, so the method has important clinical significance.
CT image examination has become the first choice for no alternatives in tuberculosis diagnosis at present. Because the pulmonary tuberculosis has the characteristics of focus polymorphism, multiple parts, multiple joints, cavities and the like in the CT image, the multiple forms are mixed and difficult to identify, and medical workers need to read a large number of CT image sequences of the lungs of different patients, and the medical workers are easy to fatigue under high-intensity working pressure, thereby causing missed diagnosis and even misdiagnosis. Therefore, a processing method based on a CT image is needed to improve the accuracy of diagnosis of doctors, reduce the rate of missed diagnosis, and reduce the work intensity of medical staff.
Disclosure of Invention
The invention aims to provide an interpretable tuberculosis classification network identification method based on a CT image.
In order to solve the technical problem, the invention provides an interpretable tuberculosis classification network identification method based on a CT image, which comprises the following steps:
s01, acquiring a Computed Tomography (CT) image of the lung of a person (such as a patient) to be detected, taking the image as an original CT image, inputting the original CT image into an upper computer for image preprocessing, extracting lung parenchyma to obtain a lung parenchyma image, and then performing normalization processing on the lung parenchyma image and zooming the lung parenchyma image to 512x512 size;
s02, establishing and training an interpretable tuberculosis classification network, wherein the interpretable tuberculosis classification network is established by taking a residual error network as a basic network and adding a Dense thought and an improved attention mechanism;
and S03, sending the lung parenchymal image which is subjected to normalization processing and is scaled to 512x512 size into a trained interpretable tuberculosis classification network, and obtaining and outputting a classification result of the tuberculosis of the person to be detected and a corresponding class activation thermodynamic diagram in an upper computer.
In practical application, a detection person (for example, a medical staff such as a doctor) can generate a diagnosis result report for assistance according to the classification result of the tuberculosis of the detection person and the corresponding class activation thermodynamic diagram.
As an improvement of the interpretable tuberculosis classification network identification method based on the CT image, the method comprises the following steps:
the interpretable tuberculosis classification network comprises a first layer which is a convolution layer, the size of the convolution kernel is 7x7, and the step length is 2; the second layer is a maximum pooling layer, the size of a convolution kernel is 3x3, and the step size is 1; the total of 14 block-1 modules from the third layer to the fifth layer, from the seventh layer to the ninth layer, from the eleventh layer to the fifteenth layer and from the seventeenth layer to the nineteenth layer, each block-1 module is equally divided into 3 convolutional layers, the convolutional kernel size of the first layer is 1x1, the convolutional kernel size of the second layer is 3x3, the step size is 1, and the convolutional kernel size of the third layer is 1x 1; the sixth layer, the tenth layer and the sixteenth layer are all the blocl-2 downsampling modules, each blocl-2 downsampling module is equally divided into three convolution layers, the size of a convolution kernel of the first layer is 1x1, the size of a convolution kernel of the second layer is 3x3, the step size is 2, and the size of a convolution kernel of the third layer is 1x 1; the twentieth layer is a global average pooling layer, and the twenty-first layer is a full connection layer; the third layer and the sixth layer, the sixth layer and the tenth layer, the tenth layer and the sixteenth layer, and the sixteenth layer and the twentieth layer are subjected to short-circuit operation, and the same residual blocks with the same size account for 4 residual blocks;
the Dense idea is to perform short-circuit operation on the third layer to the fifth layer as Dense blocks, perform short-circuit operation on the sixth layer to the ninth layer as Dense blocks, perform short-circuit operation on the tenth layer to the fifteenth layer as Dense blocks, and perform short-circuit operation on the sixteenth layer to the nineteenth layer as Dense blocks;
adding the modified attentiveness mechanism between the fifth layer and the sixth layer, between the ninth layer and the tenth layer, between the fifteen layer and the sixteen layer, and between the eighteenth layer and the nineteenth layer, respectively;
a classification function softmax is arranged behind the twenty-first full connection layer, and a classification result of the pulmonary tuberculosis of the person to be detected is finally output;
and processing the feature map output after the twentieth layer of global average pooling by adopting a gradient-weighted class activation mapping Grad-CAM method, carrying out backward by the Grad-CAM according to the output vector to obtain the gradient of the feature map so as to obtain a gradient map corresponding to the feature map, then averaging each gradient map to obtain the weight of each feature map, then carrying out weighted summation on the weight and the feature map, and finally outputting a significance region map after passing through an activation function relu.
As a further improvement of the CT image-based interpretable tuberculosis classification network identification method of the invention:
the improved attention mechanism comprises a method for splicing two pooling results of global maximum pooling and global average pooling in a channel attention branch based on a traditional attention mechanism; on a spatial attention branch, feature extraction is carried out on an input feature matrix by adopting a parallel structure of convolution kernels with the sizes of Conv1 × 1 and Conv3 × 3, the convolution kernels with the sizes of Conv3 × 3 are decomposed into convolution kernels with the sizes of Conv1 × 3 and Conv3 × 1, and two paths of features are fused by utilizing a method of multiplying corresponding elements in matrix operation.
As a further improvement of the interpretable tuberculosis classification network identification method based on the CT image, the method comprises the following steps:
the image preprocessing in step S01 includes the following processes:
binarization is carried out on an original CT image: setting a global initial threshold T as an average value of a maximum gray value and a minimum gray value in an original CT image, dividing all pixel values of the original CT image into two groups by taking the threshold T as a segmentation point, and respectively calculating the average values of the pixels of the two groups to obtain the T f And T b Then at T f And T b Is used as a new threshold value and the threshold value T is updated, and iteration is continuously carried out until the difference between the obtained threshold value T and the previous threshold value T is less than the set parameter T 0 Thereby obtaining a binarized image, T 0 Is close to 0 preset value; filling the chest cavity of the binarized image by adopting a flooding algorithm, removing holes and bed plates in the lung parenchyma, and obtaining an image with the filled holes; negating the image filled in the hole, and removing the trachea to obtain a lung mask image; and multiplying the obtained lung mask image and the original CT image to obtain a lung parenchyma image.
As a further improvement of the CT image-based interpretable tuberculosis classification network identification method of the invention:
the training interpretable tuberculosis classification network in step S02 includes:
1) building a data set
Firstly, collecting lung CT images of a large number of persons to be detected to obtain lung parenchymal images according to the image preprocessing, then labeling the lung parenchymal images by detection persons (such as doctors), and taking the lung parenchymal images and corresponding labels as a training set, a verification set and a test set according to the proportion of 20%, 20% and 60%; processing the lung parenchymal images of the training set and the verification set by adopting three data enhancement methods of turning, rotating and cutting, carrying out normalization processing on all the data-enhanced lung parenchymal images and scaling the data-enhanced lung parenchymal images to 512x512, wherein corresponding labels are used as training inputs of an interpretable tuberculosis classification network; directly carrying out normalization processing on the images of all the test sets and scaling the images to 512x512, and taking the corresponding labels as test input of an interpretable tuberculosis classification network;
2) inputting 512x512 lung parenchymal images after training set and verification set normalization by adopting a cross entropy formula as a loss function, obtaining a classification result of the pulmonary tuberculosis of a person to be tested through network propagation of an interpretable pulmonary tuberculosis classification network, comparing the classification result with a corresponding label to calculate a loss function, and then carrying out gradient descent by using the result of the loss function to update the parameters of the network to obtain the interpretable pulmonary tuberculosis classification network after training;
3) and inputting the lung parenchymal images which are normalized and scaled to 512x512 in the test set into a trained interpretable pulmonary tuberculosis classification network to obtain a classification result of the pulmonary tuberculosis of the person to be tested, and comparing the classification result with a corresponding label, wherein the statistical accuracy reaches 94.47%.
When the method is practically applied, a tester is assisted to generate a corresponding report for assisting diagnosis according to the classification result of the pulmonary tuberculosis of the person to be tested and the output of a corresponding class activation thermodynamic diagram of the CT image of the person to be tested.
The invention has the following beneficial effects:
the invention combines a deep learning method, adds an improved attention mechanism on the basis of fully utilizing the transfer of characteristics, autonomously creatively builds and adjusts the structure of a neural network aiming at actual problems so as to be suitable for realizing an interpretable tuberculosis classification network identification method based on a CT image, simultaneously generates a significance region to explain a tuberculosis classification model, avoids the condition that the prior classification convolution neural network can not explain, and can quickly and effectively provide diagnosis opinions for doctors to refer to.
Drawings
The following describes embodiments of the present invention in further detail with reference to the accompanying drawings.
FIG. 1 is a schematic diagram of the training and testing process of the interpretable tuberculosis classification network based on CT images of the present invention;
FIG. 2 is a schematic diagram of the image preprocessing process of FIG. 1;
FIG. 3 is a diagram illustrating the result of the data enhancement processing of the lung parenchymal image in FIG. 1;
FIG. 4 is a schematic diagram of a tuberculosis classification network based on a residual error network;
FIG. 5 is a schematic diagram of a tuberculosis classification network based on the Dense idea;
FIG. 6 is a schematic diagram of an improved attention mechanism;
FIG. 7 is a schematic diagram of the structure of an interpretable tuberculosis classification network based on CT images;
FIG. 8 is a schematic diagram of the output of an interpretable tuberculosis classification network based on CT images.
Detailed Description
The invention will be further described with reference to specific examples, but the scope of protection of the invention is not limited thereto:
Acquiring a Computed Tomography (Computed Tomography) image of the lung of a person (such as a patient) to be detected, taking the image as an original CT image, and inputting the original CT image into an upper computer for processing;
in order to reduce the influence of irrelevant factors in the pulmonary tuberculosis detection, reduce the calculated amount of a feature extraction network in the pulmonary tuberculosis detection and reduce the training time, firstly, image preprocessing is carried out on an original CT image, and the extraction of lung parenchyma is carried out, for example, as shown in FIG. 2, a method of an iterative threshold method is adopted;
step 1.1, image preprocessing
1) Carrying out binarization on the original CT image, namely setting a global initial threshold value T which can be determined according to the maximum gray value T in the original CT image max And minimum gray value T min Determining:
dividing all pixels into two groups by taking T as a division point, D 1 Representing the set of all pixel values less than T, D 2 Representing the set with all pixel values larger than T, and respectively calculating the average value of all pixels in the two sets to obtain T f And T b
Then continuously updating the threshold value, and continuously iterating until the difference between the obtained T value and the previous T value is less than the set parameter T 0 Thereby obtaining a binarized image, T 0 Is a preset value, generally close to 0;
2) filling the chest cavity of the binarized image by adopting a flooding algorithm, removing holes and bed plates in the lung parenchyma, and obtaining an image with the filled holes;
3) taking the image after the hole filling, and removing the trachea to obtain a lung mask image;
4) multiplying the obtained lung mask image with the original CT image to extract a lung region, namely a lung parenchyma image;
and step 1.2, carrying out normalization processing on the lung parenchymal image and scaling the lung parenchymal image to 512x512 size to be used as input of an interpretable tuberculosis classification network.
The interpretable pulmonary tuberculosis classification network takes a residual error network as a basic network, a Dense thought and an improved attention mechanism are added to obtain a more accurate classification result of the pulmonary tuberculosis, meanwhile, a class activation mapping Grad-CAM method is added to obtain a visual significance region map, and the basis of judging the type of the pulmonary tuberculosis of the CT image by the model can be directly observed.
Step 2.1, establishing a classification network
Step 2.1.1, establishing a tuberculosis classification network based on a residual error network
In the network model, lightweight design modules block-1 and block2 are used, high representation capacity is kept, parameters are reduced, and high calculation efficiency is achieved, as shown in fig. 4, the first layer of the tuberculosis classification network based on the residual error network is a convolution layer, the size of the convolution layer is 7x7, and the step is 2; the second layer is a maximum pooling layer, the convolution kernel size is 3x3, and the step size is 1; the total of 14 block-1 modules from the third layer to the fifth layer, from the seventh layer to the ninth layer, from the eleventh layer to the fifteenth layer and from the seventeenth layer to the nineteenth layer, each block-1 module is equally divided into 3 convolutional layers, the convolutional kernel size of the first layer is 1x1, the convolutional kernel size of the second layer is 3x3, the step size is 1, and the convolutional kernel size of the third layer is 1x 1; the sixth layer, the tenth layer and the sixteenth layer are all the blocl-2 downsampling modules, each blocl-2 downsampling module is divided into three convolution layers, the size of a convolution kernel of the first layer is 1x1, the size of a convolution kernel of the second layer is 3x3, the step size is 2, and the size of a convolution kernel of the third layer is 1x 1; the twentieth layer is global average pooling, the twenty-first layer is a connecting layer, the full connecting layer is mainly used for flexibly adjusting output, and finally, classification results of the pulmonary tuberculosis of the person to be detected (namely probability values of four classifications of primary pulmonary tuberculosis, hematogenous disseminated pulmonary tuberculosis, secondary pulmonary tuberculosis and normal lung) are output through a softmax classification function; based on the idea of depth residual errors, performing short-circuit operation on modules with different output sizes, namely performing short-circuit operation on a third layer and a sixth layer, a sixth layer and a tenth layer, a tenth layer and a sixteenth layer, and a sixteenth layer and a twentieth layer, wherein the same residual blocks with the same size account for 4 residual blocks, so that a tuberculosis classification network based on a residual network is established, and the specific parameters of the network are shown in the following table 1:
TABLE 1 tuberculosis Classification network principal parameters based on residual error network
Step 2.1.2, establishing a tuberculosis classification network based on the Dense idea
In order to further enhance the transfer of features and improve the accuracy of classification, by using the thought of a Dense network, performing short-circuit operation on the third layer to the fifth layer of the tuberculosis classification network based on the residual error network established in the step 2.1.1 as a Dense block, performing short-circuit operation on the sixth layer to the ninth layer as a Dense block, performing short-circuit operation on the tenth layer to the fifteenth layer as a Dense block, performing short-circuit operation on the sixteenth layer to the nineteenth layer as a Dense block, wherein the same Dense block with the same size is the same Dense block, and the total number of 4 Dense blocks is shown in fig. 5, so that the tuberculosis classification network based on the Dense thought is established;
step 2.1.3, establishing an interpretable tuberculosis classification network based on the Dense thought and the improved attention mechanism
The traditional attention mechanism is improved, and in the channel attention branch, in order to fully retain background and texture information, a method of splicing two pooling results of global maximum pooling and global average pooling is adopted; on a spatial attention branch, considering that high and low levels respectively have rich semantic features and texture information, the feature extraction is carried out on an input feature matrix by adopting the structure of parallel branches of convolution kernels with the sizes of Conv1 multiplied by 1 and Conv3 multiplied by 3 so as to obtain diversified feature information; the Conv1 × 1 branch sequentially includes a Conv1 × 1, a batch normalization BN, an activation function Relu, and a Conv1 × 1, and meanwhile, a Conv3 × 3 convolution kernel on the Conv3 × 3 branch is decomposed into Conv1 × 3 and Conv3 × 1 convolution kernels, so that the amount of computation can be effectively reduced, the computation cost can be saved, and then the batch normalization BN, the batch normalization Relu, and the Conv1 × 1 are sequentially included; by utilizing a method of multiplying corresponding elements in matrix operation, two paths of characteristics are fused, and richer space characteristic information can be obtained; padding is set to 0 when using Conv1 × 1, and (0, 1) and (1, 0) when using Conv1 × 3 and Conv3 × 1, respectively, as shown in fig. 6;
in order to enhance the fine granularity of the classification network, an improved attention mechanism is added into the tuberculosis classification network established in the step 2.1.2 based on the Dense idea, and an improved attention mechanism is added between the fifth layer and the sixth layer for carrying out feature recalibration; adding an improved attention mechanism between the ninth layer and the tenth layer for characteristic recalibration; an improved attention mechanism is added between the fifteen layers and the sixteen layers for characteristic recalibration; an improved attention mechanism is added between the eighteenth layer and the nineteenth layer for characteristic recalibration, so that an interpretable tuberculosis classification network (abbreviated as an interpretable tuberculosis classification network) based on the Dense thought and the improved attention mechanism is established, as shown in the attached figure 7;
step 2.1.4 output of interpretable tuberculosis Classification network
In order to improve the interpretability of the convolutional neural network model, a gradient-weighted class activation mapping Grad-CAM method is adopted to visualize important areas leading to deep learning model decision, as shown in FIG. 8, a lung parenchymal image which is subjected to normalization processing in step 1 and is scaled to 512x512 is input into an interpretable tuberculosis classification network, and the classification result of the tuberculosis of a person to be detected is output through the interpretable tuberculosis classification network (W) 1 ,W 2 ,W 3 …W n ) Meanwhile, for the final feature map obtained after the 20 th layer global average pooling, the Grad-CAM performs backward according to the output vector to obtain the gradient of the feature map, so as to obtain the gradient corresponding to each pixel point on each feature map, namely the gradient map corresponding to the feature map, and then performs backward clustering on each feature mapThe gradient map is averaged, the average value corresponds to the weight of each feature map, then the weight and the feature maps are subjected to weighted summation, and after a relu activation function is passed, a saliency region map, namely a class activation thermodynamic map of the tuberculosis classification network is finally output, different pixel colors in the class activation thermodynamic map represent classification bases of different tuberculosis, and the darker color represents that the pixel occupies larger weight when the map is classified into a certain tuberculosis;
step 2.2, training interpretable tuberculosis classification network
Training and testing the interpretable tuberculosis classification network established in the step 2.1 so as to obtain the trained interpretable tuberculosis classification network with clinical application level, as shown in fig. 1, the process is as follows:
step 2.2.1, data set establishment
Firstly, CT images of the lungs of a large number of persons to be detected are collected, image preprocessing is carried out according to the step 1.1, then the lung parenchyma images are obtained, detection persons (such as doctors) carry out relatively authoritative annotation on the preprocessed lung parenchyma images, the CT image label of the person to be detected with primary tuberculosis is set as 1, the CT image label of hematogenous disseminated tuberculosis is set as 2, the CT image label of the person to be detected with secondary tuberculosis is set as 3, and the CT image label of the normal lungs is set as 0; taking the lung parenchymal image and the corresponding label as a training set, a verification set and a test set according to the proportion of 20%, 20% and 60%;
for the lung parenchymal images of the training set and the verification set, the partial type pulmonary tuberculosis CT images have the condition of insufficient samples, so the data enhancement needs to be firstly carried out on the lung parenchymal images of the training set and the verification set so as to enhance the training effect of deep learning, and the operation is carried out on the lung parenchymal images of the training set and the verification set by adopting three methods of turning, rotating and cutting:
randomly rotating the lung parenchymal image of each training set and each verification set for three times to obtain three other images;
performing vertical and horizontal cutting on the lung parenchymal image of each training set and each verification set to obtain another four CT images;
horizontally turning and deforming and scaling the lung parenchyma images of each training set and each verification set to obtain four other CT images;
finally, obtaining 11 data-enhanced lung parenchymal images of each training set and verification set;
normalizing all the data-enhanced lung parenchymal images according to the step 1.2, then scaling the data-enhanced lung parenchymal images to 512x512, and taking the normalized lung parenchymal images and corresponding labels which are scaled to 512x512 as training input of an interpretable tuberculosis classification network;
the images of all the test sets are directly normalized and then scaled to 512x512 without data enhancement, and the lung parenchymal images normalized and scaled to 512x512 and the corresponding labels are used as test input of an interpretable tuberculosis classification network;
step 2.2.2 training an interpretable tuberculosis Classification network
During training, the parameters of the interpretable tuberculosis classification network are updated by adopting a cross entropy formula as a loss function. Inputting the normalized 512x512 lung parenchymal image, obtaining a classification result of the pulmonary tuberculosis of the person to be tested through network propagation of the interpretable pulmonary tuberculosis classification network, comparing the classification result with an actual classification label to calculate a loss function, and then performing gradient descent by using the result of the loss function to update the parameters of the network to obtain the interpretable pulmonary tuberculosis classification network after training.
Batch normalization BN layers are added behind each convolution layer, and then nonlinear activation of activation functions is carried out, so that the advantage that the distribution of each layer is pulled to a normal distribution can accelerate the training of the network and improve the problems of gradient explosion, gradient dispersion and the like; the activation function is commonly used Relu, and the formula is as follows, so that the network can be converged quickly, and the problem that the gradient of the neural network disappears is improved.
Training was performed on the data set by random gradient descent with momentum, and the hyper-parameters used for training were as follows: learning rate is 5e-3, momentum is 0.9, batch size is 8 at random, namely after 8 times of forward transmission, a loss function is calculated for the 8 times of results, a CNN network parameter is reversely adjusted once, and when the iteration number is 18, training is terminated to obtain a trained interpretable tuberculosis classification network;
2.2.3 testing of interpretable tuberculosis Classification network
In the classification and detection model of tuberculosis, samples can be classified into True Positive (TP), False Positive (FP), True Negative (TN) and False Negative (FN) according to different combinations of true types and model predicted types, and a confusion matrix of specific classification results is shown in table 2 below.
TABLE 2 pulmonary tuberculosis Classification results confusion matrix
In the classification of pulmonary infectious diseases and detection model problems, the main evaluation indexes used are Accuracy (Accuracy), Sensitivity (Sensitivity) and Specificity (Specificity), missed diagnosis rate and misdiagnosis rate; the accuracy rate is the proportion of the correct sample number to the total sample number predicted by the learner in the classification result, and is the most basic index for judging the quality of a classification algorithm, and the formula is defined as follows:
sensitivity (Sensitivity), also known as recall, is the probability that an actually positive sample is correctly predicted to be positive, reflecting the classifier's ability to identify tuberculosis, and is defined as follows:
the missed diagnosis rate is an index corresponding to the sensitivity, and reflects the probability that the positive tuberculosis is not identified, the higher the sensitivity is, the lower the missed diagnosis rate is, and the formula is defined as follows:
specificity (Specificity) is the probability that an actually negative sample is correctly predicted to be negative, and reflects the capability of a model to correctly identify the negative sample, and a formula is defined as follows:
the misdiagnosis rate is an index corresponding to the specificity, and reflects the probability that negative tuberculosis is misdiagnosed as positive, the higher the specificity is, the lower the misdiagnosis rate is, and the formula is defined as follows:
in order to better evaluate the quality of a tuberculosis classification model, a ROC curve is introduced, according to the definition of a confusion matrix of the tuberculosis classification model, true positive rate TPR (thermal Sensitivity) index and false positive rate FPR (misdiagnosis rate) index are firstly sorted according to the model prediction probability, then the threshold value of a classifier is set to be increased slowly from 0 to 1, when the threshold value is 0, all samples are predicted to be positive, therefore, the pair of the TPR and the FPR point is (1, 1), when the threshold value is 1, all samples are predicted to be negative, the pair of the TPR and the FPR point is (0, 0), a group of point pairs of the TPR and the FPR can be obtained by the corresponding threshold value, finally, the connected curve is the ROC curve of the model, and the closer to the upper left corner, the model performance is represented;
to describe the ROC curve more numerically, we use the AUC (area Under rock) value to represent, i.e., the area covered Under the ROC curve. The AUC values ranged from 0.5 to 1.0, with greater AUC representing better performance of the model;
after normalization, the lung parenchymal images in the test set are scaled to 512x512 and input to a trained interpretable pulmonary tuberculosis classification network to obtain a classification result of the pulmonary tuberculosis of the person to be tested, the classification result is compared with a label corresponding to the lung parenchymal images, the statistical accuracy is calculated, the final overall accuracy reaches 94.47%, and the test result is shown in the following table 3; f1 values were calculated with accuracy and recall and the results are shown in table 3.7.
TABLE 3.7 test result indices
| Rate of accuracy | Recall rate | FPR | F1 value |
| 94.47％ | 92.49％ | 7.79％ | 93.48％ |
The recall rate and the FPR respectively reflect the identification capability of positive and negative samples, and the experimental results show that the accuracy rate, the recall rate and the F1 value are all more than 90%, and the FPR value is low, which indicates that the model extracts effective characteristics from the image, and has good identification effect on the positive and negative samples, and the stability of the model is good and accords with expectations.
3. Auxiliary diagnosis
And (2) sending the lung parenchymal image obtained in the step (1) and subjected to normalization processing and scaled to 512x512 size into the trained interpretable pulmonary tuberculosis classification network obtained in the step (2), obtaining and outputting a classification result (namely probability values of four classifications of primary pulmonary tuberculosis, hematogenous disseminated pulmonary tuberculosis, secondary pulmonary tuberculosis and normal lung) of the corresponding pulmonary tuberculosis of the person to be detected and a corresponding class activation thermodynamic diagram (a significant characteristic region diagram) in an upper computer, and generating a result report for auxiliary diagnosis by the medical staff according to the classification result of the pulmonary tuberculosis of the person to be detected and the corresponding class activation thermodynamic diagram.
Experiment 1:
and testing three models on the same test set, wherein the model 1 is a tuberculosis classification network based on a residual error network, the model 2 is a tuberculosis classification network based on a Dense thought, and the model 3 is an interpretable tuberculosis classification network based on the Dense thought and an improved attention mechanism. And taking accuracy, sensitivity, specificity and AUC as indexes, and taking the table 3 as each performance index of the model 1, the model 2 and the model 3, wherein the sensitivity, the specificity and the AUC are respectively the average of the sensitivity, the specificity and the AUC of the model to the recognition results of the three types of tuberculosis.
TABLE 4 comparison of recognition indexes for different classification models
As can be seen from the table above, the model 3 is improved to a certain extent in three indexes of accuracy, sensitivity and specificity, which shows that the interpretable tuberculosis classification network based on the Dense thought and the improved attention mechanism of the invention solves the degradation problem of the deep network to a certain extent, so that the deep network can extract more characteristic information, thereby improving the performance of the model, and the generated class activation thermodynamic diagram can also be used as the diagnosis reference of a doctor.
Finally, it is also noted that the above-mentioned list is only a few specific embodiments of the present invention. It is obvious that the invention is not limited to the above embodiments, but that many variations are possible. All modifications which can be derived or suggested by a person skilled in the art from the disclosure of the present invention are to be considered within the scope of the invention.
Claims (3)
1. The interpretable tuberculosis classification network identification method based on the CT image is characterized by comprising the following steps of:
s01, acquiring a Computed Tomography (CT) image of the lung of the person to be tested, taking the CT image as an original CT image, inputting the original CT image into an upper computer for image preprocessing, extracting the lung parenchyma to obtain a lung parenchyma image, and then normalizing the lung parenchyma image and zooming the lung parenchyma image to 512x512 size;
s02, establishing and training an interpretable tuberculosis classification network, wherein the interpretable tuberculosis classification network is established by taking a residual error network as a basic network and adding a Dense thought and an improved attention mechanism;
s03, sending the lung parenchymal images which are subjected to normalization processing and scaled to 512x512 size into a trained interpretable pulmonary tuberculosis classification network, and obtaining and outputting a classification result of pulmonary tuberculosis of a person to be detected and a corresponding class activation thermodynamic diagram in an upper computer;
the interpretable tuberculosis classification network comprises a first layer which is a convolution layer, the size of the convolution kernel is 7x7, and the step length is 2; the second layer is a maximum pooling layer, the convolution kernel size is 3x3, and the step size is 1; the number of the block-1 modules from the third layer to the fifth layer, the number of the seventh layer to the ninth layer, the number of the eleventh layer to the fifteenth layer and the number of the seventeenth layer to the nineteenth layer are 14, each block-1 module is equally divided into 3 convolution layers, the convolution kernel size of the first layer is 1x1, the convolution kernel size of the second layer is 3x3, the step size is 1, and the convolution kernel size of the third layer is 1x 1; the sixth layer, the tenth layer and the sixteenth layer are all the blocl-2 downsampling modules, each blocl-2 downsampling module is equally divided into three convolution layers, the size of a convolution kernel of the first layer is 1x1, the size of a convolution kernel of the second layer is 3x3, the step size is 2, and the size of a convolution kernel of the third layer is 1x 1; the twentieth layer is a global average pooling layer, and the twenty-first layer is a full connection layer; the third layer and the sixth layer, the sixth layer and the tenth layer, the tenth layer and the sixteenth layer, and the sixteenth layer and the twentieth layer are subjected to short-circuit operation, and the same residual blocks with the same size account for 4 residual blocks;
the Dense idea is that the third layer to the fifth layer are used as Dense blocks to perform short-circuit operation, the sixth layer to the ninth layer are used as Dense blocks to perform short-circuit operation, the tenth layer to the fifteenth layer are used as Dense blocks to perform short-circuit operation, and the sixteenth layer to the nineteenth layer are used as Dense blocks to perform short-circuit operation;
adding the modified attentiveness mechanism between the fifth layer and the sixth layer, between the ninth layer and the tenth layer, between the fifteen layer and the sixteen layer, and between the eighteenth layer and the nineteenth layer, respectively;
a classification function softmax is arranged behind the twenty-first full connection layer, and a classification result of the pulmonary tuberculosis of the person to be detected is finally output;
processing the feature map output after the twentieth layer of global average pooling by adopting a gradient-weighted class activation mapping Grad-CAM method, carrying out backward by the Grad-CAM according to an output vector to obtain the gradient of the feature map so as to obtain a gradient map corresponding to the feature map, then averaging each gradient map to obtain the weight of each feature map, then carrying out weighted summation on the weight and the feature map, and finally outputting a significance region map after passing through an activation function relu;
the improved attention mechanism comprises a method for splicing two pooling results of global maximum pooling and global average pooling in a channel attention branch based on a traditional attention mechanism; on a spatial attention branch, feature extraction is carried out on an input feature matrix by adopting a parallel structure of convolution kernels with the sizes of Conv1 multiplied by 1 and Conv3 multiplied by 3, the convolution kernels with the sizes of Conv3 multiplied by 3 are decomposed into convolution kernels with the sizes of Conv1 multiplied by 3 and Conv3 multiplied by 1, and two paths of features are fused by utilizing a method of multiplying corresponding elements in matrix operation.
2. The method of claim 1, wherein the method comprises:
the image preprocessing in step S01 includes the following processes:
carrying out binarization on an original CT image: setting a global initial threshold T as an average value of the maximum gray value and the minimum gray value in the original CT image, dividing all pixel values of the original CT image into two groups by taking the threshold T as a segmentation point, and dividing the two groupsThe pixels of the two groups respectively calculate the average value to obtain T f And T b Then by T f And T b The average value of the threshold T is used as a new threshold value, the threshold value T is updated, and iteration is continuously carried out until the difference between the obtained threshold value T and the previous threshold value T is less than the set parameter T 0 Thereby obtaining a binarized image, T 0 Is close to 0 preset value; filling the chest cavity of the binarized image by adopting a flooding algorithm, removing holes and bed plates in the lung parenchyma, and obtaining an image with the filled holes; negating the image filled in the hole, and removing the trachea to obtain a lung mask image; and multiplying the obtained lung mask image and the original CT image to obtain a lung parenchyma image.
3. The method of claim 2, wherein the method comprises the steps of:
the training interpretable tuberculosis classification network in step S02 includes:
1) building a data set
Firstly, collecting lung CT images of a large number of persons to be detected to obtain lung parenchyma images according to the image preprocessing, then labeling the lung parenchyma images by the detecting persons, and taking the lung parenchyma images and corresponding labels as a training set, a verification set and a test set according to the proportion of 20%, 20% and 60%; processing the lung parenchymal images of the training set and the verification set by three data enhancement methods of turning, rotating and cutting, carrying out normalization processing on all the data-enhanced lung parenchymal images and scaling to 512x512, wherein the corresponding labels are used as training input of an interpretable tuberculosis classification network; directly carrying out normalization processing on images of all test sets and scaling the images to 512x512, and taking corresponding labels as test input of an interpretable tuberculosis classification network;
2) inputting 512x512 lung parenchymal images after the normalization in a training set and a verification set by adopting a cross entropy formula as a loss function, obtaining a classification result of the pulmonary tuberculosis of a person to be tested through network propagation of an interpretable pulmonary tuberculosis classification network, comparing the classification result with a corresponding label to calculate the loss function, and then updating the parameters of the network by gradient descent by using the result of the loss function to obtain the interpretable pulmonary tuberculosis classification network after training;
3) and inputting the lung parenchymal images which are normalized and scaled to 512x512 in the test set into a trained interpretable pulmonary tuberculosis classification network to obtain a classification result of the pulmonary tuberculosis of the person to be tested, comparing the classification result with a corresponding label, and counting the accuracy.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110337007.2A CN113095382B (en) | 2021-03-30 | 2021-03-30 | Interpretable tuberculosis classification network identification method based on CT image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110337007.2A CN113095382B (en) | 2021-03-30 | 2021-03-30 | Interpretable tuberculosis classification network identification method based on CT image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113095382A CN113095382A (en) | 2021-07-09 |
| CN113095382B true CN113095382B (en) | 2022-07-26 |
Family
ID=76670897
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110337007.2A Active CN113095382B (en) | 2021-03-30 | 2021-03-30 | Interpretable tuberculosis classification network identification method based on CT image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113095382B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114359194B (en) * | 2021-12-27 | 2024-07-12 | 浙江大学 | Multimode cerebral apoplexy infarct region image processing method based on improved U-Net network |
| CN114757942A (en) * | 2022-05-27 | 2022-07-15 | 南通大学 | Method for detecting active tuberculosis by multilayer spiral CT (computed tomography) based on deep learning |
| CN115393321A (en) * | 2022-08-26 | 2022-11-25 | 南通大学 | Multi-classification pulmonary tuberculosis detection method based on deep learning multi-layer spiral CT |
| CN116912258B (en) * | 2023-09-14 | 2023-12-08 | 天津市胸科医院 | Self-efficient estimation method for focus parameters of lung CT image |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109003274A (en) * | 2018-07-27 | 2018-12-14 | 广州大学 | A kind of diagnostic method, device and readable storage medium storing program for executing for distinguishing pulmonary tuberculosis and tumour |
| CN111046939A (en) * | 2019-12-06 | 2020-04-21 | 中国人民解放军战略支援部队信息工程大学 | CNN (CNN) class activation graph generation method based on attention |
| CN111242930A (en) * | 2020-01-15 | 2020-06-05 | 河北省胸科医院 | Method and device for acquiring pulmonary tuberculosis recognition model, storage medium and processor |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9984283B2 (en) * | 2015-02-14 | 2018-05-29 | The Trustees Of The University Of Pennsylvania | Methods, systems, and computer readable media for automated detection of abnormalities in medical images |
-
2021
- 2021-03-30 CN CN202110337007.2A patent/CN113095382B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109003274A (en) * | 2018-07-27 | 2018-12-14 | 广州大学 | A kind of diagnostic method, device and readable storage medium storing program for executing for distinguishing pulmonary tuberculosis and tumour |
| CN111046939A (en) * | 2019-12-06 | 2020-04-21 | 中国人民解放军战略支援部队信息工程大学 | CNN (CNN) class activation graph generation method based on attention |
| CN111242930A (en) * | 2020-01-15 | 2020-06-05 | 河北省胸科医院 | Method and device for acquiring pulmonary tuberculosis recognition model, storage medium and processor |
Non-Patent Citations (2)
| Title |
|---|
| Multi-Scale Aerial Target Detection Based on Densely Connected Inception ResNet;MIAOHUI ZHANG等;《IEEE Access》;20200606;第84867-84878页 * |
| 卷积神经网络在生物医学图像上的应用进展;杨培伟 等;《计算机工程与应用》;20210109;第44-58页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113095382A (en) | 2021-07-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113095382B (en) | Interpretable tuberculosis classification network identification method based on CT image | |
| Alqudah et al. | COVID-19 detection from x-ray images using different artificial intelligence hybrid models | |
| CN109543526B (en) | True and false facial paralysis recognition system based on depth difference characteristics | |
| KR20190105180A (en) | Apparatus for Lesion Diagnosis Based on Convolutional Neural Network and Method thereof | |
| CN111767952B (en) | Interpretable lung nodule benign and malignant classification method | |
| CN112784856A (en) | Channel attention feature extraction method and identification method of chest X-ray image | |
| CN111754453A (en) | Pulmonary tuberculosis detection method and system based on chest radiography image and storage medium | |
| Garg et al. | Multi-scale residual network for covid-19 diagnosis using ct-scans | |
| CN108171714A (en) | It is a kind of based on the fracture area recognition methods compared in pairs and system | |
| CN113782184A (en) | Cerebral apoplexy auxiliary evaluation system based on facial key point and feature pre-learning | |
| CN113610118A (en) | Fundus image classification method, device, equipment and medium based on multitask course learning | |
| Baghel et al. | Early Diagnosis of Emphysema using Convolutional Neural Networks | |
| Naskinova | Transfer learning with NASNet-Mobile for Pneumonia X-ray classification | |
| CN112784924B (en) | Rib fracture CT image classification method based on grouping aggregation deep learning model | |
| Sridhar et al. | A study on Co-occurrence of various lung diseases and COVID-19 by observing chest X-ray similarity using deep convolutional neural networks | |
| CN117333667A (en) | Automatic brain CT image bleeding area segmentation method and system | |
| CN116797817A (en) | Autism disease prediction technology based on self-supervision graph convolution model | |
| Tobias et al. | Android Application for Chest X-ray Health Classification From a CNN Deep Learning TensorFlow Model | |
| CN113763332B (en) | Pulmonary nodule analysis method and device based on ternary capsule network algorithm and storage medium | |
| Darapaneni et al. | Explainable diagnosis, lesion segmentation and quantification of COVID-19 infection from CT images using convolutional neural networks | |
| CN112562851B (en) | Construction method and system of oral cancer cervical lymph metastasis diagnosis algorithm | |
| CN110176298B (en) | Kernel principal component spectrum Hash method for diabetic fundus image classification | |
| CN114330484A (en) | Method and system for classification and focus identification of diabetic retinopathy through weak supervision learning | |
| Wijerathna et al. | Brain Tumor Detection Using Image Processing | |
| Dhanapala et al. | Transfer learning techniques with SVM for covid-19 disease prediction based on chest X-ray images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |