CN112819747A - Method for automatically diagnosing benign and malignant nodules based on lung tomography image - Google Patents
Method for automatically diagnosing benign and malignant nodules based on lung tomography image Download PDFInfo
- Publication number
- CN112819747A CN112819747A CN201911055430.2A CN201911055430A CN112819747A CN 112819747 A CN112819747 A CN 112819747A CN 201911055430 A CN201911055430 A CN 201911055430A CN 112819747 A CN112819747 A CN 112819747A
- Authority
- CN
- China
- Prior art keywords
- image
- nodule
- lung
- dimensional
- nodules
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 230000003211 malignant effect Effects 0.000 title claims abstract description 25
- 206010054107 Nodule Diseases 0.000 title abstract description 47
- 210000004072 lung Anatomy 0.000 title abstract description 17
- 238000003325 tomography Methods 0.000 title abstract description 4
- 206010056342 Pulmonary mass Diseases 0.000 claims abstract description 120
- 238000003745 diagnosis Methods 0.000 claims abstract description 26
- 238000012549 training Methods 0.000 claims abstract description 20
- 239000011159 matrix material Substances 0.000 claims description 32
- 238000002372 labelling Methods 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 11
- 230000002685 pulmonary effect Effects 0.000 claims description 10
- 238000001914 filtration Methods 0.000 claims description 8
- 238000007477 logistic regression Methods 0.000 claims description 8
- 238000012952 Resampling Methods 0.000 claims description 7
- 238000010606 normalization Methods 0.000 claims description 4
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000003062 neural network model Methods 0.000 claims description 2
- 238000002405 diagnostic procedure Methods 0.000 claims 1
- 238000012216 screening Methods 0.000 abstract description 15
- 238000013527 convolutional neural network Methods 0.000 abstract description 9
- 230000008569 process Effects 0.000 abstract description 6
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 201000010099 disease Diseases 0.000 abstract 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 abstract 1
- 238000012360 testing method Methods 0.000 description 17
- 206010058467 Lung neoplasm malignant Diseases 0.000 description 13
- 201000005202 lung cancer Diseases 0.000 description 13
- 208000020816 lung neoplasm Diseases 0.000 description 13
- 230000006870 function Effects 0.000 description 9
- 230000000694 effects Effects 0.000 description 7
- 230000008859 change Effects 0.000 description 6
- 238000000605 extraction Methods 0.000 description 5
- 206010028980 Neoplasm Diseases 0.000 description 4
- 201000011510 cancer Diseases 0.000 description 4
- 238000010276 construction Methods 0.000 description 4
- 238000010801 machine learning Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000003709 image segmentation Methods 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000004195 computer-aided diagnosis Methods 0.000 description 2
- 238000002790 cross-validation Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 238000013399 early diagnosis Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 238000007476 Maximum Likelihood Methods 0.000 description 1
- 230000032683 aging Effects 0.000 description 1
- 238000003915 air pollution Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000002591 computed tomography Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013075 data extraction Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 230000010339 dilation Effects 0.000 description 1
- 229940079593 drug Drugs 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 230000036210 malignancy Effects 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000010008 shearing Methods 0.000 description 1
- 230000000391 smoking effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30061—Lung
- G06T2207/30064—Lung nodule
Abstract
The invention discloses an artificial intelligence method for automatically diagnosing benign and malignant nodules based on a lung tomography image picture, which mainly comprises two parts of screening and diagnosing lung nodules, wherein in the screening process of the lung nodules, two-dimensional images of suspected nodules are firstly obtained and are used as training data to complete the input of a convolutional neural network, and the nodules are classified by predicting the probability of true and false positive nodules; in the process of diagnosing the lung nodule, the lung nodule obtained by screening is segmented and reconstructed into a three-dimensional lung nodule image, then the image characteristics of the lung nodule image are extracted, and then the image characteristics are input into a prediction model to obtain a prediction result and screen the most relevant image characteristics. The invention can automatically diagnose the suspicious region based on the CT sequence of a patient and the marking information of a doctor, and assists the doctor to make more objective and accurate disease diagnosis.
Description
Technical Field
The invention relates to a method for automatically judging whether nodules are good or malignant in an image by using a lung tomography image, which is a computer program product relating to the field of artificial intelligence image recognition and the field of biometry.
Background
Due to aging population, air pollution, high smoking rate and the like, the lung cancer becomes the cancer with the highest death rate in China since the beginning of the 21 st century, and the lung cancer is also the cancer with the highest death rate in the world at present. In 2015, the number of the lung cancer cases in China is 733300 and 610200, and the death rate is 83 percent, wherein one of the main reasons is that most patients are diagnosed in middle and advanced stages. Therefore, early diagnosis and treatment of lung cancer is critical to reduce lung cancer mortality.
In 2011, the result of the National Lung cancer Screening test (NLST) indicates that the Low-dose helical computed tomography (LDCT) can reduce the Lung cancer mortality by 20%, based on the research result, the international academic organization recommends to carry out Low-dose helical CT Screening in high-risk groups, and China also starts the Screening work of the high-risk groups of Lung cancer in 2009 in the project of "early diagnosis and early treatment of cancer in rural areas". However, in NLST studies, CT-based early screen diagnostic False positive rates (False-positive rates) were as high as 96.4%. Reducing lung cancer mortality requires not only popularization of early screening of lung cancer, but also improvement of diagnostic accuracy of early screening of lung cancer.
In order to improve the early screening accuracy of lung cancer and simultaneously reduce the working pressure of professional radiologists, many scholars propose an automatic diagnosis method based on lung CT images, wherein most of the automatic diagnosis methods are directly based on two-dimensional Nodule (Nodule) sectional images or are combined with part of two-dimensional Nodule image features to construct a deep learning model for prediction, and a good prediction effect is achieved. However, there are few studies in the fields of three-dimensional nodules and their image features, and the prediction effect has a large improvement space.
Disclosure of Invention
In order to improve the diagnosis accuracy of the benign and malignant pulmonary nodules, the invention provides an automatic diagnosis method of the benign and malignant pulmonary nodules, which combines the lung CT data with Deep Learning (Deep Learning) and classical Machine Learning (Machine Learning) methods to carry out artificial intelligent diagnosis of the malignant degree of the pulmonary nodules, and simultaneously provides a plurality of image characteristics most relevant to the benign and malignant pulmonary nodules for reference for doctors.
The automatic diagnosis method for benign and malignant pulmonary nodules comprises the following steps:
-acquiring a two-dimensional image of a suspected nodule from a lung CT sequence and a physician marker location;
-introducing the two-dimensional image as input into a nodule classification model, differentiating nodules from non-nodules;
-using the physician labeling coordinates of the classified nodule and the corresponding CT sequence of the nodule as input data for the benign and malignant diagnosis part of the lung nodule;
-segmenting lung nodule regions to obtain three-dimensional lung nodule images;
-extracting three-dimensional lung nodule image features;
-introducing the nodule image features as input data into a predictive model for diagnosing benign and malignant lung nodules.
The lung CT sequence is defined as the image of all 2D CT in the lung of a diagnosis object, and one diagnosis object corresponds to a specific CT serial number. The shapes and sizes of all lung CT tomograms in the same sequence are consistent, the physical distances (Pixel spacing) between adjacent Pixel points of all the CT tomograms are the same, and the distances (layer thickness) between adjacent CT tomograms are the same. The lung CT sequence image data may be acquired by a medical professional with the aid of a CT machine. The DICOM protocol (Digital Imaging and Communications in Medicine) currently in common use indicates that the content of the CT data store contains, in addition to the CT image data of the patient, the slice thickness, information on the machine that acquired the CT image, the timestamp, basic information on the patient, etc.
The doctor labeling information refers to labeling of a region of a lung nodule by a doctor, and the labeling information contains boundary coordinates of the region of the lung nodule. The boundary coordinates need to be continuous and closed, when the boundary coordinates are observed on the 2D CT sectional image, the boundary coordinates form a ring, suspicious lung nodules are arranged in the ring, and normal lung tissues are arranged outside the ring; when viewed in conjunction with a CT sequence, the boundary coordinates form a closed curved surface with the entire suspect lung nodule inside the curved surface and normal lung tissue outside the curved surface. The physician can mark the nodule region with the aid of a Computer-aided diagnosis (CAD) and save this coordinate information.
And the center position of the suspected nodule is averaged according to the nodule area boundary coordinates in the marking information of the doctor.
And segmenting a 2-dimensional image of the suspected nodule, taking the lung CT sequence and the center position of the suspected nodule as input data, searching the corresponding relation between the center position of the suspected nodule and a single slice in the CT sectional image, and segmenting the image of 25 pixels in the positive and negative directions of the X axis and the Y axis of the center position of the suspected nodule in the slice.
And (3) HU value conversion of the CT image, setting HU value pixels in segmented 2-dimensional image data to be more than 400 and 1 and setting HU value pixels to be less than-1000 and 0, zooming the intermediate range between 0 and 1, and converting the HU value pixels into a gray image.
The method for segmenting the lung nodule region takes a lung CT sequence of an object to be diagnosed and screened doctor labeling information as input parameters, searches for a corresponding relation between nodule boundary labeling information and a CT sectional image, respectively segments the inner region of a labeled boundary in each CT sectional image containing the labeling information, and all segmented regions belong to the nodule to be diagnosed of the object to be diagnosed.
The method for acquiring the three-dimensional lung nodule image is characterized in that CT tomographic image segmentation regions of nodules to be diagnosed are used as input data, the sequence of each CT tomographic image segmentation region is determined, and then the segmentation regions are sequentially arranged and reconstructed to form the three-dimensional image of the lung nodules. The ideal distance between adjacent voxels of the three-dimensional image is (Slice Thickness =1, Pixel spacing x =1, Pixel spacing y = 1), but the distance between the actual adjacent voxels is not necessarily the same, so to obtain a lung nodule of consistent size, a resampling (resampling) is further required, the resampling is performed by scaling the adjacent voxel distance according to a ratio (1/Slice Thickness, 1/Pixel spacing x, 1/Pixel spacing), that is, the actual physical distance (1 mm, 1mm, 1 mm) between the adjacent voxels after scaling, and the original three-dimensional lung nodule image can be obtained by using Python tool package pyradiomicity and scaling ratio (1/Slice Thickness, 1/Pixel spacing x, 1/Pixel spacing y).
Three-dimensional lung nodule image features are extracted by a feature extractor (based on Python toolkit Pyradiomics). The extracted features comprise different types of features such as First-order features (First order), Shape features (Shape), Texture features (Texture) and the like; the object for extracting the features may be an original three-dimensional lung nodule image, or a result obtained by processing different filters (filters) such as Laplacian filtering (Laplacian), Wavelet filtering (Wavelet), and logarithmic filtering (Logarithm). And the feature extraction takes the three-dimensional lung nodule image as input data, and extracts the three-dimensional lung nodule image features. As will be described in more detail in the embodiments below, each type of feature of a three-dimensional lung nodule image has its corresponding physical interpretation, and different filters play a corresponding role for the three-dimensional lung nodule image.
The prediction and important feature screening model based on machine learning is based on Logistic Regression (Logistic Regression, hereinafter referred to as Logistic Regression), and is normalized by using an Elastic network (Elastic Net), and the model is used for predicting the benign and malignant degree of lung nodules. The construction details of the prediction model can be seen in the following embodiments, the AUC of the prediction model on the test data (testing dataset) reaches 96.57%, the prediction accuracy reaches 92.55%, and the diagnosis of the lung nodule by using the prediction model can achieve a more accurate diagnosis result, and can also provide the most relevant image features for the doctor to refer.
The classifier model of the true and false positive nodules based on the deep convolutional neural network adopts a multilayer convolutional layer and a pooling layer to obtain the image characteristics of the positive nodules, and is regularized by using Dropout. Details of the construction of the prediction model can be found in the following embodiments, where the AUC of the model on the test data reaches 97.1% and the prediction accuracy reaches 94.45%. The model is used for assisting a doctor to screen out positive nodules from suspected lung cancer nodules, and interference of false positive nodules on judgment of next good and malignant nodules is reduced.
One embodiment provides that the data for constructing the CT sequences and candidate nodule locations for the screening nodule model is obtained by:
the 888 CT sequences used to train the screening Nodule model were from LUNA16 (Lung Nodule Analysis 16) published data as LIDC/IDRI (the Lung Image Database Consortium Image collection). Among the candidates files in the challenge for reducing false positive nodules provided in LUNA16, 1351 of 551065 nodules were positive nodules with a percentage of non-nodules classified as <3% due to a diameter <3 mm. Each candidate nodule corresponds to a CT file name, world coordinates (x, y, z) and a true and false positive label;
the CT image provided by the LUNA16 is stored by mdh and a raw file of the same name, the mdh file stores header information of the CT image data, such as origin coordinates (Offset) and pixel pitch (elementary spacing), and the raw file stores pixel information.
One embodiment provides a method of preprocessing a model-built data set:
and acquiring the actual position of the candidate nodule in the CT image, and obtaining the position of the nodule in a coordinate system of the CT image according to the world coordinates of the position of the candidate nodule, the origin coordinates in the head information of the CT image and the pixel point distance. Nodule position = (world coordinate-origin coordinate) ÷ pixel dot pitch;
acquiring 2-dimensional image data of a candidate nodule, segmenting an image of 25 pixels in the positive and negative directions of an X axis and a Y axis of the actual position of the candidate nodule, performing HU value processing on the segmented image, setting HU values smaller than-1000 as 0, setting HU values larger than 400 as 1, and scaling the HU values between 0 and 1. Storing the processed 2-dimensional image and the corresponding tag data in the form of an h5py file;
due to the huge imbalance of the positive and negative candidate nodules, the data set randomly selects negative nodules with 5 times of the number of the positive nodules from the negative nodules as negative samples, and rotates the positive nodules by 90 degrees and 180 degrees clockwise to increase the data amount;
and randomly dividing the processed positive and negative samples into three parts serving as a training set, a verification set and a test set, selecting a proper model in the training and verification, and evaluating the false positive rate of the model on the test set.
One embodiment provides that the steps for training the convolutional neural network model are:
a TFLearn open source framework is utilized to construct a convolutional neural network structure, a two-dimensional convolutional layer, a two-dimensional pooling layer and a full-connection layer are adopted in the structure for nonlinear fitting, and a Dropout layer is used for preventing over-fitting;
before the network structure input layer, the image data processing of zero centralization and normalization is carried out on the sample, the training speed of the neural network is improved, and the convergence of the weight parameters is accelerated;
training a classifier model of true and false positive nodules on a neural network structure by using a training set, and comparing the quality of a plurality of models on a verification set to obtain an optimal two-dimensional deep convolution neural network model;
and predicting the probability of the true and false positive nodules of the test set sample by using the trained network model, and distinguishing the true and false positive nodules according to a preset probability threshold value, so that the accuracy of the model in the test set can be obtained, and the ROC curve of the prediction model in the test set sample can be analyzed. The accuracy of the model on the test set reaches 94.45%, and the AUC value reaches 97.1%.
One embodiment provides that the original three-dimensional lung nodule image used to train the predictive model is obtained by:
the CT sequence to which the three-dimensional lung nodule image used to train the predictive model belongs is from the LIDC-IDRI lung nodule public database, which contains 1012 patients, each patient's CT sequence lung nodule being labeled independently by four experienced medical professionals and giving diagnostic opinions;
the lung nodule areas marked by two different doctors may overlap, and for the lung nodule areas with overlapping, whether the two marked areas can be identified as the same lung nodule is judged according to whether the proportion of the overlapping area to the respective marked areas reaches a set threshold (60% is adopted in the training of the invention);
-merging two labeled regions identified as one lung nodule into a new labeled region by merging them, and further merging all different labeled regions identified as one lung nodule into a new labeled region, i.e. the lung nodule region eventually used to obtain the original three-dimensional lung nodule image;
and for the final diagnosis result of the lung nodule region, the diagnosis opinions of different doctors are included, and the final diagnosis conclusion can be obtained by selecting the mean value, the median, the highest frequency opinion and the like.
One embodiment provides that the original three-dimensional lung nodule image used to train the prediction model may be filtered using a laplacian filter and a wavelet filter, each of which has the following effects:
the mathematical form of the laplacian filter operator is:
formula (1)
Wherein
The expression of the laplacian operator is shown,
is the second order partial differential of the image grey value in the x-direction,
is the second order partial differential of the image grey value in the y-direction. The Laplacian operator enables the image gray value to enhance the area with severe gray change in the image, weakens the area with slow gray change, and superimposes the Laplacian image on the original image, so that the effect of sharpening the original image can be achieved. When the method is applied to a three-dimensional lung nodule image, the lung nodule image can be sharpened, and the change details of the image are enhanced;
wavelet transforms can be used as laplace transforms to enhance image-changing details, while having the advantage of resolving noise. In the application, the wavelet function selects the first derivative of Gaussian (Gaussian) shaped pulse, adjusts the scale through the actual image gray scale change, respectively detects the areas with severe gray scale change in the transverse direction and the longitudinal direction, and then sums and applies the areas to the original image to obtain the effect of enhancing the image details.
One embodiment provides for training three-dimensional lung nodule image features of a predictive model, comprising the following features:
first-order features are of interestAnd describing the regional voxel intensity distribution. Such as Energy (Energy = Energy)
) Is the sum of squares of all voxel gray values, and if there is a definite difference between the intensity values of the benign and malignant nodules in the CT image, the energy features can play a key role. Also as Entropy (Encopy = -
) The method is larger in clear images with wide gray scale distribution than in blurred or single-tone images, and can effectively distinguish benign nodules from malignant nodules;
the shape feature is a shape description of the region of interest. Nodules with more distinct differences in shape can be distinguished by characteristics such as Volume (Volume), Surface Area (Surface Area), Maximum three-dimensional diameter (Maximum 3D diameter), and the like;
texture features are a description of the intensity distribution law of voxels in a region of interest, and are usually obtained indirectly based on texture matrices, which are introduced as follows:
a Gray Level Co-occurrence Matrix GLCM (Gray Level Co-occurrence Matrix) is a Matrix of voxel pairs arranged according to a specific rule, for example, if a voxel value m appears p times to the right of a voxel value n in an image, the value of m rows and n columns of the GLCM Matrix is p;
-the gray level Length matrix GLRLM (gray level Run Length matrix) is a matrix of the number of consecutive voxel lengths with the same gray level value, e.g. the gray level value m in an image, the number of horizontally connected voxels of Length n is p, the GLRLM matrix has a value p for m rows and n columns;
-the grey scale area Size matrix GLSZM (gray Level Size zone) is the number of 26 connected voxel forming areas with the same grey scale value, e.g. the number of grey scale values m in the image, 26 connected area sizes n is p, the value of m rows and n columns of the GLSZM matrix is p;
-the gray-Level dependency matrix gldm (gray Level dependency matrix) is the number of dependent voxels in the neighborhood of the central voxel, wherein the neighborhood is selectively enlarged or reduced, and the difference between the gray-Level value of the neighborhood voxel and the gray-Level value of the central voxel is smaller than a predetermined threshold value, which indicates that the neighborhood voxel depends on the central voxel. For example, when a voxel with a gray value m is found in an image as a central voxel, the number of n dependent voxels in the neighborhood is p, and the value of the m row and the n column of the GLDM matrix is p;
-the neighborhood Gray level Difference matrix ngtdm (neighbor Gray Tone Difference matrix) contains the sum of the number of voxels per Gray level, the probability and the absolute value of the Difference from the neighborhood mean Gray level. For example, a voxel with a gray value m, the number in the image is n, the probability is p, and the sum of the absolute values of the differences from the neighborhood mean gray value is s, then the mth behavior of NGTDM [ n, p, s ];
the texture features are based on the texture matrix, and further extracted features such as a gray level region Entropy (Zone Entropy) representing the Entropy value of the gray level region size matrix.
One embodiment is configured to train a three-dimensional lung nodule image feature matrix and lung nodule diagnosis conclusion data of a predictive model, which may be further processed in the following manner:
the same type of features need to be normalized (normalization), i.e. the three-dimensional lung nodule image feature matrix is normalized by columns;
data from single or two equally few physician-labeled lung nodules may be selectively screened out;
-optionally screening for lung nodule data with ambiguous diagnosis intent (LIDC-IDRI dataset limits the suspicious degree of malignancy of a diagnosed lung nodule to 1-5, where a smaller value is more likely to belong to a benign nodule, a larger value is more likely to belong to a malignant nodule, and a value of 3 indicates that the nodule attribute cannot be determined by the physician either);
-the data marked as out of range for the diagnostic decision are screened out.
One embodiment provides that the regularization method of the prediction model can select Lasso, Ridge or ElasticNet, and their respective principles are as follows:
the purpose of diagnosing lung nodules is to judge the benign and malignant nature of the nodule, belonging to a two-class problem, thus assuming that the regularity of the logistic regression model is complied with between the benign and malignant nature of the lung nodule and the image features of the lung nodule, namely:
formula (2)
Wherein p isiIs the probability, x, that the lung nodule i is benign or malignantiIs a vector formed by the image features of the lung nodule i, and β is a coefficient vector for each image feature. Logistic regression models typically estimate coefficients using a Maximum likelihood (Maximum likehood) method, the form of the likelihood function being:
formula (3)
Wherein, yiIs the corresponding probability p in logistic regressioniIs 0 or 1, and n is the number of lung nodules. As can be seen from this equation, the larger the likelihood function value l (β) is, the better the classification effect of the prediction model is, and therefore, the model problem of optimizing the prediction can be converted into the problem of maximizing the likelihood function value. Taking logarithm of two sides of the above formula, and then taking inverse number to obtain:
formula (4)
Substituting formula (2) into it and simplifying:
formula (5)
Derivation of equation (5):
formula (6)
Let equation (6) be 0, the optimal solution for the available likelihood function is:
formula (7)
Since the rank (rank) of X always obeys: rank (X) is less than or equal to min (n, p); at the same time XTThe rank of the X matrix always obeys: rank (X)TX) is less than or equal to rank (X). So, the p × p dimensional matrix XTThe rank of X is always less than or equal to n, X is always less than or equal to n when p > nTX matrix is not full rank and cannot be solved
. In other words, when p > n, L (β) has numerous solutions and we cannot simply minimize the likelihood function to determine which is the best, and one way is to add a constraint within which to determine the best solution, i.e., regularization.
The way of Lasso regularization is to add l in the likelihood function (5)1Regularization, i.e.:
formula (8)
To simplify the solution, consider the case of a single variable, i.e.
Assuming that it is derivable for any value of β, from the property that the extreme point is a differential of 0, the optimal solution can be found as:
formula (9)
I.e. by adding l1The regular limiting condition punishs and constrains each coefficient of the characteristic to ensure thatObtaining the optimal solution of L (beta);
the way of Ridge regularization is to add l to the likelihood function (5)2Regularization, i.e.:
formula (10)
Similar to the Lasso solution process, the optimal solution can be obtained as follows:
formula (11)
I.e. by adding l2And (3) a regular limiting condition, namely punishing and constraining each coefficient of the characteristic so that an optimal solution exists for L (beta).
-ElasticNet is the reaction of l1And l2The regularizations are linearly combined and then added to the mean square error function:
formula (12)
Similar to the Lasso solution process, the optimal solution can be found as follows:
formula (13)
That is, each coefficient of the characteristic is punished and restrained by adding a restriction condition of ElasticNet regular, so that the L (beta) has an optimal solution, and the optimal solution follows the L (beta)
And
may vary.
One embodiment provides that the training step of the predictive model follows the following steps:
-the data set used to construct the predictive model comprises three-dimensional lung nodule image features and physician diagnosis, with 70% randomly selected as the training set for constructing the predictive model and the remaining 30% as the test set;
-selecting different regularization methods to build the prediction model;
inputting different regularization penalty coefficients for the same regularization method, and determining an optimal prediction model under the regularization method by using a k-fold (10-fold in training) cross validation method;
respectively inputting the test set into the prediction model selected under each regularization method, and selecting an optimal model by using the prediction accuracy and AUC of the test set, wherein the prediction accuracy of the optimal model to the test set reaches 92.55%, and the AUC reaches 96.57%.
The method has the advantages that by using the advantage of deep learning, the suspected nodules are judged in advance whether to be the lesion nodules, false positive nodules are screened out, and the efficiency is improved for further diagnosis of the positive nodules. By combining the image segmentation, feature extraction and machine learning methods, the automatic diagnosis method for the pulmonary nodules is provided, the accuracy of the automatic diagnosis result is high, some more important image features are provided for doctors, and reliable reference opinions can be provided for the diagnosis of the doctors.
Drawings
FIG. 1 shows a flow chart of an automated diagnosis of pulmonary nodules.
Fig. 2 shows a flow chart of the construction of a convolutional neural network model for screening lung nodules. A in the figure shows data extraction, the position of a candidate nodule is positioned in a CT sectional image, and a segmented image with a fixed size of 50 multiplied by 50 is obtained; b in the figure shows that the data is preprocessed, all the data is divided into three parts, namely training, verifying and testing, and the training and verifying data is subjected to CNN model fitting and verifying; in the figure C, the loss value and the accuracy of the training and verifying data sets in the training and verifying process are shown; the representation of the test set on the model is shown at D. The construction of a Convolutional Neural Network (Convolutional Neural Network) of this model is shown in E of the figure.
Fig. 3 shows a flowchart for diagnosing a lung nodule. In the figure, a shows a process of acquiring a lung nodule, which includes segmenting the lung nodule from a two-dimensional image and reconstructing the lung nodule into a three-dimensional space; in the figure, B shows a feature extraction process, including preprocessing of the three-dimensional lung nodule image, filtering, i.e., feature extraction; the graph C shows the analysis of the three-dimensional lung nodule image characteristics, and it can be seen from the Cluster heat map that the characteristics are approximately aggregated into two clusters, and the lung nodules are approximately aggregated into two clusters, and the difference is obvious; in the graph D, an ROC graph and an AUC value of the prediction result of the test set by the prediction model are shown, and the prediction effect is good.
FIG. 4 shows a graph of the variation of cross validation error bars (Errorbar) with respect to penalty coefficients when the model uses ElasticNet.
Detailed Description
Fig. 1 shows that the embodiment of the present invention comprises two steps:
-a first step of screening for lung nodules;
-a second step of diagnosing lung nodules.
As shown in fig. 2A, first, the center position of the labeling region of the suspected nodule of the doctor is corresponded to the CT sequence, the center position coordinates are converted into voxel coordinates and corresponded to the 3-dimensional CT image, and the suspected nodule is cut out into a 2-dimensional image of 50 × 50 size according to the center position coordinates of the suspected nodule.
And performing HU conversion on the 2-dimensional image, setting HU larger than 400 as 1, setting HU smaller than-1000 as 0, and scaling the HU between 0 and 1 in the middle. The pixels are then converted into grayscale image pixels.
As shown in fig. 2B, the sample is subjected to zero-centering and normalization processing, and is input into the trained convolutional neural network prediction model to obtain the true and false positive probability of the sample, and the true and false positive nodules are distinguished according to a preset probability threshold. The CT image and the corresponding nodule physician label information are recorded and used as the input of the second step of diagnosing the nodule.
As described in fig. 3A, after the lung nodules are screened, the lung nodules are first segmented by using the physician labeling information that is confirmed to be the lung nodules in combination with the CT sequence, which specifically includes the following steps:
-associating the doctor-annotated lung nodule boundary with the CT tomogram, i.e. corresponding the z-coordinate of the lung nodule boundary coordinates to the z-coordinate of the CT tomogram, to ensure that the boundary delineating points are located on the correct CT tomogram;
-associating the pulmonary nodule boundary coordinates marked by the doctor with pixel points on the CT tomographic image, and obtaining the boundary coordinates and all pixel points inside the boundary coordinates by using an image processing morphological dilation method, thereby obtaining the coordinates of the whole pulmonary nodule boundary and inside and the gray value corresponding to the coordinates;
when a plurality of doctors label the same CT sequence or the regions labeled by the same doctor overlap, calculating the repetition degree of the coordinates of the overlapping region (the proportion of the number of the coordinates of the overlapping region to the number of the coordinates of each lung nodule labeling region), merging the two lung nodules into the same lung nodule for processing if the repetition degree is greater than a set threshold (60%), and otherwise, respectively processing the two lung nodules separately if the repetition degree is less than the set threshold;
distributing the acquired lung nodule coordinates and the gray value corresponding to each lung nodule coordinate to a three-dimensional space (three-dimensional matrix), wherein the value of the coordinate point labeled by the doctor is the corresponding gray value, and the rest points are filled with 0, so that a three-dimensional image only containing one lung nodule can be acquired.
As depicted in fig. 3B, after the three-dimensional lung nodule image is obtained by segmentation, the image features are extracted according to the following steps:
-first resampling the three-dimensional lung nodule image, the purpose of the resampling being to cause distances between neighboring voxels to be scaled to the same size, reducing the interference of the CT image acquisition device on segmenting the lung nodule size, such that the lung nodule is as true size as possible;
shearing off 0 filling areas around the lung nodules to reduce the three-dimensional matrix containing the lung nodules to a minimum amount, wherein when the 0 filling areas are large, although the extracted image features cannot be influenced, the calculation amount of feature extraction is greatly increased, the running speed of the computer is reduced, and even the running memory of the computer is insufficient, so that the computer is broken down;
for three-dimensional lung nodulesThe node image is filtered using a laplacian filter and a wavelet filter, both based on a gaussian function
The value must be set to be greater than 0 and the size of any dimension of the three-dimensional lung nodule image must be greater than
A value;
extracting three-dimensional lung nodule image related features including three types of features including first-order features, shape features and texture features by using a feature extractor, but the images after laplace filtering and wavelet filtering do not need to extract the shape features because the shape and the size of the filtered images relative to the original images do not change, and data are repeated if the images are extracted again.
After the characteristics are extracted, the lung nodules are diagnosed by utilizing the three-dimensional lung nodule image characteristics, the probability that the lung nodules belong to benign or malignant can be obtained by inputting the lung nodules into the prediction model, and different benign and malignant threshold values can be set according to actual conditions to classify the benign and malignant lung nodules. For example, in predicting the probability of benign, it is expected that the probability of fewer False-positive (diagnosed benign nodules are actually malignant nodules) will be lower, and the threshold may be raised appropriately to ensure that more lung nodules are not misdiagnosed as benign.
While the invention has been shown and described in further detail with reference to preferred embodiments thereof, the invention is not limited to the examples disclosed, and other variations can be made by those skilled in the art without departing from the scope of the invention.
Claims (9)
1. A method for automated diagnosis of pulmonary nodules comprising the method steps of:
1) segmenting a suspected nodule area to obtain a two-dimensional nodule image;
2) introducing the two-dimensional nodule image into a neural network prediction model, and distinguishing true and false positive nodules;
3) segmenting a lung nodule region to obtain a three-dimensional lung nodule image;
4) extracting three-dimensional lung nodule image features;
5) and (4) introducing the nodule image characteristics as input parameters into a prediction model to diagnose the benign and malignant lung nodules.
2. The method 1 according to claim 1, wherein the method for segmenting the suspected nodule region is to calculate the central position of the labeled region of a single slice in the CT image according to the labeling information of the physician on the lung nodule region and the corresponding CT sequence, and then take 25 pixels in the positive and negative directions of the X axis and the Y axis according to the central position as the suspected nodule image information to be detected, perform HU value processing on the image information of the CT image, and convert the image information into a gray image.
3. The method 2 according to claim 1, wherein the two-dimensional nodule image is introduced into the neural network model by performing zero-centering and normalization processing on the processed two-dimensional image, then the processed two-dimensional image is used as an input of the network model obtained by training, the probability of true and false positive nodules is predicted, and the true and false positive nodules are distinguished according to a set threshold value.
4. The method 3 according to claim 1, wherein the method for segmenting the lung nodule region is to segment the lung nodule region by combining with the patient CT sequence according to labeling information of a doctor on the lung nodule region, the labeling information of the doctor specifies nodule region boundary coordinates, and the boundary coordinates of each CT tomogram correspond to the CT tomograms; the method for acquiring the three-dimensional lung nodule image comprises the steps of distributing gray values corresponding to coordinates of the boundaries and the internal coordinates of the lung nodule marked by a doctor into a three-dimensional matrix, wherein the obtained three-dimensional matrix is the three-dimensional lung nodule image; when more than one physician is used for labeling, the labeled nodule regions may overlap, in this case, according to whether the proportion of the nodule overlapping region in each nodule region is greater than a set threshold (60%), whether two nodule regions marked by segments belong to the same nodule is judged, if the two nodule regions belong to the same nodule, the two nodules are combined into a new nodule in a union mode, and if the two nodule regions do not belong to the same nodule, the two nodule regions are separately processed according to the respective nodule regions.
5. The method 4 according to claim 1, wherein the pre-processing of extracting three-dimensional lung nodule image features includes resampling, clipping and filtering processing of the three-dimensional lung nodule image; resampling and scaling the distance between adjacent voxels of the three-dimensional lung nodule image to the same size to ensure that all three-dimensional lung nodule images have the same size proportion with the real lung nodule; cropping is the scaling of the three-dimensional lung nodule image to the smallest cube that can contain lung nodules, minimizing the volume of the three-dimensional matrix; filtering is to convolve the original three-dimensional nodule image with a Filter such as a Laplacian Filter (Laplacian Filter) or a Wavelet Filter (Wavelet Filter).
6. The method 4 according to claim 1, wherein the features are extracted from the three-dimensional lung nodule image by a feature extractor, and the extracted three-dimensional lung nodule image features include, but are not limited to, the following three categories:
-First Order features (First Order) of the three-dimensional lung nodule image, including basic statistical features such as mean, variance and entropy of image gray values;
-Shape features (Shape) of the three-dimensional lung nodule image, including image volume, surface area and sphericity features;
-Texture features (Texture) of the three-dimensional lung nodule image comprising image gray level co-occurrence matrix (GLCM) derived features, image gray level length matrix (GLRLM) derived features, image gray level region size matrix (GLSZM) derived features, image Gray Level Dependency Matrix (GLDM) derived features, image neighborhood gray level difference matrix (NGTDM) derived features.
7. The method 5 of claim 1, wherein the training set used to train the predictive model is a nodule image feature and a corresponding diagnosis including processed results of labeling information from a plurality of physicians, and the processing of the training set includes selectively excluding ambiguous results or less ambiguous results from physician labeling information; inputting the lung nodule image characteristics into a prediction model, obtaining the probability that the lung nodule image characteristics are diagnosed as benign or malignant, and obtaining a benign and malignant diagnosis result of the lung nodule according to comparison with a set probability threshold (generally 0.5); in practical applications, the threshold value may be selected according to different scenarios to achieve a specific purpose.
8. The method of claim 7, wherein the prediction model is constructed based on a Logistic Regression (Logistic Regression) model, the regularization method is elastic network (ELasticNet), and the regularization method for training the prediction model is selected from the group consisting of i 1 regularization, i 2 regularization, and elastic network according to the prediction AUC and accuracy optimality.
9. The automated diagnostic method comprising the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911055430.2A CN112819747A (en) | 2019-10-31 | 2019-10-31 | Method for automatically diagnosing benign and malignant nodules based on lung tomography image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911055430.2A CN112819747A (en) | 2019-10-31 | 2019-10-31 | Method for automatically diagnosing benign and malignant nodules based on lung tomography image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112819747A true CN112819747A (en) | 2021-05-18 |
Family
ID=75851338
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911055430.2A Pending CN112819747A (en) | 2019-10-31 | 2019-10-31 | Method for automatically diagnosing benign and malignant nodules based on lung tomography image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112819747A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113554640A (en) * | 2021-07-30 | 2021-10-26 | 四川大学华西医院 | AI model training method, use method, computer device and storage medium |
| CN114266774A (en) * | 2022-03-03 | 2022-04-01 | 中日友好医院(中日友好临床医学研究所) | Method, equipment and system for diagnosing pulmonary embolism based on flat-scan CT image |
| CN114758755A (en) * | 2022-06-14 | 2022-07-15 | 数聚(山东)医疗科技有限公司 | Medical data collaborative management platform based on big data analysis |
| CN115294401A (en) * | 2022-08-23 | 2022-11-04 | 北京医准智能科技有限公司 | CT image-based classification method, device, equipment and storage medium |
| CN115661132A (en) * | 2022-12-05 | 2023-01-31 | 泸州职业技术学院 | Computer storage medium and device for detecting benign and malignant pulmonary nodules |
| CN116309514A (en) * | 2023-03-31 | 2023-06-23 | 广东省人民医院 | Method for constructing benign and malignant predictive model of pulmonary nodule, model and diagnostor |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110142301A1 (en) * | 2006-09-22 | 2011-06-16 | Koninklijke Philips Electronics N. V. | Advanced computer-aided diagnosis of lung nodules |
| CN107016665A (en) * | 2017-02-16 | 2017-08-04 | 浙江大学 | A kind of CT pulmonary nodule detection methods based on depth convolutional neural networks |
| US20170337687A1 (en) * | 2015-11-30 | 2017-11-23 | Shanghai United Imaging Healthcare Co., Ltd. | System and method for computer aided diagnosis |
-
2019
- 2019-10-31 CN CN201911055430.2A patent/CN112819747A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110142301A1 (en) * | 2006-09-22 | 2011-06-16 | Koninklijke Philips Electronics N. V. | Advanced computer-aided diagnosis of lung nodules |
| US20170337687A1 (en) * | 2015-11-30 | 2017-11-23 | Shanghai United Imaging Healthcare Co., Ltd. | System and method for computer aided diagnosis |
| CN107016665A (en) * | 2017-02-16 | 2017-08-04 | 浙江大学 | A kind of CT pulmonary nodule detection methods based on depth convolutional neural networks |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113554640A (en) * | 2021-07-30 | 2021-10-26 | 四川大学华西医院 | AI model training method, use method, computer device and storage medium |
| CN114266774A (en) * | 2022-03-03 | 2022-04-01 | 中日友好医院(中日友好临床医学研究所) | Method, equipment and system for diagnosing pulmonary embolism based on flat-scan CT image |
| CN114758755A (en) * | 2022-06-14 | 2022-07-15 | 数聚(山东)医疗科技有限公司 | Medical data collaborative management platform based on big data analysis |
| CN114758755B (en) * | 2022-06-14 | 2022-08-16 | 数聚(山东)医疗科技有限公司 | Medical data collaborative management platform based on big data analysis |
| CN115294401A (en) * | 2022-08-23 | 2022-11-04 | 北京医准智能科技有限公司 | CT image-based classification method, device, equipment and storage medium |
| CN115294401B (en) * | 2022-08-23 | 2023-04-14 | 北京医准智能科技有限公司 | CT image-based classification method, device, equipment and storage medium |
| CN115661132A (en) * | 2022-12-05 | 2023-01-31 | 泸州职业技术学院 | Computer storage medium and device for detecting benign and malignant pulmonary nodules |
| CN116309514A (en) * | 2023-03-31 | 2023-06-23 | 广东省人民医院 | Method for constructing benign and malignant predictive model of pulmonary nodule, model and diagnostor |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112819747A (en) | Method for automatically diagnosing benign and malignant nodules based on lung tomography image | |
| EP3432263B1 (en) | Semantic segmentation for cancer detection in digital breast tomosynthesis | |
| CN107016665B (en) | CT pulmonary nodule detection method based on deep convolutional neural network | |
| CN106940816B (en) | CT image pulmonary nodule detection system based on 3D full convolution neural network | |
| US9123095B2 (en) | Method for increasing the robustness of computer-aided diagnosis to image processing uncertainties | |
| CN112101451B (en) | Breast cancer tissue pathological type classification method based on generation of antagonism network screening image block | |
| CN104751178A (en) | Pulmonary nodule detection device and method based on shape template matching and combining classifier | |
| US20120237109A1 (en) | Histology analysis | |
| Reshma et al. | Detection of breast cancer using histopathological image classification dataset with deep learning techniques | |
| JP2003532934A (en) | Method and system for automatic detection of clustered microcalcifications from digital mammograms | |
| Jaffar et al. | Fuzzy entropy based optimization of clusters for the segmentation of lungs in CT scanned images | |
| CN111415352B (en) | Cancer metastasis panoramic pathological section analysis method based on deep cascade network | |
| Fan et al. | Lung nodule detection based on 3D convolutional neural networks | |
| Liao et al. | A segmentation method for lung parenchyma image sequences based on superpixels and a self-generating neural forest | |
| CN104616289A (en) | Removal method and system for bone tissue in 3D CT (Three Dimensional Computed Tomography) image | |
| CN114092450A (en) | Real-time image segmentation method, system and device based on gastroscopy video | |
| CN112820399A (en) | Method and device for automatically diagnosing benign and malignant thyroid nodules | |
| Pradhan et al. | Lung cancer detection using 3D convolutional neural networks | |
| Kousalya et al. | Improved the detection and classification of breast cancer using hyper parameter tuning | |
| Malik et al. | Lung cancer detection at initial stage by using image processing and classification techniques | |
| Srivastav et al. | Breast cancer detection in mammogram images using machine learning methods and clahe algorithm | |
| Mastouri et al. | A morphological operation-based approach for Sub-pleural lung nodule detection from CT images | |
| Fauci et al. | A massive lesion detection algorithm in mammography | |
| CN114693698B (en) | Computer-aided lung airway segmentation method based on neural network | |
| Lee et al. | Automated identification of lung nodules |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |