CN110458801A - A kind of 3D dual path neural network and the pulmonary nodule detection method based on the network - Google Patents
A kind of 3D dual path neural network and the pulmonary nodule detection method based on the network Download PDFInfo
- Publication number
- CN110458801A CN110458801A CN201910551583.XA CN201910551583A CN110458801A CN 110458801 A CN110458801 A CN 110458801A CN 201910551583 A CN201910551583 A CN 201910551583A CN 110458801 A CN110458801 A CN 110458801A
- Authority
- CN
- China
- Prior art keywords
- dual path
- neural network
- tubercle
- lung
- path neural
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30061—Lung
- G06T2207/30064—Lung nodule
Abstract
The invention discloses a kind of 3D dual path neural network, the general frame of the 3D dual path neural network is class U-net structure, and the connection in the 3D dual path neural network is dual path connection structure.The invention also discloses the pulmonary nodule detection methods based on 3D dual path neural network, which comprises the following steps: constructs 3D dual path neural network above-mentioned；Medical image in pretreated training set is inputted into the 3D dual path neural network, to be trained to the 3D dual path neural network, until each index achieves the desired results；Medical image to be processed is input to the 3D dual path neural network, obtains testing result.3D dual path neural network proposed by the present invention and pulmonary nodule detection method based on the network model can reduce the requirement to Lung neoplasm training data, improve the detection accuracy to medical image Lung neoplasm, and can be used for computer-aided diagnosis system.
Description
Technical field
The present invention relates to field of medical image processing more particularly to a kind of 3D dual path neural network and it is based on the network mould
The pulmonary nodule detection method of type.
Background technique
Clinically, the lesion (lymph node of atelectasis and enlargement except needing) that 3cm is less than or equal in pulmonary parenchyma is known as tying
Section, the lesion greater than 3cm is known as lump, and the tubercle less than 1cm is known as lesser tubercle, and the tubercle less than 5mm is known as micronodule,
Why in this way definition be because greater than 3cm lesion it is mostly pernicious, and smaller lesion may be it is benign or malignant, tubercle
Size has correlation with the good, pernicious of tubercle.
Lung neoplasm is the early forms of lung cancer, over time part Lung neoplasm occur malignant change (angiosynizesis,
Volume becomes larger) it is converted into lung cancer, seriously threaten the health of the mankind.Worldwide, lung cancer death is most common leads
One of lethal the reason of dying.In China, lung cancer is the death rate and the highest malignant tumour of disease incidence.Operation is still uniquely may be used at present
To eradicate the treatment method of lung cancer.The lung cancer of early stage is no any symptom, and the lung cancer patient of most of China is once finding
It is advanced stage, the chance of operative treatment is lost, therefore early detection lung cancer can only be carried out by physical examination, if Small pulmonary nodule is made a definite diagnosis
For the early stage of lung cancer, operation excision is carried out, survival rate can reach 80% or more within 5 years.Inspection for lung cancer shares four classes: image
Inspection, bronchoscopy, phlegm inspection, the biochemical marker inspection of serum and other body fluid are learned, chest x-ray and CT examination are diagnosis chests
The most common means of portion's disease.Up to the present, result of study shows that only CT examination can reduce lung cancer mortality, and the U.S. is large-scale
Randomized controlled trial has proven to that lung cancer mortality can be reduced using low-dose CT progress lung cancer inspection.
Lung neoplasm is a kind of common pulmonary disease, and the cause of disease is complicated, and clinical manifestation lacks specificity, therefore diagnosis
It acquires a certain degree of difficulty, in recent years, becoming increasingly popular and develop with spiral CT, the recall rate of nodular lesion significantly improves, and is less than 2mm
Small pulmonary artery can be found, increase the difficulty of diagnosis, at the same also to patient whether receive treatment bring it is certain
Puzzlement.In addition, for lung CT image, the picture of up to a hundred DICOM formats can be once generated, doctor needs to read these figures
Piece determines that lesion diagnoses the state of an illness, but picture number is big, needs to take a significant amount of time and goes through, and with current doctor
Learn the continuous development and improvement, the increase of clinical demand of image technology, the trend that medical image data increases still clearly,
This causes the working strength of diagnostician significantly to rise, and causes the fatigue of these medical workers, and then causes them to doctor
Reading efficiency and the quality decline for learning tired picture, cause the mistaken diagnosis of certain probability, fail to pinpoint a disease in diagnosis.
The disclosure of background above technology contents is only used for auxiliary and understands design and technical solution of the invention, not necessarily
The prior art for belonging to present patent application, no tangible proof show above content present patent application the applying date
In disclosed situation, above-mentioned background technique should not be taken to the novelty and creativeness of evaluation the application.
Summary of the invention
In order to solve the above technical problems, the present invention proposes a kind of 3D dual path neural network and the lung based on the network model
Portion's nodule detection methods can reduce the requirement to Lung neoplasm training data, improve the detection accuracy to medical image Lung neoplasm, and
It can be used for computer-aided diagnosis system.
In order to achieve the above object, the invention adopts the following technical scheme:
The invention discloses a kind of 3D dual path neural network, the general frame of the 3D dual path neural network is class U-
Net structure, and the connection in the 3D dual path neural network is dual path connection structure.
Preferably, dual path connection structure therein is by combining residual error study and intensive in 3D convolutional neural networks
Connection and formed.
Preferably, dual path connection structure therein is specifically stated are as follows:
Y=G (x [: d], F (x) [: d], F (x) [d :]+x [d :])
Wherein, y is the Feature Mapping of dual path connection structure, and G is ReLU activation primitive, and F is convolution layer functions, and x is double
The input of path connection structure, and F (x) [d :] learns for residual error, F (x) [: d] for intensively connecting, d is determined using new
The hyper parameter of the quantity of feature.
The invention also discloses a kind of pulmonary nodule detection methods based on 3D dual path neural network, including following step
It is rapid:
S1: above-mentioned 3D dual path neural network is constructed；
S2: the medical image in pretreated training set is inputted into the 3D dual path neural network, to institute
It states 3D dual path neural network to be trained, until each index achieves the desired results；
S3: medical image to be processed is input to the 3D dual path neural network, obtains testing result.
Preferably, the training set in step S2 include chest medical image and corresponding diagnostic result lesion
Markup information, wherein diagnostic result lesion markup information includes non-nodules lesion, tubercle diameter is less than 3mm, tubercle diameter is greater than
Or it is equal to 3mm three classes.
Preferably, the pretreatment in step S2 specifically includes: becoming world coordinates to the medical image in training set
Voxel coordinate is changed to unified spacing, extracts the bianry image of complete lung areas to obtain one for entire lung areas
The minimum cube that all surrounds simultaneously obtains the borderline region of lung；Then reduce the shadow of the Pixel Information to nodule detection on boundary
It rings, and removes extraneous areas to the detection of tubercle and the influence of arithmetic speed, finally handle markup information.
Preferably, the borderline region for obtaining lung areas includes carrying out the left-right parts of lung at expansion respectively
Reason, obtains entire lung areas and all carries out the image after expansion process, itself and lung's exposure mask are then carried out xor operation and obtained
The borderline region of lung；Further, expansion process includes: first to judge whether to deposit to each slice of medical image
In lung areas, if it is present first carrying out convex closure operation: if the pixel that pixel value is 1 after convex closure operates is greater than original
1.5 times of beginning pixel number purpose, will remove convex closure operating result, retains original image, the binary map after otherwise retaining convex closure operation；Such as
There is no lung areas in fruit slice, then does nothing.
Preferably, influence of the Pixel Information for reducing boundary to nodule detection includes by the picture in medical image
The range processing of plain value narrows down between 0 to 255, the pixel value in the place of lung's masked areas after will not belong to expansion process
It is set as 170；Further, the removal extraneous areas includes: to medicine to the detection of tubercle and the influence of arithmetic speed
Image data carries out the operation of bilinearity difference up-sampling to carry out the expansion of preset range, then intercepts from the image after expansion
The sub-cube block of predefined size.
Preferably, the processing markup information includes: the conversion for carrying out coordinate system first, to the markup information of tubercle according to
Nodule boundary after expansion converts the initial coordinate of tubercle, retain tubercle diameter information and original markup information it is big
It is small.
Preferably, in step S2 before medical image to be inputted to the 3D dual path neural network also to medicine figure
As data are balanced, further, equilibrium step includes: the mark letter by tubercle diameter d within the scope of 2.5mm < d < 10mm
Breath retains a parts, and the tubercle information within the scope of 10mm<d<20mm retains b parts, and the tubercle information within the scope of d>20mm retains c parts, so
The tubercle markup information after expanding again is stored afterwards, wherein a, b, c are positive integer, and a < b≤c；Further, a
=1, b=2,3,4 or 5, c=5,6,7,8 or 9.
Compared with prior art, the beneficial effects of the present invention are the nets of 3D dual path neural network proposed by the present invention
Network frame uses class U-net type structure, and connection is dual path connection structure, and wherein 3D module facilitates the medical treatment figure to lung
As aggregation of data analysis, feature is extracted, dual path connection structure had both facilitated the recycling of feature, also new feature easy to use
Advantage, class U-net type structure merges the feature of different scale, so that the 3D dual path neural network is easy to
It realizes, the calculation amount of training pattern is relatively small, lower to requiring for trained computer equipment；And further it is being applied to
The precision that model when pulmonary nodule detects relative to other equivalent parameters amounts detects Small pulmonary nodule is higher；It is bis- by the 3D
Path neural network alleviates the requirement to Lung neoplasm training data, improves the detection accuracy to medical image Lung neoplasm, can
For computer-aided diagnosis system；To solve traditional computer visual field to the defect of medical image analysis and to doctor
The high request of raw professional knowledge.
Detailed description of the invention
Fig. 1 is the connection schematic diagram of the dual path connection structure of the preferred embodiment of the present invention；
Fig. 2 is the structural schematic diagram of each layer of the D dual path neural network of a specific embodiment of the invention；
Fig. 3 is the CT slice schematic diagram that a sample is included；
Fig. 4 is the 3D FROC curve graph that 3DDPN26 network training 150 times are used in specific example of the present invention；
Fig. 5 is in specific example of the present invention using 3D DPN 100 corresponding true positive rate TPR of training and true negative rate
TNR；
Fig. 6 a to Fig. 6 d is the visual of Lung neoplasm testing result in the 4 CT images randomly selected in specific example of the present invention
Change result.
Specific embodiment
Below against attached drawing and in conjunction with preferred embodiment, the invention will be further described.
The preferred embodiment of the present invention discloses a kind of 3D dual path neural network, the whole frame of the 3D dual path neural network
Frame is class U-net structure, and the connection in the 3D dual path neural network is dual path connection structure.
Class U-net structure therein refers to the neural network model frame of the similar alphabetical " u "-shaped of structure, passes through such U-
The integral frame structure of net structure so that 3D convolutional neural networks have used jump to connect in same stage, rather than directly exists
Backpropagation is exercised supervision and lost in high-level semantics feature, ensures that the characteristic pattern finally recovered has merged more in this way
Feature is stitched together in channel dimension but also the feature of different scale is merged by the feature of more low scales.
Dual path connection structure therein connects particular by 3D convolutional neural networks in conjunction with residual error study and intensively
What the advantages of connecing was formed, that is, the Feature Mapping of 3D convolutional neural networks is divided into two parts, F (x) by dual path connection structure
[d :] learns for residual error, F (x) [: d] for intensively connecting, d is the hyper parameter for determining the quantity using new feature, specifically,
Dual path connection structure can be expressed as:
Y=G (x [: d], F (x) [: d], F (x) [d :]+x [d :])
Wherein, y is the Feature Mapping of dual path connection structure, and G is ReLU activation primitive, and F is convolution layer functions, and x is double
The input of path connection structure.
It is the mathematical expression of each coupling part in dual path connection structure represented by above-mentioned function, is number with the function
The connection for learning basis is the pith for constituting 3D convolutional neural networks model, and specific connection type is as shown in Figure 1, by feature
It is stitched together in channel (channel) dimension, forms thicker feature.
In a specific embodiment, the 3D dual path neural network is by several convolutional layers, pond layer, active coating, warp
The operations composition such as the laminated structures such as lamination and matrix splicing, dimension transformation, as shown in Fig. 2, the 3D dual path neural network is each
The structure of layer is as follows:
First layer: 3D convolutional layer, convolution kernel size are 3*3*3, and quantity 24 and is filled with 1 at step-length, for exporting 24
Characteristic pattern；
The second layer: dual path layer, intermediate channel 24, output channel 32, step-length 2, for exporting 24 characteristic patterns；
Third layer: dual path layer, intermediate channel 24, output channel 32, step-length 1, for exporting 24 characteristic patterns；
4th layer: dual path layer, intermediate channel 48, output channel 56, step-length 2, for exporting 56 characteristic patterns；
Layer 5: dual path layer, intermediate channel 48, output channel 56, step-length 1, for exporting 56 characteristic patterns；
Layer 6: dual path layer, intermediate channel 72, output channel 80, step-length 2, for exporting 80 characteristic patterns；
Layer 7: dual path layer, intermediate channel 72, output channel 80, step-length 1, for exporting 80 characteristic patterns；
8th layer: dual path layer, intermediate channel 96, output channel 104, step-length 2, for exporting 104 characteristic patterns；
9th layer: dual path layer, intermediate channel 96, output channel 104, step-length 1, for exporting 104 characteristic patterns；
Tenth layer: warp lamination, output channel 120, for exporting 120 characteristic patterns；
Eleventh floor: splicing layer is connected for the 9th layer and layer 7 to be exported in the dimension of channel, for exporting 216
Characteristic pattern；
Floor 12: dual path layer, intermediate channel 128, output channel 136, step-length 2, for exporting 136 features
Figure；
13rd layer: dual path layer, intermediate channel 128, output channel 136, step-length 1, for exporting 136 features
Figure；
14th layer: warp lamination, output channel 152, for exporting 152 characteristic patterns；
15th layer: splicing layer is connected for the 14th layer and layer 5 to be exported in the dimension of channel, for exporting 227
A characteristic pattern；
16th layer: dual path layer, intermediate channel 224, output channel 232, step-length 2, for exporting 232 features
Figure；
17th layer: dual path layer, intermediate channel 224, output channel 232, step-length 1, for exporting 248 features
Figure；
18th layer: 3D convolutional layer, convolution kernel size are 1*1*1, and quantity 64 and is filled with 1 at step-length, for exporting 64
Characteristic pattern；
19th layer: 3D convolutional layer, convolution kernel size are 1*1*1, and quantity 15 and is filled with 1 at step-length, for exporting 15
Characteristic pattern；
Wherein, there is Rule activation primitive between each layer；1 dual path layer includes 4 3D convolutional layers, specific connection type
As shown in Figure 1.
In further preferred embodiment of the present invention, a kind of pulmonary nodule based on 3D dual path neural network is disclosed
Detection method, comprising the following steps:
S1: the 3D dual path neural network in building above preferred embodiment；
S2: inputting 3D dual path neural network for the medical image in the training set after pretreatment and equilibrium data,
To be trained to 3D dual path neural network, until each index achieves the desired results；
Wherein the pretreatment of data specifically includes: obtaining CT picture of patient, and is by world coordinate transformation to image data
After voxel coordinate unifies spacing, one is obtained by minimum cube that entire lung areas is all surrounded and obtains the frontier district of lung
Domain removes extraneous areas to the detection of tubercle and the influence of arithmetic speed: due to the size of the cube block of each CT image
It is different, therefore the up-sampling operation of bilinearity difference first should be carried out to CT image, a certain range expansion then is carried out to cube block
Some small sub-cube blocks are intercepted afterwards；Then it handles markup information: carrying out the conversion of coordinate system first, coordinate system is carried out to it
Conversion after retain the diameter information of tubercle and the size of original markup information, then according to being not empty diameter information in mark
Carry out equilibrium data, original image is finally cut into some small sub-cube blocks at random as the input of network, and by cube block
Location information be input to 3D dual path neural network together and be trained.
Wherein equilibrium data specifically includes: obtaining tubercle diameter d in 2.5mm < d according to the markup information of existing data
The data of < 10mm are relatively more, and the data of d > 10mm are fewer, therefore in a specific example, by 2.5mm < d <
The tubercle markup information of 10mm retains 1 part, and the tubercle markup information of 10mm < d < 20mm retains 3 parts, the tubercle mark of d > 20mm
It infuses information and retains 7 parts, then store the tubercle markup information after expanding again.The number of d > 10mm is wherein greater than 1
Part purpose be to eliminate in medical image this few with respect to sample of impressions data volume of positive sample data volume
Extreme uneven, the number retained is not necessarily 3 parts or 7 parts, and specific number should be depending on the circumstances, for example,
The tubercle markup information of 10mm < d < 20mm can also retain 2,4 or 5 parts, the tubercle markup information of d > 20mm can also retain 5,6,
8 or 9 parts.
S3: medical image to be processed is input to the 3D dual path neural network, obtains testing result.
The medical image to be processed wherein inputted can also equally carry out the pretreatment in step S2.
The pulmonary nodule detection method based on 3D dual path neural network of the following pairs of preferred embodiment of the present invention is made into one
Walk explanation.
(1) training data is obtained:
LIDC-IDRI data are to initiate to receive by National Cancer Institute (National Cancer Institute)
Collection, in order to study the detection of people at highest risk's early-stage cancer；The data set is by chest medical image files (such as CT, X-ray
Piece) and corresponding diagnostic result lesion mark composition.In the data set, 1018 case studies, LIDC-IDRI number have been included altogether
The annotation that 4 experienced radiologist collect in two stages annotation procedure is further comprised according to library.Every radiologist
Label be divided into three classes: non-nodules lesion, tubercle diameter be less than 3mm, tubercle diameter be greater than 3mm.Wherein reference standard is by 4
At least 3 receive institute nodosity composition of the tubercle diameter greater than 3mm in name radiologist；It is not included in reference standard
Annotation (tubercle that non-nodules, tubercle are less than 3mm or are only annotated by 1 or 2 radiologist) is considered as incoherent knot
Fruit.LUNA16 data set is the largest a subset of Lung neoplasm public data collection LIDC-IDRI, uses Creative
Commons Attribution 3.0Unported (knowledge sharing signature 3.0 is non-localized) license.LUNA16 data set only wraps
The annotation information containing detection, and LIDC-IDRI then includes the relevant information of nearly all low-dose scan in lung CT, including doctor is big to tubercle
Small, position, diagnostic result, nodular structure, tubercle edge and other information annotation.LUNA16 data set eliminates LIDC-
Slice thickness is greater than 2.5mm in IDRI, and slice spacing is inconsistent or the scan image of the CT of missing, and clearly provides this data set
10 times of cross validations segmentation in patient's level；Therefore data include 888 low dosage CT lung scans in total.
In this embodiment, data set organization is as follows:
Subset0.zip to subset9.zip: 10 zip files comprising all CT images；Annotations.csv:
Csv file comprising being used as " nodule detection " reference standard annotation；SampleSubmission.csv: the correct text for submitting format
Part example；V2.csv: the csv file comprising false positive reduction position candidate；
Other data include: assessment script file folder used in evaluation script:LUNA16 frame；lung_
Segmentation: the catalogue of the lung segmentation comprising the CT image for using automatic algorithms to calculate；additional\_
Annotations.csv: the csv file comprising the additional tubercle annotation from observational study；Complete data set is divided into 10
A subset is used for 10 times of cross validations；All subsets are .zip file.
Wherein, in each subset, CT image is all with the storage of MetaImage (mhd/raw) format.Each .mhd file
Individually stored together with pixel scale data, that is.raw binary file.
(2) preprocessing process of training data
Primary data information (pdi) is read, is after voxel coordinate unifies spacing, to extract complete lung by world coordinate transformation
The bianry image in region then obtains one for minimum cube that entire lung areas is all surrounded and obtains the frontier district of lung
Domain；After the Pixel Information for reducing boundary is to the influence of nodule detection, detection and operation of the extraneous areas to tubercle are got rid of
The influence of speed；It finally handles markup information: carrying out the conversion of coordinate system first, believe then according to the diameter in mark not for sky
Breath carrys out equilibrium data, and original image is finally cut into some small cubic blocks at random as the input of network, and by the position of cubic block
Confidence breath is passed to network together and is trained.It is the specific implementation process of each step below:
It obtains the borderline region of lung: the left-right parts of lung is subjected to expansion process respectively, method particularly includes: CT is schemed
Each slice of picture first judges whether there is lung areas, if in the presence of, then convex closure operation is first carried out, if by convex closure
The pixel that pixel value is 1 after operation is greater than original pixels and counts 1.5 times of purpose and will remove convex closure operating result, retains original image,
Otherwise the binary map after reservation operations then does nothing, is sentenced by aforesaid operations if not having lung areas in slice
Break and expansion process is carried out to entire CT image after all slices, has obtained entire lung areas and all carry out the figure after expansion process
Then itself and lung's exposure mask are carried out xor operation and obtain the borderline region of lung, CT included in one of sample by picture
It is as shown in Figure 3 to be sliced schematic diagram.
It removes influence of the Pixel Information on boundary to nodule detection: the range of the pixel value in CT image then being handled into contracting
The small pixel value to the place that between (0,255), will not belong to lung's masked areas after expanding is set as 170, due to boundary
Expansion obtain, so need by pixel value in lung's borderline region it is bigger be set as 170, in order to reduce the picture on boundary
Influence of the prime information to nodule detection.
Remove detection of the extraneous areas to tubercle: after carrying out the up-sampling operation of bilinearity difference to CT image, to its into
Row a certain range expand after, interception expand after image in a certain size cube block (cube block of each CT image
It is in different size), in order to extraneous areas is removed to the detection of tubercle and the influence of arithmetic speed.
Handle tubercle markup information: first carry out coordinate system conversion, will be according to expansion to the markup information of tubercle after
Nodule boundary the initial coordinate of tubercle is converted, retain the diameter information of tubercle and the size of original markup information.
(3) equilibrium data and training network model
It extracts from pretreated data to trained data, when there is no y tubercle mark in data, then the data
Corresponding markup information is [0,0,0,0], is finally empty when storage；Then respectively by the mark of each example of all training datas
It infuses in information for empty data through the information of wherein diameter come equilibrium data: can be with according to the markup information of existing data
Obtain that tubercle diameter 2.5mm < d < 10mm data are relatively more, and it is fewer greater than 10 data, in order to guarantee to detect to tie greatly
Section, needs to carry out the sample of different size of tubercle the balance of data, and concrete operations are 2.5mm < d < 10mm tubercle marks
Information retains 1 part, and 10mm<d<20mm tubercle information retains 3 parts, and the tubercle information of d>20mm retains 7 parts, then will expand again
Tubercle markup information after filling stores.
Original image is finally cut into some small cubic blocks at random as the input of network, and by the location information of cubic block
Network is passed to together to be trained.Parameter sharing is set as to layer each in neural network model, is gradually adjusted using backpropagation
Whole each layer parameter.
(4) testing model effect
Learning rate: initial learning rate is 0.01, is 0.001 after the half of total frequency of training；
Optimizer: stochastic gradient descent；
Training batch: 64；
Criticize standardization: true.
(5) evaluation index
FROC (Free-response Receiver Operating Characteristic curve): free response
Receiver operating characteristic curve is a kind of tool, for characterizing the performance of free response system under all decision thresholds simultaneously；
FROC curve is the substitute of ROC curve.Represent the average false positive number (FP) scanned every time in x-axis, rather than false positive
Rate (FP/N has N number of negative number).The figure is substantially less than the uneven test problems of negative number N very for positive number P
It is useful.Uneven test problems will lead to all significant information and be tightly attached to the left side of ROC curve figure, and be not easy to explain.
Therefore FROC curve is incorporated herein as evaluation index.
Following specific examples of enumerating examine the pulmonary nodule based on 3D dual path neural network of invention preferred embodiment
Survey method is described further.
On LUNA16 data set utilize class U-Net structure 3D network frame, and on this basis introduce dual path with
It constructs a 3D dual path neural network to be trained and test, and has carried out 10 times of cross validations.In training, by with
Machine overturning exchange rotation adjusts image and enhances data set using the cutting ratio between 0.75 to 1.25.Evaluation index
The average recall rate that FROC is the false positive number that averagely scans every time when being 0.125,0.25,0.5,1,2,4,8, this is LUNA16
Official's evaluation index of data set.0.1 is set as in the IoU threshold value of the non-maxima suppression of test phase.
It is the 3D FROC curve graph using 3DDPN26 network training 150 times such as Fig. 4；It such as Fig. 5, is instructed using 3D DPN
Practice 100 corresponding true positive rate TPR (true positive rate) and true negative rate TNR (True Negative
Rate)。
In the present embodiment, it is also proved by visualization Lung neoplasm testing result proposed in this paper based on class U-net type
The practicability of the 3D dual path deep neural network model of network frame, selects Lung neoplasm at random from test set subset9, and
Their mark is visualized with box in Fig. 6 a to Fig. 6 d, the Lung neoplasm detected is also marked with box；Since CT is
Three-dimensional voxel data, therefore center slice can only be drawn and visualized；It, can from the visualization of the center slice of Fig. 6 a to Fig. 6 d
To observe that the nodule position detected and the position for marking tubercle are almost the same (box of two marks essentially coincides).Cause
This, the 3D dual path deep neural network model based on class U-net type network frame of the preferred embodiment of the present invention can be fine
Detect the tubercle in LUNA16 data set in ground.
The preferred embodiment of the present invention designs the algorithm of a deep learning for detecting Lung neoplasm, reaches the algorithm
The purpose of Lung neoplasm diagnosis is carried out to auxiliary doctor.During usually diagnosing, doctor compares the detection of larger Lung neoplasm
It is easy, relatively difficult compared with lesser tubercle, doctor can only generally have found some rules for having causal relation or statistics aspect, if
It is just extremely difficult to the progress potential factor excavation of CT image and independent of the knowledge of machine learning.Pass through computer aided manufacturing
Assistant's section, can be improved the speed and precision of detection, reduce the probability of Lung neoplasm failed to pinpoint a disease in diagnosis with mistaken diagnosis, improve diagnosis efficiency and standard
True rate, the patient for allowing needs to obtain medical treatment obtain medical treatment in time, to prevent missing the treatment best opportunity.The preferred embodiment of the present invention with
Deep learning network is core, using CT images data (such as DICOM format) training pattern of patient, in independent test number
The position of the pulmonary nodule in CT images is found out according to concentration and provides the probability for being determined as Lung neoplasm；The network model passes through training
The feature of tubercle is obtained, and the tubercle in given image can be found out, to realize medical knowledge and artificial intelligence technology
In conjunction with automatic identification simultaneously marks suspicious tubercle.
The above content is a further detailed description of the present invention in conjunction with specific preferred embodiments, and it cannot be said that
Specific implementation of the invention is only limited to these instructions.For those skilled in the art to which the present invention belongs, it is not taking off
Under the premise of from present inventive concept, several equivalent substitute or obvious modifications can also be made, and performance or use is identical, all answered
When being considered as belonging to protection scope of the present invention.
Claims (10)
1. a kind of 3D dual path neural network, which is characterized in that the general frame of the 3D dual path neural network is class U-net
Structure, and the connection in the 3D dual path neural network is dual path connection structure.
2. 3D dual path neural network according to claim 1, which is characterized in that dual path connection structure therein is logical
It crosses in 3D convolutional neural networks in conjunction with residual error study and intensive connect and formed.
3. 3D dual path neural network according to claim 2, which is characterized in that dual path connection structure therein is specific
Statement are as follows:
Y=G (x [: d], F (x) [: d], F (x) [d :]+x [d :])
Wherein, y is the Feature Mapping of dual path connection structure, and G is ReLU activation primitive, and F is convolution layer functions, and x is dual path
The input of connection structure, and F (x) [d :] learns for residual error, F (x) [: d] for intensively connecting, d is determined using new feature
Quantity hyper parameter.
4. a kind of pulmonary nodule detection method based on 3D dual path neural network, which comprises the following steps:
S1: the building described in any item 3D dual path neural networks of claims 1 to 3；
S2: the medical image in pretreated training set is inputted into the 3D dual path neural network, to the 3D
Dual path neural network is trained, until each index achieves the desired results；
S3: medical image to be processed is input to the 3D dual path neural network, obtains testing result.
5. pulmonary nodule detection method according to claim 4, which is characterized in that the training set in step S2 includes
The medical image of chest and corresponding diagnostic result lesion markup information, wherein diagnostic result lesion markup information includes non-
Tubercle lesion, tubercle diameter are less than 3mm, tubercle diameter is greater than or equal to 3mm three classes.
6. pulmonary nodule detection method according to claim 4, which is characterized in that the pretreatment in step S2 is specifically wrapped
It includes: by world coordinate transformation being voxel coordinate with unified spacing to the medical image in training set, extract complete lung
The bianry image in portion region is to obtain one for minimum cube that entire lung areas is all surrounded and obtain the frontier district of lung
Domain；Then reduce influence of the Pixel Information on boundary to nodule detection, and remove detection and operation of the extraneous areas to tubercle
The influence of speed, finally handles markup information.
7. pulmonary nodule detection method according to claim 6, which is characterized in that the frontier district for obtaining lung areas
Domain includes that the left-right parts of lung are carried out expansion process respectively, obtains entire lung areas and all carries out the figure after expansion process
Then itself and lung's exposure mask are carried out xor operation and obtain the borderline region of lung by picture；Further, expansion process includes: pair
Each slice of medical image first judges whether there is lung areas, if it is present first carrying out convex closure operation: if
The pixel that pixel value is 1 after convex closure operates is greater than original pixels and counts 1.5 times of purpose, will remove convex closure operating result,
Retain original image, the binary map after otherwise retaining convex closure operation；If there is no lung areas in slice, do nothing.
8. pulmonary nodule detection method according to claim 7, which is characterized in that the Pixel Information pair for reducing boundary
The influence of nodule detection includes narrowing down to the range processing of the pixel value in medical image between 0 to 255, be will not belong to
The pixel value in the place of lung's masked areas after expansion process is set as 170；Further, the removal extraneous areas is to knot
The detection of section and the influence of arithmetic speed include: pre- to carry out to medical image progress bilinearity difference up-sampling operation
Determine the expansion of range, the sub-cube block of predefined size is then intercepted from the image after expansion.
9. pulmonary nodule detection method according to claim 6, which is characterized in that the processing markup information includes: head
The conversion for first carrying out coordinate system, turns the markup information of tubercle according to initial coordinate of the nodule boundary after expansion to tubercle
It changes, retains the diameter information of tubercle and the size of original markup information.
10. pulmonary nodule detection method according to claim 4, which is characterized in that by medical image number in step S2
Also medical image is balanced according to before inputting the 3D dual path neural network, further, equilibrium step includes:
Markup information of the tubercle diameter d within the scope of 2.5mm < d < 10mm is retained a parts, the tubercle information within the scope of 10mm < d < 20mm
Retaining b parts, the tubercle information within the scope of d > 20mm retains c parts, then the tubercle markup information after expanding again stored,
Wherein a, b, c are positive integer, and a < b≤c；Further, a=1, b=2,3,4 or 5, c=5,6,7,8 or 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910551583.XA CN110458801A (en) | 2019-06-24 | 2019-06-24 | A kind of 3D dual path neural network and the pulmonary nodule detection method based on the network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910551583.XA CN110458801A (en) | 2019-06-24 | 2019-06-24 | A kind of 3D dual path neural network and the pulmonary nodule detection method based on the network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110458801A true CN110458801A (en) | 2019-11-15 |
Family
ID=68480796
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910551583.XA Pending CN110458801A (en) | 2019-06-24 | 2019-06-24 | A kind of 3D dual path neural network and the pulmonary nodule detection method based on the network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110458801A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110992285A (en) * | 2019-11-30 | 2020-04-10 | 温州大学 | Image defogging method based on hierarchical neural network |
| CN111127484A (en) * | 2019-12-25 | 2020-05-08 | 北京小白世纪网络科技有限公司 | Liver CT image segmentation method and device based on deep learning neural network |
| CN111311589A (en) * | 2020-03-05 | 2020-06-19 | 上海市肺科医院（上海市职业病防治院） | Pulmonary nodule detection and property judgment system and method |
| CN113706541A (en) * | 2020-05-20 | 2021-11-26 | 青岛海信医疗设备股份有限公司 | Image processing method and device |
| CN117054396A (en) * | 2023-10-11 | 2023-11-14 | 天津大学 | Raman spectrum detection method and device based on double-path multiplicative neural network |
| CN113706541B (en) * | 2020-05-20 | 2024-04-19 | 青岛海信医疗设备股份有限公司 | Image processing method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106940816A (en) * | 2017-03-22 | 2017-07-11 | 杭州健培科技有限公司 | Connect the CT image Lung neoplasm detecting systems of convolutional neural networks entirely based on 3D |
| CN107316294A (en) * | 2017-06-28 | 2017-11-03 | 太原理工大学 | One kind is based on improved depth Boltzmann machine Lung neoplasm feature extraction and good pernicious sorting technique |
| CN108648172A (en) * | 2018-03-30 | 2018-10-12 | 四川元匠科技有限公司 | A kind of CT figure Lung neoplasm detecting systems based on 3D-Unet |
-
2019
- 2019-06-24 CN CN201910551583.XA patent/CN110458801A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106940816A (en) * | 2017-03-22 | 2017-07-11 | 杭州健培科技有限公司 | Connect the CT image Lung neoplasm detecting systems of convolutional neural networks entirely based on 3D |
| CN107316294A (en) * | 2017-06-28 | 2017-11-03 | 太原理工大学 | One kind is based on improved depth Boltzmann machine Lung neoplasm feature extraction and good pernicious sorting technique |
| CN108648172A (en) * | 2018-03-30 | 2018-10-12 | 四川元匠科技有限公司 | A kind of CT figure Lung neoplasm detecting systems based on 3D-Unet |
Non-Patent Citations (2)
| Title |
|---|
| WENTAO ZHU等: "DeepLung: Deep 3D Dual Path Nets for Automated Pulmonary Nodule Detection and Classification", 《WACV》 * |
| 刘兵著: "《基于流形正则化的核方法研究与应用》", 30 June 2016 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110992285A (en) * | 2019-11-30 | 2020-04-10 | 温州大学 | Image defogging method based on hierarchical neural network |
| CN110992285B (en) * | 2019-11-30 | 2022-05-20 | 温州大学 | Image defogging method based on hierarchical neural network |
| CN111127484A (en) * | 2019-12-25 | 2020-05-08 | 北京小白世纪网络科技有限公司 | Liver CT image segmentation method and device based on deep learning neural network |
| CN111311589A (en) * | 2020-03-05 | 2020-06-19 | 上海市肺科医院（上海市职业病防治院） | Pulmonary nodule detection and property judgment system and method |
| CN113706541A (en) * | 2020-05-20 | 2021-11-26 | 青岛海信医疗设备股份有限公司 | Image processing method and device |
| CN113706541B (en) * | 2020-05-20 | 2024-04-19 | 青岛海信医疗设备股份有限公司 | Image processing method and device |
| CN117054396A (en) * | 2023-10-11 | 2023-11-14 | 天津大学 | Raman spectrum detection method and device based on double-path multiplicative neural network |
| CN117054396B (en) * | 2023-10-11 | 2024-01-05 | 天津大学 | Raman spectrum detection method and device based on double-path multiplicative neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110458801A (en) | A kind of 3D dual path neural network and the pulmonary nodule detection method based on the network | |
| Xu et al. | MSCS-DeepLN: Evaluating lung nodule malignancy using multi-scale cost-sensitive neural networks | |
| CN108648172B (en) | CT (computed tomography) map pulmonary nodule detection system based on 3D-Unet | |
| Kooi et al. | Large scale deep learning for computer aided detection of mammographic lesions | |
| Van Ginneken et al. | Comparing and combining algorithms for computer-aided detection of pulmonary nodules in computed tomography scans: the ANODE09 study | |
| CN109003299A (en) | A method of the calculating cerebral hemorrhage amount based on deep learning | |
| Li et al. | Lung nodule detection with deep learning in 3D thoracic MR images | |
| Vidal et al. | A U-Net Ensemble for breast lesion segmentation in DCE MRI | |
| Li et al. | Explainable multi-instance and multi-task learning for COVID-19 diagnosis and lesion segmentation in CT images | |
| CN108549912A (en) | A kind of medical image pulmonary nodule detection method based on machine learning | |
| Lu et al. | DeepAAA: clinically applicable and generalizable detection of abdominal aortic aneurysm using deep learning | |
| Suzuki et al. | CT colonography: Advanced computer‐aided detection scheme utilizing MTANNs for detection of “missed” polyps in a multicenter clinical trial | |
| CN107767362A (en) | A kind of early screening of lung cancer device based on deep learning | |
| Wang et al. | Anomaly segmentation in retinal images with poisson-blending data augmentation | |
| Harvey et al. | Deep learning in breast cancer screening | |
| CN101968851B (en) | Medical image processing method based on dictionary studying upsampling | |
| Nijiati et al. | Artificial intelligence assisting the early detection of active pulmonary tuberculosis from chest X-rays: A population-based study | |
| Li et al. | Ct scan synthesis for promoting computer-aided diagnosis capacity of covid-19 | |
| CN112967254A (en) | Lung disease identification and detection method based on chest CT image | |
| Yang et al. | 3D multi‐view squeeze‐and‐excitation convolutional neural network for lung nodule classification | |
| CN116883660A (en) | Honeycomb lung segmentation method based on class imbalance and multi-stage downsampling feature fusion | |
| Xu et al. | Improved cascade R-CNN for medical images of pulmonary nodules detection combining dilated HRNet | |
| Shi et al. | Darnet: Dual-attention residual network for automatic diagnosis of covid-19 via ct images | |
| Varshney et al. | A comparative study of deep learning models for detecting pulmonary embolism | |
| Xue et al. | Region-of-interest aware 3D ResNet for classification of COVID-19 chest computerised tomography scans |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191115 |