CN116309368A - Lung cancer pathological diagnosis system based on deep migration learning - Google Patents
Lung cancer pathological diagnosis system based on deep migration learning Download PDFInfo
- Publication number
- CN116309368A CN116309368A CN202310147859.4A CN202310147859A CN116309368A CN 116309368 A CN116309368 A CN 116309368A CN 202310147859 A CN202310147859 A CN 202310147859A CN 116309368 A CN116309368 A CN 116309368A
- Authority
- CN
- China
- Prior art keywords
- lung cancer
- lung
- training
- model
- cancer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 206010058467 Lung neoplasm malignant Diseases 0.000 title claims abstract description 180
- 201000005202 lung cancer Diseases 0.000 title claims abstract description 180
- 208000020816 lung neoplasm Diseases 0.000 title claims abstract description 180
- 238000010827 pathological analysis Methods 0.000 title claims abstract description 66
- 238000013508 migration Methods 0.000 title claims abstract description 38
- 230000005012 migration Effects 0.000 title claims abstract description 38
- 238000012549 training Methods 0.000 claims abstract description 160
- 210000004072 lung Anatomy 0.000 claims abstract description 129
- 201000011510 cancer Diseases 0.000 claims abstract description 108
- 206010028980 Neoplasm Diseases 0.000 claims abstract description 107
- 238000003745 diagnosis Methods 0.000 claims abstract description 90
- 238000013526 transfer learning Methods 0.000 claims abstract description 87
- 238000011156 evaluation Methods 0.000 claims abstract description 75
- 230000001575 pathological effect Effects 0.000 claims abstract description 65
- 238000000034 method Methods 0.000 claims abstract description 54
- 238000002372 labelling Methods 0.000 claims abstract description 45
- 230000011218 segmentation Effects 0.000 claims abstract description 37
- 238000001914 filtration Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 23
- 238000007781 pre-processing Methods 0.000 claims abstract description 20
- 238000006243 chemical reaction Methods 0.000 claims abstract description 14
- 238000012360 testing method Methods 0.000 claims description 66
- 238000005457 optimization Methods 0.000 claims description 18
- 238000013136 deep learning model Methods 0.000 claims description 15
- 238000002405 diagnostic procedure Methods 0.000 claims description 12
- 230000007170 pathology Effects 0.000 claims description 11
- 208000005718 Stomach Neoplasms Diseases 0.000 claims description 8
- 206010017758 gastric cancer Diseases 0.000 claims description 8
- 238000012216 screening Methods 0.000 claims description 8
- 201000011549 stomach cancer Diseases 0.000 claims description 8
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 5
- 238000004458 analytical method Methods 0.000 claims description 4
- 238000004043 dyeing Methods 0.000 claims description 4
- 238000012854 evaluation process Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 4
- 201000005249 lung adenocarcinoma Diseases 0.000 claims description 4
- 208000000649 small cell carcinoma Diseases 0.000 claims description 4
- 206010041823 squamous cell carcinoma Diseases 0.000 claims description 4
- 210000000115 thoracic cavity Anatomy 0.000 claims description 4
- 230000000007 visual effect Effects 0.000 claims description 4
- 210000001519 tissue Anatomy 0.000 description 83
- 230000000694 effects Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 11
- 238000013528 artificial neural network Methods 0.000 description 9
- 238000013135 deep learning Methods 0.000 description 4
- 230000035945 sensitivity Effects 0.000 description 4
- 238000013509 system migration Methods 0.000 description 4
- 230000007812 deficiency Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 210000000056 organ Anatomy 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 210000001165 lymph node Anatomy 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000036210 malignancy Effects 0.000 description 1
- 210000005075 mammary gland Anatomy 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 210000002307 prostate Anatomy 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000010186 staining Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/776—Validation; Performance evaluation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30061—Lung
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention provides a lung cancer pathological diagnosis system based on deep migration learning, which comprises the following components: the digital processing cancer area labeling module is used for collecting the existing lung tissue slices at a selected time interval, forming lung tissue digital pathological sections through digital conversion processing, and carrying out digital set division and cancer area pixel level labeling; the pretreatment filtering segmentation recognition module is used for preprocessing digital pathological sections of lung tissues, filtering a section background area through an algorithm, and establishing a lung cancer area pixel-level recognition model through two-class semantic segmentation; the transfer learning secondary tuning module is used for training the lung cancer area pixel level identification model, and performing secondary training tuning on lung cancer area pixel level identification model parameters by using lung cancer training data through a transfer learning method; and the lung cancer pathological diagnosis prediction module is used for performing performance evaluation on the transfer learning lung cancer auxiliary diagnosis model through a plurality of evaluation indexes and outputting lung cancer pathological diagnosis prediction results.
Description
Technical Field
The invention relates to the technical field of intelligent recognition and prediction of digital pathology, in particular to a lung cancer pathology diagnosis system based on deep migration learning.
Background
Currently, lung cancer is a malignancy with very high global morbidity and mortality; the clear lung cancer pathological type is the precondition of accurate treatment, and the responsibility is important; however, as the number of patients increases, the workload increases; high intensity work leads to fatigue, with a risk of diagnosis; technological advances in digital pathology and artificial intelligence make it possible for lung cancer pathology auxiliary diagnosis system; a plurality of medical diagnosis systems based on deep learning are constructed, for natural image analysis, the deep learning plays an important role in target detection and image segmentation, the defect of poor popularization capability of an expert system can be avoided, and the complexity of artificial feature engineering of the traditional machine learning method is made up;
in the pathological field, deep learning has achieved a certain result in the aspects of artificial intelligent diagnosis of organs such as mammary gland, lymph node, lung, prostate and the like; however, training models from scratch is a time-consuming and labor-consuming process, which is increasingly prominent in the pathology field; the file volume of a 400-fold digitized scanning pathological image is usually more than 1GB, and the file volume is more than 100 hundred million pixels, so that a great challenge is provided for labeling, and the method has important clinical significance for exploring the effectiveness of migration learning among different organ pathological models; transfer learning is a very efficient artificial intelligence model building method aimed at transferring knowledge learned in one domain to another domain or domains; in order to effectively utilize experience knowledge in the modeled field, the migration learning generally defines a model in the current field as an initial model, namely, considers that initial parameters of the model are the same as those of the current model, and then trains the model by using labeling data in the other field, so that a migration effect is obtained; the data between the two fields are required to have a certain similarity in the transfer learning, so that the initial parameters can generate beneficial contribution in the training process; therefore, there is a need to propose a lung cancer pathological diagnosis system based on deep migration learning to at least partially solve the problems existing in the prior art.
Disclosure of Invention
A series of concepts in simplified form are introduced in the summary section, which will be described in further detail in the detailed description section; the summary of the invention is not intended to define the key features and essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
To at least partially solve the above problems, the present invention provides a lung cancer pathological diagnosis system based on deep migration learning, including:
the digital processing cancer area labeling module is used for collecting the existing lung tissue slices at a selected time interval, forming lung tissue digital pathological sections through digital conversion processing, and carrying out digital set division and cancer area pixel level labeling;
the pretreatment filtering segmentation recognition module is used for preprocessing digital pathological sections of lung tissues, filtering a section background area through an algorithm, and establishing a lung cancer area pixel-level recognition model through two-class semantic segmentation;
the transfer learning secondary optimization module is used for training the lung cancer area pixel level identification model, performing secondary training optimization on lung cancer area pixel level identification model parameters by using lung cancer training data through a transfer learning method, and acquiring a transfer learning lung cancer auxiliary diagnosis model;
And the lung cancer pathological diagnosis prediction module is used for performing prediction evaluation on the transfer learning lung cancer auxiliary diagnosis model through an external test set and performing performance evaluation through a plurality of evaluation indexes, extracting and inputting all lung tissue image block data, and outputting a lung cancer pathological diagnosis prediction result.
Preferably, the digital processing cancer area labeling module comprises:
the section data covering and screening submodule is used for butt-jointing and collecting section data of a thoracic hospital and screening lung tissue pathological sections covering various lung tissue forms; the pathological section of the lung tissue comprises: normal lung histopathological section, adenocarcinoma lung histopathological section, squamous cell carcinoma lung histopathological section and small cell carcinoma lung histopathological section;
the pathological section digital processing submodule is used for carrying out digital conversion processing on the pathological section of the lung tissue to obtain the digital pathological section of the lung tissue; the digital conversion process includes: digital scanning, digital recording or digital recognition;
and the collection dividing cancer area labeling submodule is used for carrying out digital collection dividing and cancer area pixel-level labeling on digital pathological sections of lung tissues.
Preferably, the preprocessing filtering segmentation recognition module includes:
The sub-module for filtering the background area of the slice is used for preprocessing the digital pathological section of the lung tissue to filter the background area, and the Otsu algorithm is used for filtering the background area of the slice;
the training image block segmentation submodule segments the whole slice into training image blocks with the size of 320 multiplied by 320 according to the 200x visual field by a set step length, and the training image blocks comprise: 215876 blocks containing positive cancer area and 955416 blocks not containing negative cancer area;
and the lung cancer area identification model submodule is used for carrying out pixel level identification on a cancer area and a non-cancer area through a DeepLab v3 classification semantic segmentation model based on ResNet-50, so as to establish a lung cancer area pixel level identification model.
Preferably, the transfer learning secondary optimization module includes:
the initial parameter recognition training sub-module is used for training a lung cancer area pixel level recognition model; introducing a stomach cancer area identification model, and taking parameters of the stomach cancer area identification model as initial values to perform early initial training;
the transfer learning secondary training sub-module is used for performing secondary training optimization on model parameters by using lung cancer training data through a transfer learning method;
training equipment process parameter submodule, setting deep learning model training equipment and training process parameters; performing data enhancement of a lung tissue pathological diagnosis process; and obtaining an auxiliary diagnosis model of the lung cancer through transfer learning.
Preferably, the lung cancer pathological diagnosis prediction module comprises:
the large-scale external test data acquisition sub-module is used for acquiring external test lung tissue slice data of a large-scale medical database as an external test set;
the prediction diagnosis index evaluation sub-module is used for performing prediction evaluation on the transfer learning lung cancer auxiliary diagnosis model and performing performance evaluation through a plurality of evaluation indexes;
and the transfer learning lung cancer diagnosis sub-module is used for extracting all effective lung tissue image blocks of the external test lung tissue slice data, inputting all the effective lung tissue image block data into the transfer learning lung cancer auxiliary diagnosis model, obtaining output data of the transfer learning lung cancer auxiliary diagnosis model and obtaining a lung cancer pathological diagnosis prediction result.
Preferably, the collection dividing cancer area labeling submodule includes:
the digital pathological section set dividing unit is used for dividing digital sets of digital pathological sections of lung tissues and dividing a training set and an internal test set;
the set division mode selection unit, the training set and the internal test set division mode include: dividing the training set into 316 training sets and 203 internal test sets;
and the cancer area pixel labeling unit is used for labeling the cancer area of the digital pathological section of the lung tissue with canceration in the training set at a pixel level through a labeling tool.
Preferably, the lung cancer area identification model submodule includes:
the second-class semantic segmentation model unit is used for establishing a deep Lab v3 second-class semantic segmentation model based on ResNet-50;
the pixel-level model identification unit is used for carrying out pixel-level identification on the cancer area and the non-cancer area through the two-classification semantic segmentation model;
and the cancer area pixel level identification model unit establishes a lung cancer area pixel level identification model through the pixel level identification of the cancer area and the non-cancer area, and performs the pixel level identification of the cancer area and the non-cancer area.
Preferably, the training device process parameter submodule includes:
the training equipment process setting unit is used for setting deep learning model training equipment and training process parameters; the deep learning model training apparatus includes: a multiple GPU server and ADAM optimizer; the training process parameters include: training learning rate, batch size and number of rounds of training;
a diagnostic process data enhancement unit for performing data enhancement of a lung tissue pathological diagnostic process by random rotation and mirroring; data enhancement of lung histopathological diagnostic procedures by random rotation and mirroring includes: applying a random scale of 1.0x to 1.5x to the training data to make the model tolerant to small variations in the scaling ratio of the various digital scanning devices; and applying random Gaussian and dynamic blurring to the training image, and carrying out random disturbance on a brightness interval, a contrast interval, a tone interval and a saturation interval of the image to enhance the compatibility of the model to different dyeing configurations.
Preferably, the prediction diagnosis index evaluation submodule includes:
the external test set prediction evaluation unit is used for performing prediction evaluation on the lung cancer auxiliary diagnosis model through the external test set;
the evaluation index performance evaluation unit is used for performing performance evaluation on the migration learning lung cancer auxiliary diagnosis model through a plurality of evaluation indexes;
an evaluation index setting operation unit that performs a plurality of evaluation index operations in an evaluation process; the plurality of evaluation metrics includes: TP, FP, TN, and FN represent true positive, false positive, true negative, and false negative, respectively.
Preferably, the lung cancer diagnosis submodule for transfer learning includes:
the lung tissue image block extraction unit is used for preprocessing the external test lung tissue slice data and extracting all valid lung tissue image blocks of the external test lung tissue slice data;
the lung cancer diagnosis model prediction unit inputs all effective lung tissue image blocks into a transfer learning lung cancer auxiliary diagnosis model, and the transfer learning lung cancer auxiliary diagnosis model outputs a pixel level prediction result of each image block to obtain transfer learning lung cancer auxiliary diagnosis model output data;
the lung cancer diagnosis probability selection unit is used for carrying out image block analysis to obtain a lung tissue slice pixel level prediction probability value; taking the average value of the top 100 probability values in the pixel level prediction probability values of each slice as the probability of positive prediction of the whole slice; and obtaining a lung cancer pathological diagnosis prediction result.
Compared with the prior art, the invention at least comprises the following beneficial effects:
the invention provides a lung cancer pathological diagnosis system based on deep migration learning, which is characterized in that a cancer region labeling module is used for digitally processing, existing lung tissue slices in a selected period are collected and are subjected to digital conversion processing to form lung tissue digital pathological slices, and digital set division and cancer region pixel level labeling are performed; the pretreatment filtering segmentation recognition module is used for preprocessing digital pathological sections of lung tissues, filtering a section background area through an algorithm, and establishing a lung cancer area pixel-level recognition model through two-class semantic segmentation; the transfer learning secondary optimization module is used for training the lung cancer area pixel level identification model, performing secondary training optimization on lung cancer area pixel level identification model parameters by using lung cancer training data through a transfer learning method, and acquiring a transfer learning lung cancer auxiliary diagnosis model; the lung cancer pathological diagnosis prediction module predicts and evaluates the migration learning lung cancer auxiliary diagnosis model through an external test set and performance evaluation through a plurality of evaluation indexes, extracts and inputs all lung tissue image block data, and outputs a lung cancer pathological diagnosis prediction result; under the condition of less sample size, the model of the invention shows better recognition accuracy than the common model; in addition, for the external test set, the diagnosis AUC of the migration learning model established in the study is 0.968, kappa=0.828, which indicates that the model has good popularization; the artificial intelligent lung cancer pathology auxiliary diagnosis system established by the invention has better accuracy and external popularization; the training period of the diagnostic model is shortened, and the accuracy of the diagnostic model is improved; the lung cancer pathological diagnosis system assists a pathologist in marking a pathological area and making preliminary diagnosis, improves the working efficiency of a pathology department, solves the problem of serious strength deficiency of the pathological diagnosis technology in China, and reduces missed diagnosis and misdiagnosis caused by subjective reasons such as fatigue, experience deficiency and the like of the pathologist; along with the increase of the training data quantity, the deep migration learning model is continuously optimized, and the accuracy of the lung cancer auxiliary diagnosis model can be continuously improved.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention. In the drawings:
fig. 1 is a framework diagram of a lung cancer pathological diagnosis system based on deep migration learning.
Fig. 2 is a diagram of a deep learning model training and testing embodiment of a lung cancer pathological diagnosis system based on deep transfer learning according to the present invention.
FIG. 3 is a diagram of comparing a lung cancer pathological diagnosis system migration learning model with a model prediction result from zero training according to an embodiment of the present invention.
Fig. 4 is a diagram of a lung cancer pathological diagnosis system transfer learning model based on deep transfer learning and a comparison between a prediction result from a zero training model according to another embodiment of the present invention.
Fig. 5 is a difference chart of a lung cancer pathological diagnosis system film-making staining-induced migration learning model and a predicted result from a zero training model based on deep migration learning.
Fig. 6 is a graph of the effect of the deep learning model of the lung cancer pathological diagnosis system on identifying the small-focus cancer based on deep migration learning.
Fig. 7 is a graph of false positives predicted by the lung cancer pathological diagnosis system based on deep migration learning.
Fig. 8 is a graph of false negative prediction of a lung cancer pathological diagnosis system based on deep transfer learning.
Fig. 9 is a partial prediction result diagram of a lung cancer pathological diagnosis system migration learning model based on deep migration learning in the present invention on an internal test set a and an external test set b.
Fig. 10 is a graph comparing a lung cancer pathological diagnosis system migration learning model based on deep migration learning with curves from an internal test set and an external test set of a zero training model.
Detailed Description
The present invention is described in further detail below with reference to the drawings and examples to enable those skilled in the art to practice the same and to refer to the description; as shown in fig. 1-10 and tables 1-3, the present invention provides a lung cancer pathological diagnosis system based on deep migration learning, comprising:
the digital processing cancer area labeling module is used for collecting the existing lung tissue slices at a selected time interval, forming lung tissue digital pathological sections through digital conversion processing, and carrying out digital set division and cancer area pixel level labeling;
The pretreatment filtering segmentation recognition module is used for preprocessing digital pathological sections of lung tissues, filtering a section background area through an algorithm, and establishing a lung cancer area pixel-level recognition model through two-class semantic segmentation;
the transfer learning secondary optimization module is used for training the lung cancer area pixel level identification model, performing secondary training optimization on lung cancer area pixel level identification model parameters by using lung cancer training data through a transfer learning method, and acquiring a transfer learning lung cancer auxiliary diagnosis model;
and the lung cancer pathological diagnosis prediction module is used for performing prediction evaluation on the transfer learning lung cancer auxiliary diagnosis model through an external test set and performing performance evaluation through a plurality of evaluation indexes, extracting and inputting all lung tissue image block data, and outputting a lung cancer pathological diagnosis prediction result.
The principle and effect of the technical scheme are as follows: the invention provides a lung cancer pathological diagnosis system based on deep migration learning, which comprises the following components: the digital processing cancer area labeling module is used for collecting the existing lung tissue slices at a selected time interval, forming lung tissue digital pathological sections through digital conversion processing, and carrying out digital set division and cancer area pixel level labeling; the pretreatment filtering segmentation recognition module is used for preprocessing digital pathological sections of lung tissues, filtering a section background area through an algorithm, and establishing a lung cancer area pixel-level recognition model through two-class semantic segmentation; the transfer learning secondary optimization module is used for training the lung cancer area pixel level identification model, performing secondary training optimization on lung cancer area pixel level identification model parameters by using lung cancer training data through a transfer learning method, and acquiring a transfer learning lung cancer auxiliary diagnosis model; the lung cancer pathological diagnosis prediction module predicts and evaluates the migration learning lung cancer auxiliary diagnosis model through an external test set and performance evaluation through a plurality of evaluation indexes, extracts and inputs all lung tissue image block data, and outputs a lung cancer pathological diagnosis prediction result; auc=0.988 vs. prior art auc=0.971; inventive kappa=0.852 vs prior art kappa=0.832; the diagnostic AUC of the migration learning model established by the invention is 0.968, and kappa=0.828; under the condition of less sample size, the model of the invention shows better recognition accuracy than the common model; in addition, for the external test set, the diagnosis AUC of the migration learning model established in the study is 0.968, kappa=0.828, which indicates that the model has good popularization; the artificial intelligent lung cancer pathology auxiliary diagnosis system established by the invention has better accuracy and external popularization; the training period of the diagnostic model is shortened, and the accuracy of the diagnostic model is improved; the lung cancer pathological diagnosis system can assist a pathologist to mark a pathological area and make preliminary diagnosis, improves the working efficiency of a pathology department, solves the problem of serious shortage of the strength of the pathological diagnosis technology in China, and reduces missed diagnosis and misdiagnosis caused by subjective reasons such as fatigue, experience shortage and the like of the pathologist; along with the increase of the training data quantity, the deep migration learning model is continuously optimized, and the accuracy of the lung cancer auxiliary diagnosis model can be continuously improved.
In one embodiment, the digitally processed cancer area labeling module comprises:
the section data covering and screening submodule is used for butt-jointing and collecting section data of a thoracic hospital and screening lung tissue pathological sections covering various lung tissue forms; the pathological section of the lung tissue comprises: normal lung histopathological section, adenocarcinoma lung histopathological section, squamous cell carcinoma lung histopathological section and small cell carcinoma lung histopathological section;
the pathological section digital processing submodule is used for carrying out digital conversion processing on the pathological section of the lung tissue to obtain the digital pathological section of the lung tissue; the digital conversion process includes: digital scanning, digital recording or digital recognition;
and the collection dividing cancer area labeling submodule is used for carrying out digital collection dividing and cancer area pixel-level labeling on digital pathological sections of lung tissues.
The principle and effect of the technical scheme are as follows: the digital processing cancer area labeling module comprises: the section data covering and screening submodule is used for butt-jointing and collecting section data of a thoracic hospital and screening lung tissue pathological sections covering various lung tissue forms; the pathological section of the lung tissue comprises: normal lung histopathological section, adenocarcinoma lung histopathological section, squamous cell carcinoma lung histopathological section and small cell carcinoma lung histopathological section; the pathological section digital processing submodule is used for carrying out digital conversion processing on the pathological section of the lung tissue to obtain the digital pathological section of the lung tissue; the digital conversion process includes: digital scanning, digital recording or digital recognition; the collection dividing cancer area labeling submodule carries out digital collection dividing and cancer area pixel level labeling on digital pathological sections of lung tissues; the sections were scanned by a 400 x digitization using a Jiang Feng KF-PRO-005 scanner; for 316 slices with canceration in the training set, a pathologist uses a marking tool based on iPad and Apple Pencil to mark the cancer area of the digital pathological slice at the pixel level; can assist pathologists to mark pathological areas, make preliminary diagnosis, improve the working efficiency of pathology departments, and solve the problem of serious shortage of pathological diagnosis technology in China.
In one embodiment, the preprocessing filter partition identification module comprises:
the sub-module for filtering the background area of the slice is used for preprocessing the digital pathological section of the lung tissue to filter the background area, and the Otsu algorithm is used for filtering the background area of the slice;
the training image block segmentation submodule segments the whole slice into training image blocks with the size of 320 multiplied by 320 according to the 200x visual field by a set step length, and the training image blocks comprise: 215876 blocks containing positive cancer area and 955416 blocks not containing negative cancer area;
and the lung cancer area identification model submodule is used for carrying out pixel level identification on a cancer area and a non-cancer area through a DeepLab v3 classification semantic segmentation model based on ResNet-50, so as to establish a lung cancer area pixel level identification model.
The principle and effect of the technical scheme are as follows: the preprocessing filtering segmentation recognition module comprises: the sub-module for filtering the background area of the slice is used for preprocessing the digital pathological section of the lung tissue to filter the background area, and the Otsu algorithm is used for filtering the background area of the slice; the training image block segmentation submodule segments the whole slice into training image blocks with the size of 320 multiplied by 320 according to the 200x visual field by a set step length, and the training image blocks comprise: 215876 blocks containing positive cancer area and 955416 blocks not containing negative cancer area; the lung cancer area identification model submodule is used for carrying out pixel level identification on a cancer area and a non-cancer area through a DeepLab v3 classification semantic segmentation model based on ResNet-50, and establishing a lung cancer area pixel level identification model; in the preprocessing process of the training set and the verification set, firstly, filtering out the background area of the slice by using an Otsu algorithm, then dividing the whole slice into training image blocks with the size of 320 multiplied by 320 according to the 200 multiplied field of view by half the size of the training image blocks, wherein 1215876 positive cancer-containing image blocks and 955416 negative cancer-free image blocks are used in training of the model;
Calculating pixel-level characteristic values of a cancer area and a non-cancer area:
wherein GTQZj represents pixel level characteristic values of a cancer area and a non-cancer area, ras represents a deep migration learning parameter, initialization approaches 0 and continuously learns and updates in a deep learning process, i represents an ith position identification number on a pixel level characteristic diagram of the cancer area and the non-cancer area, j represents a jth position identification number influenced by the ith position on the pixel level characteristic diagram of the cancer area and the non-cancer area, K represents all positions on the pixel characteristic diagrams of the cancer area and the non-cancer area, ei represents an ith position initial input characteristic diagram convolution transformation first characteristic value, fj represents a jth position initial input characteristic diagram convolution transformation second characteristic value, gi represents an ith position initial input characteristic diagram convolution transformation third characteristic value, and Dj represents a jth position initial input characteristic value; exp represents an exponential expression of the natural constant e; by calculating the pixel-level characteristic values of the cancer area and the non-cancer area, similar semantic segmentation characteristics can be mutually promoted, so that related pixels far away can be identified and classified, and the compactness of identification and classification and the consistency of identification semantics are improved.
In one embodiment, the transfer learning secondary tuning module includes:
the initial parameter recognition training sub-module is used for training a lung cancer area pixel level recognition model; introducing a stomach cancer area identification model, and taking parameters of the stomach cancer area identification model as initial values to perform early initial training;
The transfer learning secondary training sub-module is used for performing secondary training optimization on model parameters by using lung cancer training data through a transfer learning method;
training equipment process parameter submodule, setting deep learning model training equipment and training process parameters; performing data enhancement of a lung tissue pathological diagnosis process; and obtaining an auxiliary diagnosis model of the lung cancer through transfer learning.
The principle and effect of the technical scheme are as follows: the transfer learning secondary optimization module comprises: the initial parameter recognition training sub-module is used for training a lung cancer area pixel level recognition model; introducing a stomach cancer area identification model, and taking parameters of the stomach cancer area identification model as initial values to perform early initial training; the transfer learning secondary training sub-module is used for performing secondary training optimization on model parameters by using lung cancer training data through a transfer learning method; training equipment process parameter submodule, setting deep learning model training equipment and training process parameters; performing data enhancement of a lung tissue pathological diagnosis process; acquiring an auxiliary diagnosis model of lung cancer through transfer learning; in the model training process, taking the parameters of the lung cancer area identification model as initial values, and performing secondary training optimization on the lung cancer area identification model parameters through a transfer learning method; the transfer learning uses lung training data to perform secondary training and tuning on model parameters, and then diagnosis can be well made; reduce missed diagnosis and misdiagnosis caused by subjective reasons such as fatigue, experience deficiency and the like of pathologists.
In one embodiment, the lung cancer pathological diagnosis prediction module comprises:
the large-scale external test data acquisition sub-module is used for acquiring external test lung tissue slice data of a large-scale medical database as an external test set;
the prediction diagnosis index evaluation sub-module is used for performing prediction evaluation on the transfer learning lung cancer auxiliary diagnosis model and performing performance evaluation through a plurality of evaluation indexes;
and the transfer learning lung cancer diagnosis sub-module is used for extracting all effective lung tissue image blocks of the external test lung tissue slice data, inputting all the effective lung tissue image block data into the transfer learning lung cancer auxiliary diagnosis model, obtaining output data of the transfer learning lung cancer auxiliary diagnosis model and obtaining a lung cancer pathological diagnosis prediction result.
The principle and effect of the technical scheme are as follows: the lung cancer pathological diagnosis prediction module comprises: the large-scale external test data acquisition sub-module is used for acquiring external test lung tissue slice data of a large-scale medical database as an external test set; the prediction diagnosis index evaluation sub-module is used for performing prediction evaluation on the transfer learning lung cancer auxiliary diagnosis model and performing performance evaluation through a plurality of evaluation indexes; the lung cancer diagnosis sub-module for transfer learning extracts all effective lung tissue image blocks of the external test lung tissue slice data, inputs all effective lung tissue image block data into the lung cancer auxiliary diagnosis model for transfer learning, obtains output data of the lung cancer auxiliary diagnosis model for transfer learning, and obtains a lung cancer pathological diagnosis prediction result; verifying the established auxiliary diagnostic model by using the internal test set and the external test set obtained from the database respectively; along with the increase of the training data quantity, the deep migration learning model is continuously optimized, and the accuracy of the lung cancer auxiliary diagnosis model can be continuously improved.
In one embodiment, the set partitioning cancer area labeling submodule includes:
the digital pathological section set dividing unit is used for dividing digital sets of digital pathological sections of lung tissues and dividing a training set and an internal test set;
the set division mode selection unit, the training set and the internal test set division mode include: dividing the training set into 316 training sets and 203 internal test sets;
and the cancer area pixel labeling unit is used for labeling the cancer area of the digital pathological section of the lung tissue with canceration in the training set at a pixel level through a labeling tool.
The principle and effect of the technical scheme are as follows: the set dividing cancer area labeling submodule comprises: the digital pathological section set dividing unit is used for dividing digital sets of digital pathological sections of lung tissues and dividing a training set and an internal test set; the set division mode selection unit, the training set and the internal test set division mode include: dividing the training set into 316 training sets and 203 internal test sets; the cancer region pixel labeling unit is used for labeling the cancer region of the digital pathological section of the lung tissue with canceration in the training set at a pixel level through a labeling tool; the model training set and the internal test set can be divided in a richer way; carrying out pixel-level labeling on a cancer area of the digital pathological section through a labeling tool; more efficient and accurate pixel-level labeling of training data can be performed.
In one embodiment, the lung cancer area identification model submodule includes:
the second-class semantic segmentation model unit is used for establishing a deep Lab v3 second-class semantic segmentation model based on ResNet-50;
the pixel-level model identification unit is used for carrying out pixel-level identification on the cancer area and the non-cancer area through the two-classification semantic segmentation model;
and the cancer area pixel level identification model unit establishes a lung cancer area pixel level identification model through the pixel level identification of the cancer area and the non-cancer area, and performs the pixel level identification of the cancer area and the non-cancer area.
The principle and effect of the technical scheme are as follows: the lung cancer area identification model submodule comprises: the second-class semantic segmentation model unit is used for establishing a deep Lab v3 second-class semantic segmentation model based on ResNet-50; the pixel-level model identification unit is used for carrying out pixel-level identification on the cancer area and the non-cancer area through the two-classification semantic segmentation model; the cancer area pixel level identification model unit establishes a lung cancer area pixel level identification model through the pixel level identification of the cancer area and the non-cancer area, and carries out the pixel level identification of the cancer area and the non-cancer area; the two-classification semantic segmentation model can be used for carrying out pixel-level identification of cancer areas and non-cancer areas, so that the identification of the cancer areas is more precise and fine.
In one embodiment, the training device process parameter submodule includes:
the training equipment process setting unit is used for setting deep learning model training equipment and training process parameters; the deep learning model training apparatus includes: a multiple GPU server and ADAM optimizer; the training process parameters include: training learning rate, batch size and number of rounds of training;
a diagnostic process data enhancement unit for performing data enhancement of a lung tissue pathological diagnostic process by random rotation and mirroring; data enhancement of lung histopathological diagnostic procedures by random rotation and mirroring includes: applying a random scale of 1.0x to 1.5x to the training data to make the model tolerant to small variations in the scaling ratio of the various digital scanning devices; and applying random Gaussian and dynamic blurring to the training image, and carrying out random disturbance on a brightness interval, a contrast interval, a tone interval and a saturation interval of the image to enhance the compatibility of the model to different dyeing configurations.
The principle and effect of the technical scheme are as follows: the training device process parameter submodule comprises: the training equipment process setting unit is used for setting deep learning model training equipment and training process parameters; the deep learning model training apparatus includes: a multiple GPU server and ADAM optimizer; the training process parameters include: training learning rate, batch size and number of rounds of training; a diagnostic process data enhancement unit for performing data enhancement of a lung tissue pathological diagnostic process by random rotation and mirroring; data enhancement of lung histopathological diagnostic procedures by random rotation and mirroring includes: applying a random scale of 1.0x to 1.5x to the training data to make the model tolerant to small variations in the scaling ratio of the various digital scanning devices; applying random Gaussian and dynamic blurring to the training image, and carrying out random disturbance on a brightness interval, a contrast interval, a tone interval and a saturation interval of the image to enhance the compatibility of the model to different dyeing configurations; the deep learning model training apparatus includes: the Ubuntu servers and optimizers of the 8 NVIDIA GTX1080Ti GPUs adopt ADAM; the training learning rate is fixed to be 0.0001, the batch size is set to be 256, and the number of stop rounds of the training process is 21000; the brightness interval of the image is 0.0-0.2, the contrast interval is 0.0-0.2, the tone interval is 0.0-0.04 and the saturation interval is 0.0-0.25; using 2123 sections, wherein 1391 sections contain malignant tumors, to obtain 6887275 cancer area image blocks and 4126011 non-cancer area image blocks; the sensitivity and the specificity of the model on 3212 daily lung sections in a hospital can reach 0.996 and 0.806 respectively; training a deep neural network by using training data of the existing lung tissue pathological diagnosis process to respectively obtain a deep neural network hidden layer weight parameter and a deep neural network bias parameter, and a deep neural network output layer weight parameter and a deep neural network bias parameter; adjusting the deep neural network model by utilizing the data enhanced in the lung tissue pathological diagnosis process, and updating the weight parameters of the deep neural network output layer and the bias parameters of the deep neural network to obtain the weight adjustment parameters of the deep neural network output layer and the bias adjustment parameters of the deep neural network; the judgment accuracy of the model can be enhanced, the accuracy of distinguishing the cancer area from the non-cancer area in the prediction can be improved, and the prediction false alarm or prediction missing report can be reduced.
In one embodiment, the predictive diagnostic index evaluation submodule includes:
the external test set prediction evaluation unit is used for performing prediction evaluation on the lung cancer auxiliary diagnosis model through the external test set;
the evaluation index performance evaluation unit is used for performing performance evaluation on the migration learning lung cancer auxiliary diagnosis model through a plurality of evaluation indexes;
an evaluation index setting operation unit that performs a plurality of evaluation index operations in an evaluation process; the plurality of evaluation metrics includes: TP, FP, TN, and FN represent true positive, false positive, true negative, and false negative, respectively.
The principle and effect of the technical scheme are as follows: the prediction diagnosis index evaluation submodule comprises: the external test set prediction evaluation unit is used for performing prediction evaluation on the lung cancer auxiliary diagnosis model through the external test set; the evaluation index performance evaluation unit is used for performing performance evaluation on the migration learning lung cancer auxiliary diagnosis model through a plurality of evaluation indexes; an evaluation index setting operation unit that performs a plurality of evaluation index operations in an evaluation process; the plurality of evaluation metrics includes: TP, FP, TN and FN represent true positive, false positive, true negative and false negative, respectively; the performance of the description model is calculated by selecting 4 evaluation indexes, and the calculation is as follows:
kappa＝(Po+Pe)/(1-Pe)
Pzl＝(TP+TN)/(TP+FN+FP+TN)
Gmn＝TP/(TP+FN)
Cti＝TN/(TN+FP)
Po＝(TP+TN)/(TP+FP+FN+TN),
Pe＝[(TP+FP)×(TP+FN)+(FP+TN)×(FN+TN)]/(TP+FP+FN+TN)2；
Wherein kappa represents the coincidence rate of the evaluation result, po represents the evaluation coincidence rate, pe represents the probability coincidence rate, pzl represents the accuracy, gmn represents the sensitivity, and Cti represents the specificity; the performance of the model is described through evaluation index calculation, so that the performance of the model is described more accurately and specifically.
In one embodiment, the transfer learning lung cancer diagnosis submodule includes:
the lung tissue image block extraction unit is used for preprocessing the external test lung tissue slice data and extracting all valid lung tissue image blocks of the external test lung tissue slice data;
the lung cancer diagnosis model prediction unit inputs all effective lung tissue image blocks into a transfer learning lung cancer auxiliary diagnosis model, and the transfer learning lung cancer auxiliary diagnosis model outputs a pixel level prediction result of each image block to obtain transfer learning lung cancer auxiliary diagnosis model output data;
the lung cancer diagnosis probability selection unit is used for carrying out image block analysis to obtain a lung tissue slice pixel level prediction probability value; taking the average value of the top 100 probability values in the pixel level prediction probability values of each slice as the probability of positive prediction of the whole slice; and obtaining a lung cancer pathological diagnosis prediction result.
The principle and effect of the technical scheme are as follows: the transfer learning lung cancer diagnosis submodule comprises: the lung tissue image block extraction unit is used for preprocessing the external test lung tissue slice data and extracting all valid lung tissue image blocks of the external test lung tissue slice data; the lung cancer diagnosis model prediction unit inputs all effective lung tissue image blocks into a transfer learning lung cancer auxiliary diagnosis model, and the transfer learning lung cancer auxiliary diagnosis model outputs a pixel level prediction result of each image block to obtain transfer learning lung cancer auxiliary diagnosis model output data; the lung cancer diagnosis probability selection unit is used for carrying out image block analysis to obtain a lung tissue slice pixel level prediction probability value; taking the average value of the top 100 probability values in the pixel level prediction probability values of each slice as the probability of positive prediction of the whole slice; obtaining a lung cancer pathological diagnosis prediction result; in addition to the internal test set, the 1081 open lung tissue slice data of the cancer image archive was used as the external test set; in the prediction process, after preprocessing an input slice, extracting image blocks of all effective tissues and inputting the image blocks into a model to obtain a pixel-level prediction result of each image block; after the image block is analyzed, taking the average value of the top 100 probability values in the pixel-level prediction probability values of each slice as the probability of positive prediction of the whole slice; in the external test set, the popularization capability of the test model on new samples is kept for 102 samples of other cancer species, and the typing is not learned in model training; from zero training and transfer learning, the model recognizes the samples with higher accuracy; training parameters from zero: sensitivity=0.843, specificity=0.917; transfer learning parameters: sensitivity=0.892, specificity=0.937; whether from zero training or transfer learning, the model recognizes samples with very high accuracy, and shows better popularization capability; greatly improves the accuracy of lung cancer pathological diagnosis and identification.
Table 1 is a distribution table of slice types of training data sets and slice types of test data sets of the lung cancer pathological diagnosis system based on deep transfer learning.
Table 2 is a table of comparative data of the lung cancer pathological diagnosis system based on deep transfer learning according to the present invention from the zero training model on the test set.
Table 3 is a table of performance data of the lung cancer pathological diagnosis system migration learning model on a test set based on deep migration learning.
Although embodiments of the present invention have been disclosed above, it is not limited to the details and embodiments shown and described, it is well suited to various fields of use for which the invention would be readily apparent to those skilled in the art, and accordingly, the invention is not limited to the specific details and illustrations shown and described herein, without departing from the general concepts defined in the claims and their equivalents.
Claims (10)
1. A lung cancer pathological diagnosis system based on deep migration learning, comprising:
the digital processing cancer area labeling module is used for collecting the existing lung tissue slices at a selected time interval, forming lung tissue digital pathological sections through digital conversion processing, and carrying out digital set division and cancer area pixel level labeling;
The pretreatment filtering segmentation recognition module is used for preprocessing digital pathological sections of lung tissues, filtering a section background area through an algorithm, and establishing a lung cancer area pixel-level recognition model through two-class semantic segmentation;
the transfer learning secondary optimization module is used for training the lung cancer area pixel level identification model, performing secondary training optimization on lung cancer area pixel level identification model parameters by using lung cancer training data through a transfer learning method, and acquiring a transfer learning lung cancer auxiliary diagnosis model;
and the lung cancer pathological diagnosis prediction module is used for performing prediction evaluation on the transfer learning lung cancer auxiliary diagnosis model through an external test set and performing performance evaluation through a plurality of evaluation indexes, extracting and inputting all lung tissue image block data, and outputting a lung cancer pathological diagnosis prediction result.
2. The lung cancer pathological diagnosis system based on deep migration learning of claim 1, wherein the digital processing cancer region labeling module comprises:
the section data covering and screening submodule is used for butt-jointing and collecting section data of a thoracic hospital and screening lung tissue pathological sections covering various lung tissue forms; the pathological section of the lung tissue comprises: normal lung histopathological section, adenocarcinoma lung histopathological section, squamous cell carcinoma lung histopathological section and small cell carcinoma lung histopathological section;
The pathological section digital processing submodule is used for carrying out digital conversion processing on the pathological section of the lung tissue to obtain the digital pathological section of the lung tissue; the digital conversion process includes: digital scanning, digital recording or digital recognition;
and the collection dividing cancer area labeling submodule is used for carrying out digital collection dividing and cancer area pixel-level labeling on digital pathological sections of lung tissues.
3. The lung cancer pathological diagnosis system based on deep migration learning of claim 1, wherein the preprocessing filtering segmentation recognition module comprises:
the sub-module for filtering the background area of the slice is used for preprocessing the digital pathological section of the lung tissue to filter the background area, and the Otsu algorithm is used for filtering the background area of the slice;
the training image block segmentation submodule segments the whole slice into training image blocks with the size of 320 multiplied by 320 according to the 200x visual field by a set step length, and the training image blocks comprise: 215876 blocks containing positive cancer area and 955416 blocks not containing negative cancer area;
and the lung cancer area identification model submodule is used for carrying out pixel level identification on a cancer area and a non-cancer area through a DeepLab v3 classification semantic segmentation model based on ResNet-50, so as to establish a lung cancer area pixel level identification model.
4. The lung cancer pathological diagnosis system based on deep transfer learning according to claim 1, wherein the transfer learning secondary optimization module comprises:
the initial parameter recognition training sub-module is used for training a lung cancer area pixel level recognition model; introducing a stomach cancer area identification model, and taking parameters of the stomach cancer area identification model as initial values to perform early initial training;
the transfer learning secondary training sub-module is used for performing secondary training optimization on model parameters by using lung cancer training data through a transfer learning method;
training equipment process parameter submodule, setting deep learning model training equipment and training process parameters; performing data enhancement of a lung tissue pathological diagnosis process; and obtaining an auxiliary diagnosis model of the lung cancer through transfer learning.
5. The lung cancer pathological diagnosis system based on deep transfer learning according to claim 1, wherein the lung cancer pathological diagnosis prediction module comprises:
the large-scale external test data acquisition sub-module is used for acquiring external test lung tissue slice data of a large-scale medical database as an external test set;
the prediction diagnosis index evaluation sub-module is used for performing prediction evaluation on the transfer learning lung cancer auxiliary diagnosis model and performing performance evaluation through a plurality of evaluation indexes;
And the transfer learning lung cancer diagnosis sub-module is used for extracting all effective lung tissue image blocks of the external test lung tissue slice data, inputting all the effective lung tissue image block data into the transfer learning lung cancer auxiliary diagnosis model, obtaining output data of the transfer learning lung cancer auxiliary diagnosis model and obtaining a lung cancer pathological diagnosis prediction result.
6. The lung cancer pathological diagnosis system based on deep migration learning of claim 2, wherein the set-dividing cancer region labeling submodule comprises:
the digital pathological section set dividing unit is used for dividing digital sets of digital pathological sections of lung tissues and dividing a training set and an internal test set;
the set division mode selection unit, the training set and the internal test set division mode include: dividing the training set into 316 training sets and 203 internal test sets;
and the cancer area pixel labeling unit is used for labeling the cancer area of the digital pathological section of the lung tissue with canceration in the training set at a pixel level through a labeling tool.
7. A lung cancer pathology diagnosis system based on deep transfer learning according to claim 3, wherein the lung cancer area recognition model submodule comprises:
The second-class semantic segmentation model unit is used for establishing a deep Lab v3 second-class semantic segmentation model based on ResNet-50;
the pixel-level model identification unit is used for carrying out pixel-level identification on the cancer area and the non-cancer area through the two-classification semantic segmentation model;
and the cancer area pixel level identification model unit establishes a lung cancer area pixel level identification model through the pixel level identification of the cancer area and the non-cancer area, and performs the pixel level identification of the cancer area and the non-cancer area.
8. The lung cancer pathological diagnosis system based on deep transfer learning according to claim 4, wherein the training device process parameter submodule comprises:
the training equipment process setting unit is used for setting deep learning model training equipment and training process parameters; the deep learning model training apparatus includes: a multiple GPU server and ADAM optimizer; the training process parameters include: training learning rate, batch size and number of rounds of training;
a diagnostic process data enhancement unit for performing data enhancement of a lung tissue pathological diagnostic process by random rotation and mirroring; data enhancement of lung histopathological diagnostic procedures by random rotation and mirroring includes: applying a random scale of 1.0x to 1.5x to the training data to make the model tolerant to small variations in the scaling ratio of the various digital scanning devices; and applying random Gaussian and dynamic blurring to the training image, and carrying out random disturbance on a brightness interval, a contrast interval, a tone interval and a saturation interval of the image to enhance the compatibility of the model to different dyeing configurations.
9. The lung cancer pathological diagnosis system based on deep migration learning of claim 5, wherein the predictive diagnosis index evaluation submodule comprises:
the external test set prediction evaluation unit is used for performing prediction evaluation on the lung cancer auxiliary diagnosis model through the external test set;
the evaluation index performance evaluation unit is used for performing performance evaluation on the migration learning lung cancer auxiliary diagnosis model through a plurality of evaluation indexes;
an evaluation index setting operation unit that performs a plurality of evaluation index operations in an evaluation process; the plurality of evaluation metrics includes: TP, FP, TN, and FN represent true positive, false positive, true negative, and false negative, respectively.
10. The lung cancer pathological diagnosis system based on deep transfer learning according to claim 5, wherein the transfer learning lung cancer diagnosis submodule comprises:
the lung tissue image block extraction unit is used for preprocessing the external test lung tissue slice data and extracting all valid lung tissue image blocks of the external test lung tissue slice data;
the lung cancer diagnosis model prediction unit inputs all effective lung tissue image blocks into a transfer learning lung cancer auxiliary diagnosis model, and the transfer learning lung cancer auxiliary diagnosis model outputs a pixel level prediction result of each image block to obtain transfer learning lung cancer auxiliary diagnosis model output data;
The lung cancer diagnosis probability selection unit is used for carrying out image block analysis to obtain a lung tissue slice pixel level prediction probability value; taking the average value of the top 100 probability values in the pixel level prediction probability values of each slice as the probability of positive prediction of the whole slice; and obtaining a lung cancer pathological diagnosis prediction result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310147859.4A CN116309368B (en) | 2023-02-21 | 2023-02-21 | Lung cancer pathological diagnosis system based on deep migration learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310147859.4A CN116309368B (en) | 2023-02-21 | 2023-02-21 | Lung cancer pathological diagnosis system based on deep migration learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116309368A true CN116309368A (en) | 2023-06-23 |
| CN116309368B CN116309368B (en) | 2023-11-14 |
Family
ID=86831584
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310147859.4A Active CN116309368B (en) | 2023-02-21 | 2023-02-21 | Lung cancer pathological diagnosis system based on deep migration learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116309368B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117373695A (en) * | 2023-10-12 | 2024-01-09 | 北京透彻未来科技有限公司 | Extreme deep convolutional neural network-based diagnosis system for diagnosis of cancer disease |
| CN117495800A (en) * | 2023-11-03 | 2024-02-02 | 北京透彻未来科技有限公司 | Pathological change prediction analysis system and method for pathological change image large model |
| CN117576127A (en) * | 2024-01-17 | 2024-02-20 | 神州医疗科技股份有限公司 | Liver cancer area automatic sketching method based on pathological image |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019110583A1 (en) * | 2017-12-07 | 2019-06-13 | Ventana Medical Systems, Inc. | Deep-learning systems and methods for joint cell and region classification in biological images |
| CN110459300A (en) * | 2019-07-22 | 2019-11-15 | 中国石油大学(华东) | A kind of lung cancer pathology type diagnostic method based on computer vision and CT images |
| CN110599448A (en) * | 2019-07-31 | 2019-12-20 | 浙江工业大学 | Migratory learning lung lesion tissue detection system based on MaskScoring R-CNN network |
| CN111814893A (en) * | 2020-07-17 | 2020-10-23 | 首都医科大学附属北京胸科医院 | Lung full-scan image EGFR mutation prediction method and system based on deep learning |
| CN112270666A (en) * | 2020-11-03 | 2021-01-26 | 辽宁工程技术大学 | Non-small cell lung cancer pathological section identification method based on deep convolutional neural network |
| CN112884724A (en) * | 2021-02-02 | 2021-06-01 | 广州智睿医疗科技有限公司 | Intelligent judgment method and system for lung cancer histopathological typing |
| CN112950461A (en) * | 2021-03-27 | 2021-06-11 | 刘文平 | Global and superpixel segmentation fused color migration method |
| CN114119977A (en) * | 2021-12-01 | 2022-03-01 | 昆明理工大学 | Graph convolution-based Transformer gastric cancer canceration region image segmentation method |
| CN115456961A (en) * | 2022-08-24 | 2022-12-09 | 四川大学华西医院 | Prediction system for benign pulmonary nodules, rapidly-progressing lung cancer and slowly-progressing lung cancer |
| CN115662635A (en) * | 2022-10-14 | 2023-01-31 | 中国人民解放军联勤保障部队第九二〇医院 | Method for establishing combined regression prediction model for radiation pneumonitis |
-
2023
- 2023-02-21 CN CN202310147859.4A patent/CN116309368B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019110583A1 (en) * | 2017-12-07 | 2019-06-13 | Ventana Medical Systems, Inc. | Deep-learning systems and methods for joint cell and region classification in biological images |
| CN110459300A (en) * | 2019-07-22 | 2019-11-15 | 中国石油大学(华东) | A kind of lung cancer pathology type diagnostic method based on computer vision and CT images |
| CN110599448A (en) * | 2019-07-31 | 2019-12-20 | 浙江工业大学 | Migratory learning lung lesion tissue detection system based on MaskScoring R-CNN network |
| CN111814893A (en) * | 2020-07-17 | 2020-10-23 | 首都医科大学附属北京胸科医院 | Lung full-scan image EGFR mutation prediction method and system based on deep learning |
| CN112270666A (en) * | 2020-11-03 | 2021-01-26 | 辽宁工程技术大学 | Non-small cell lung cancer pathological section identification method based on deep convolutional neural network |
| CN112884724A (en) * | 2021-02-02 | 2021-06-01 | 广州智睿医疗科技有限公司 | Intelligent judgment method and system for lung cancer histopathological typing |
| CN112950461A (en) * | 2021-03-27 | 2021-06-11 | 刘文平 | Global and superpixel segmentation fused color migration method |
| CN114119977A (en) * | 2021-12-01 | 2022-03-01 | 昆明理工大学 | Graph convolution-based Transformer gastric cancer canceration region image segmentation method |
| CN115456961A (en) * | 2022-08-24 | 2022-12-09 | 四川大学华西医院 | Prediction system for benign pulmonary nodules, rapidly-progressing lung cancer and slowly-progressing lung cancer |
| CN115662635A (en) * | 2022-10-14 | 2023-01-31 | 中国人民解放军联勤保障部队第九二〇医院 | Method for establishing combined regression prediction model for radiation pneumonitis |
Non-Patent Citations (4)
| Title |
|---|
| GE REN等: "A Transfer Learning Framework for Deep Learning-Based CT-to-Perfusion Mapping on Lung Cancer Patients", 《FRONT ONCOL》, pages 1 - 11 * |
| 姜云鹏: "面向医疗图像诊断的迁移学习策略", 《中国优秀硕士学位论文全文数据库 信息科技辑》, no. 1, pages 138 - 3540 * |
| 孙小涵 等: "基于灰度特征值的混杂性肺原位腺癌与微浸润性腺癌的鉴别", 《电脑与信息技术》, vol. 29, no. 4, pages 8 - 12 * |
| 那郎: "深度神经网络Deeplabv3_resnet50详解", pages 1 - 3, Retrieved from the Internet <URL:https://blog.csdn.net/weixin_44816589/article/details/115266935> * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117373695A (en) * | 2023-10-12 | 2024-01-09 | 北京透彻未来科技有限公司 | Extreme deep convolutional neural network-based diagnosis system for diagnosis of cancer disease |
| CN117373695B (en) * | 2023-10-12 | 2024-05-17 | 北京透彻未来科技有限公司 | Extreme deep convolutional neural network-based diagnosis system for diagnosis of cancer disease |
| CN117495800A (en) * | 2023-11-03 | 2024-02-02 | 北京透彻未来科技有限公司 | Pathological change prediction analysis system and method for pathological change image large model |
| CN117495800B (en) * | 2023-11-03 | 2024-05-17 | 北京透彻未来科技有限公司 | Pathological change prediction analysis system and method for pathological change image large model |
| CN117576127A (en) * | 2024-01-17 | 2024-02-20 | 神州医疗科技股份有限公司 | Liver cancer area automatic sketching method based on pathological image |
| CN117576127B (en) * | 2024-01-17 | 2024-04-19 | 神州医疗科技股份有限公司 | Liver cancer area automatic sketching method based on pathological image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116309368B (en) | 2023-11-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116309368B (en) | Lung cancer pathological diagnosis system based on deep migration learning | |
| CN111488921B (en) | Intelligent analysis system and method for panoramic digital pathological image | |
| Shi et al. | Automatic road crack detection using random structured forests | |
| CN112101451B (en) | Breast cancer tissue pathological type classification method based on generation of antagonism network screening image block | |
| CN113454733A (en) | Multi-instance learner for prognostic tissue pattern recognition | |
| CN110991536B (en) | Training method of early warning model of primary liver cancer | |
| CN112884724B (en) | Intelligent judgment method and system for lung cancer histopathological typing | |
| CN110097974A (en) | A kind of nasopharyngeal carcinoma far-end transfer forecasting system based on deep learning algorithm | |
| CN110111895A (en) | A kind of method for building up of nasopharyngeal carcinoma far-end transfer prediction model | |
| CN110796661B (en) | Fungal microscopic image segmentation detection method and system based on convolutional neural network | |
| CN109410238A (en) | A kind of fructus lycii identification method of counting based on PointNet++ network | |
| Wang et al. | A hybrid method for the segmentation of a ferrograph image using marker-controlled watershed and grey clustering | |
| CN112990214A (en) | Medical image feature recognition prediction model | |
| D’Acunto et al. | From human mesenchymal stromal cells to osteosarcoma cells classification by deep learning | |
| CN112820399A (en) | Method and device for automatically diagnosing benign and malignant thyroid nodules | |
| CN110391022A (en) | A kind of deep learning breast cancer pathological image subdivision diagnostic method based on multistage migration | |
| CN113657449A (en) | Traditional Chinese medicine tongue picture greasy classification method containing noise labeling data | |
| CN114782948B (en) | Global interpretation method and system for cervical fluid-based cytological smear | |
| CN115471701A (en) | Lung adenocarcinoma histology subtype classification method based on deep learning and transfer learning | |
| CN116363647A (en) | Lung cancer pathological tissue typing system based on deep semantic segmentation network | |
| KR20240012738A (en) | Cluster analysis system and method of artificial intelligence classification for cell nuclei of prostate cancer tissue | |
| Junod et al. | The development of an automatic recognition system for earmark and earprint comparisons | |
| CN113222944B (en) | Cell nucleus segmentation method and cancer auxiliary analysis system and device based on pathological image | |
| CN115239613A (en) | Full-field digital slice image classification modeling method and device based on integrated deep learning | |
| Pan et al. | A review of machine learning approaches, challenges and prospects for computational tumor pathology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |