CN110781953B - Lung cancer pathological section classification method based on multi-scale pyramid convolution neural network - Google Patents
Lung cancer pathological section classification method based on multi-scale pyramid convolution neural network Download PDFInfo
- Publication number
- CN110781953B CN110781953B CN201911016749.4A CN201911016749A CN110781953B CN 110781953 B CN110781953 B CN 110781953B CN 201911016749 A CN201911016749 A CN 201911016749A CN 110781953 B CN110781953 B CN 110781953B
- Authority
- CN
- China
- Prior art keywords
- lung cancer
- cancer
- neural network
- small blocks
- pathological
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/03—Recognition of patterns in medical or anatomical images
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention provides a lung cancer pathological section classification algorithm based on a multi-scale pyramid convolution neural network, and relates to the technical field of medical pathological image processing, artificial intelligence and computer vision. The medical pathological image processing technology comprises living tissue section diagnosis and labeling, and the artificial intelligence and computer vision technology comprises data cleaning of pathological section images, construction of a GPU cluster server and training of a multi-scale pyramid convolution neural network. The invention utilizes the characteristic that deep learning can automatically extract and classify the image features to replace the process of naked eye diagnosis by a pathologist in the traditional method. This greatly shortens the time of diagnosis. Meanwhile, as an auxiliary diagnosis measure, the method can reduce the misjudgment caused by human negligence and improve the diagnosis accuracy.
Description
Technical Field
The invention relates to the technical field of medical image processing, artificial intelligence and pattern recognition, in particular to a lung cancer pathological section classification method based on a multi-scale pyramid convolutional neural network.
Background
Lung cancer is a common cancer. In recent years, the incidence and mortality of lung cancer are continuously increased, and great threat is brought to the health and life of people. Medical research shows that the survival rate of lung cancer patients is improved by 30% if lung cancer can be discovered as early as possible and the lung cancer can be treated in a targeted way according to the types of the lung cancer. Currently, the analysis of lung cancer pathological images mainly depends on professional pathologist diagnosis. However, each histopathology image is a high-resolution image at the level of billions of pixels, and classification by manpower alone is not only heavy in workload, but also prone to erroneous judgment and missed judgment, which is very disadvantageous to lung cancer treatment. Therefore, the automatic classification and judgment of lung cancer pathological images is a very popular interdisciplinary research direction. At present, some researches are carried out to automatically detect and classify by combining the traditional manual characteristics with the SVM. However, such methods rely on the design of artificial features and have poor generalization performance.
In recent years, convolutional neural networks have been highlighted in ImageNet image sorting games. The convolutional neural network based on deep learning creates more possibilities for the image classification task. The image classification is realized by simulating a neural network mechanism of a human brain. At present, the image classification precision of the convolutional neural network exceeds that of human beings, and the development of the field of artificial intelligence is greatly promoted. In medical images, the resolution due to pathological sections is on the order of billions of pixels. At low resolution, the overall pathology can be acquired, while at low resolution, the pathology can be identified in more detail. The multi-scale fusion mode can greatly enhance the robustness of the model and improve the classification precision.
Disclosure of Invention
The invention provides a lung cancer pathological section classification method based on a multi-scale pyramid convolutional neural network, which can automatically detect whether a pathological section is cancerous, a cancerous area and a cancerous category.
In order to achieve the technical effects, the technical scheme of the invention is as follows:
a lung cancer pathological section classification method based on a multi-scale pyramid convolutional neural network judges whether a pathological section is cancerous or not and determines a lung cancer region of the cancerous pathological section, and comprises the following specific steps:
s11: collecting lung cancer pathological section and lung cancer-free normal section. For pathological lung cancer sections, a plurality of medical professionals judge whether the sections are cancerous, and determine the area of cancerous cells and frame the area by a labeling frame. For normal sections, the positions of normal cells are framed by labeling frames by a plurality of professional doctors;
s12: extracting a plurality of small blocks with the resolution of 10x,20x and 40x for the labeling boxes of the lung cancer pathological section and the lung cancer-free normal section in the step S11, and balancing the number of the small blocks in the two categories;
s13: inputting the small blocks with the same central coordinates and multiple scales into a multi-scale pyramid convolutional neural network, outputting the probability of cancer in the region, calculating error cost, and updating network parameters through back propagation;
s15: in the testing process, judging the small blocks with cancer probability higher than a preset threshold value as cancer small blocks, and judging the rest as cancer-free small blocks;
s16: for the test section, if the number of cancer small blocks is larger than the preset threshold value, we consider the section to be the section with lung cancer, otherwise, we consider the section to be the normal section.
Further, classifying the lung cancer types of the pathological sections with cancer, which comprises the following steps:
s21: and (5) selecting all lung cancer pathological section images from the S11. Framing the cancer-affected area by using a marking frame by a plurality of professional doctors, and determining the type of the sliced lung cancer;
s22: extracting a plurality of small blocks at the resolution of 10x,20x and 40x for the labeling box of the lung cancer pathological section in the S21, and balancing the number of the small blocks in each category;
s23: inputting the small blocks with the same central coordinates and multiple scales into a multi-scale pyramid convolutional neural network, and outputting the lung cancer category of each small block;
s24: in the test stage, the cancer-affected small blocks of the section judged to be affected with lung cancer in S16 are classified into categories, and the lung cancer category of the pathological section is determined by a voting mechanism.
Further, the multi-scale pyramid convolutional neural network building and training process is as follows:
s31: the multi-scale pyramid convolutional neural network is composed of three branch networks with the same structure and different input scales, a multi-scale feature fusion layer and a lung cancer local slice prediction layer. The parameters of each branch network at the multi-scale feature fusion layer are initialized by the corresponding parameters in the ResNet50 network pre-trained in the ImageNet dataset. Parameters of the multi-scale feature fusion layer and the lung cancer local slice prediction layer are randomly initialized by truncated normal distribution;
s32: in the forward propagation process, the network lung cancer local slice prediction layer outputs the probability that each small block belongs to each lung cancer category;
s33: calculating the prediction precision, and performing cost calculation on each small batch of samples by adopting a cross entropy cost function;
s34: and adjusting the model parameters by back propagation until the prediction accuracy of the model is converged.
Compared with the prior art, the technical scheme of the invention has the beneficial effects that:
the invention provides a lung cancer pathological section classification method based on a multi-scale pyramid convolutional neural network. The multi-scale pyramid convolution neural network provides multi-scale image input and multi-scale feature fusion on the basis of the existing image classification model ResNet50, and improves the robustness of the network model to billions of pixel level pathological images. Meanwhile, the neural network can quickly calculate the cancer area and output the corresponding lung cancer category, so that the workload of doctors is reduced, and the possibility of misjudgment and missed judgment can be reduced.
Drawings
FIG. 1 is a multi-scale pyramid convolutional neural network model;
fig. 2 is a flowchart of a lung cancer pathological section classification method based on a multi-scale pyramid convolutional neural network.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the patent;
for the purpose of better illustrating the present embodiments, certain elements of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product;
it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The technical solution of the present invention is further described below with reference to the accompanying drawings and examples.
Example 1
As shown in fig. 1, a lung cancer pathological section classification method based on a multi-scale pyramid convolutional neural network includes the following steps:
s1, judging whether pathological section is cancerous or not and determining lung cancer area of cancerous pathological section
S11: collecting lung cancer pathological section and lung cancer-free normal section. For pathological lung cancer sections, a plurality of medical professionals judge whether the sections are cancerous, and determine the area of cancerous cells and frame the area by a labeling frame. For normal sections, the positions of normal cells are framed by labeling frames by a plurality of professional doctors;
s12: extracting a plurality of small blocks with the resolution of 10x,20x and 40x for the labeling boxes of the lung cancer pathological section and the lung cancer-free normal section in the step S11, and balancing the number of the small blocks in the two categories;
s13: inputting the small blocks with the same central coordinates and multiple scales into a multi-scale pyramid convolutional neural network, outputting the probability of cancer in the region, calculating error cost, and updating network parameters through back propagation;
s15: in the testing process, for small blocks with cancer probability higher than a preset threshold value, judging the small blocks as small blocks with cancer, and judging the rest small blocks as small blocks without cancer;
s16: for the test section, if the number of cancer small blocks is larger than the preset threshold value, we consider the section to be the section with lung cancer, otherwise, we consider the section to be the normal section.
S2, classifying the lung cancer categories of cancer pathological sections
S21: and (5) selecting all lung cancer pathological section images from the S11. Framing the cancer-affected area by using a marking frame by a plurality of professional doctors, and determining the type of the sliced lung cancer;
s22: extracting a plurality of small blocks at the resolution of 10x,20x and 40x for the labeling box of the lung cancer pathological section in the S21, and balancing the number of the small blocks in each category;
s23: inputting the small blocks with the same central coordinates and multiple scales into a multi-scale pyramid convolutional neural network, and outputting the lung cancer category of each small block;
s24: in the test stage, the cancer-affected small blocks of the section judged to be affected with lung cancer in S16 are classified into categories, and the lung cancer category of the pathological section is determined by a voting mechanism.
S3 multi-scale pyramid convolution neural network building and training
S31: the multiscale pyramid convolution neural network is composed of three branch networks with the same structure and different input scales, a multiscale feature fusion layer and a lung cancer local slice prediction layer, parameters of each branch network in the multiscale feature fusion layer are initialized by corresponding parameters in a ResNet50 network pre-trained in an ImageNet data set, and parameters of the multiscale feature fusion layer and the lung cancer local slice prediction layer are randomly initialized by truncation normal distribution;
s32: in the forward propagation process, the network lung cancer local slice prediction layer outputs the probability that each small block belongs to each lung cancer category;
s33: calculating the prediction precision, and performing cost calculation on each small batch of samples by adopting a cross entropy cost function;
s34: and adjusting the model parameters by back propagation until the prediction accuracy of the model is converged.
The same or similar reference numerals correspond to the same or similar parts;
the positional relationships depicted in the drawings are for illustrative purposes only and are not to be construed as limiting the present patent;
it should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. This need not be, nor should it be exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.
Claims (2)
1. A lung cancer pathological section classification method based on a multi-scale pyramid convolution neural network is characterized by comprising the following steps:
the method comprises the following steps: judging whether the pathological section is cancerous and determining the lung cancer area of the pathological section with cancer;
step two: classifying the lung cancer category of the pathological section with cancer;
the specific steps of the first step are as follows:
s11: collecting lung cancer pathological sections and normal sections without lung cancer, judging whether the sections are cancerous or not by a plurality of professional doctors for the lung cancer pathological sections, determining the cancerous cell area and framing the area by using a labeling frame, and framing the normal cell position by using the labeling frame for the normal sections by the plurality of professional doctors;
s12: extracting a plurality of small blocks at the resolution of 10x,20x and 40x from the labeling boxes of the lung cancer pathological section and the lung cancer-free normal section in S11, and balancing the number of the small blocks in the two categories;
s13: inputting the small blocks with the same central coordinates and multiple scales in the S11 into a multi-scale pyramid convolutional neural network, outputting the probability of cancer in the region, calculating error cost, and updating network parameters through back propagation;
s15: judging the small blocks with the cancer probability higher than a preset threshold value as cancer small blocks, judging the rest as cancer-free small blocks, if the number of the cancer small blocks is larger than or equal to that of pathological sections with the preset threshold value, considering the sections with the lung cancer, and if the number of the cancer small blocks is smaller than that of the pathological sections with the preset threshold value, considering the sections as normal sections;
the second step comprises the following specific steps:
s21: selecting all lung cancer pathological section images from the S11, framing the cancer-affected area by using a labeling frame by a plurality of professional doctors, and determining the section lung cancer category;
s22: extracting a plurality of small blocks at the resolution of 10x,20x and 40x for the labeling box of the lung cancer pathological section in the S21, and balancing the number of the small blocks in each category;
s23: inputting the small blocks with the same central coordinates and multiple scales in the S22 into the multi-scale pyramid convolutional neural network, outputting the lung cancer category of each small block, calculating error cost according to the cross entropy cost function, and updating parameters of the convolutional neural network through back propagation;
s24: given the test section confirmed to have cancer by step one, the small blocks predicted to have cancer in S15 are classified into categories, and the lung cancer category of the pathological section is determined by a voting mechanism.
2. The method for classifying lung cancer pathological sections based on the multi-scale pyramid convolutional neural network as claimed in claim 1, wherein the multi-scale pyramid convolutional neural network is built and trained as follows:
s31: the multiscale pyramid convolutional neural network is composed of branch networks with the same structure, a multiscale feature fusion layer and a small cancer-affected block type prediction layer of a lung cancer slice, parameters of each branch network in front of the multiscale feature fusion layer are initialized by corresponding parameters in a ResNet50 network pre-trained in an ImageNet data set, and parameters of the multiscale feature fusion layer and the lung cancer local slice prediction layer are initialized randomly by truncation normal distribution;
s32: in the forward propagation process, the network lung cancer local slice prediction layer outputs the probability that each small block belongs to each lung cancer category;
s33: calculating the prediction accuracy of the cancer small block types, and performing cost calculation on each small batch of samples by adopting a cross entropy cost function;
s34: and (5) adjusting the model parameters through back propagation until the prediction precision of the model is converged.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911016749.4A CN110781953B (en) | 2019-10-24 | 2019-10-24 | Lung cancer pathological section classification method based on multi-scale pyramid convolution neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911016749.4A CN110781953B (en) | 2019-10-24 | 2019-10-24 | Lung cancer pathological section classification method based on multi-scale pyramid convolution neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110781953A CN110781953A (en) | 2020-02-11 |
| CN110781953B true CN110781953B (en) | 2023-03-31 |
Family
ID=69387016
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911016749.4A Active CN110781953B (en) | 2019-10-24 | 2019-10-24 | Lung cancer pathological section classification method based on multi-scale pyramid convolution neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110781953B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111310841B (en) * | 2020-02-24 | 2023-06-20 | 中南大学湘雅医院 | Medical image classification method, medical image classification device, medical image classification apparatus, medical image classification computer device, and medical image classification storage medium |
| CN111652927B (en) * | 2020-05-11 | 2023-12-19 | 广东亿云付科技有限公司 | Cancer cell multi-scale scaling positioning detection method based on CNN |
| CN111784671B (en) * | 2020-06-30 | 2022-07-05 | 天津大学 | Pathological image focus region detection method based on multi-scale deep learning |
| CN113034443A (en) * | 2021-03-04 | 2021-06-25 | 广州乐智医疗科技有限公司 | Method and system for processing lung cancer pathological section image based on multi-level discrimination |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107368670A (en) * | 2017-06-07 | 2017-11-21 | 万香波 | Stomach cancer pathology diagnostic support system and method based on big data deep learning |
| CN108305249A (en) * | 2018-01-24 | 2018-07-20 | 福建师范大学 | The quick diagnosis and methods of marking of full size pathological section based on deep learning |
| CN108765408A (en) * | 2018-05-31 | 2018-11-06 | 杭州同绘科技有限公司 | Build the method in cancer pathology image virtual case library and the multiple dimensioned cancer detection system based on convolutional neural networks |
| CA3081643A1 (en) * | 2017-11-06 | 2019-05-09 | University Health Network | Platform, device and process for annotation and classification of tissue specimens using convolutional neural network |
-
2019
- 2019-10-24 CN CN201911016749.4A patent/CN110781953B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107368670A (en) * | 2017-06-07 | 2017-11-21 | 万香波 | Stomach cancer pathology diagnostic support system and method based on big data deep learning |
| CA3081643A1 (en) * | 2017-11-06 | 2019-05-09 | University Health Network | Platform, device and process for annotation and classification of tissue specimens using convolutional neural network |
| CN108305249A (en) * | 2018-01-24 | 2018-07-20 | 福建师范大学 | The quick diagnosis and methods of marking of full size pathological section based on deep learning |
| CN108765408A (en) * | 2018-05-31 | 2018-11-06 | 杭州同绘科技有限公司 | Build the method in cancer pathology image virtual case library and the multiple dimensioned cancer detection system based on convolutional neural networks |
Non-Patent Citations (2)
| Title |
|---|
| 基于深度卷积神经网络算法的肺结节检测模型;杨晶晶等;《数学建模及其应用》;20171215(第04期);全文 * |
| 基于深度学习的胃癌病理图像分类方法;张泽中等;《计算机科学》;20181115;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110781953A (en) | 2020-02-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110781953B (en) | Lung cancer pathological section classification method based on multi-scale pyramid convolution neural network | |
| CN108806792B (en) | Deep learning face diagnosis system | |
| CN105069304B (en) | A kind of device of the assessment prediction ASD based on machine learning | |
| CN111178197B (en) | Mass R-CNN and Soft-NMS fusion based group-fed adherent pig example segmentation method | |
| CN110837803B (en) | Diabetic retinopathy grading method based on depth map network | |
| CN108464840B (en) | Automatic detection method and system for breast lumps | |
| CN109543526B (en) | True and false facial paralysis recognition system based on depth difference characteristics | |
| Beevi et al. | A multi-classifier system for automatic mitosis detection in breast histopathology images using deep belief networks | |
| CN107977671A (en) | A kind of tongue picture sorting technique based on multitask convolutional neural networks | |
| CN109583342A (en) | Human face in-vivo detection method based on transfer learning | |
| CN114846507A (en) | Method and system for non-invasive gene detection using Artificial Intelligence (AI) models | |
| CN109508644A (en) | Facial paralysis grade assessment system based on the analysis of deep video data | |
| CN106340000A (en) | Bone age assessment method | |
| Mahapatra et al. | A novel framework for retinal vessel segmentation using optimal improved frangi filter and adaptive weighted spatial FCM | |
| CN112819821B (en) | Cell nucleus image detection method | |
| CN109448307A (en) | A kind of recognition methods of fire disaster target and device | |
| Yu et al. | A recognition method of soybean leaf diseases based on an improved deep learning model | |
| CN112712122A (en) | Corneal ulcer classification detection method and system based on neural network model | |
| CN113343755A (en) | System and method for classifying red blood cells in red blood cell image | |
| CN112215268A (en) | Method and device for classifying disaster weather satellite cloud pictures | |
| Yang et al. | Retinal vessel segmentation based on an improved deep forest | |
| CN116484916A (en) | Chicken health state detection method and method for building detection model thereof | |
| CN110598724B (en) | Cell low-resolution image fusion method based on convolutional neural network | |
| Sun et al. | Detection of breast tumour tissue regions in histopathological images using convolutional neural networks | |
| CN116310525A (en) | Pathological image classification method based on contrast representation distillation and output distillation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |