CN114841979A - Multi-scale attention-fused deep learning cancer molecular typing prediction method - Google Patents
Multi-scale attention-fused deep learning cancer molecular typing prediction method Download PDFInfo
- Publication number
- CN114841979A CN114841979A CN202210543763.5A CN202210543763A CN114841979A CN 114841979 A CN114841979 A CN 114841979A CN 202210543763 A CN202210543763 A CN 202210543763A CN 114841979 A CN114841979 A CN 114841979A
- Authority
- CN
- China
- Prior art keywords
- convolution
- blocks
- pyramid
- module
- attention
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16B—BIOINFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR GENETIC OR PROTEIN-RELATED DATA PROCESSING IN COMPUTATIONAL MOLECULAR BIOLOGY
- G16B40/00—ICT specially adapted for biostatistics; ICT specially adapted for bioinformatics-related machine learning or data mining, e.g. knowledge discovery or pattern finding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Biomedical Technology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Databases & Information Systems (AREA)
- Epidemiology (AREA)
- Bioethics (AREA)
- Public Health (AREA)
- Biotechnology (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a multi-scale attention-fused deep learning cancer molecular typing prediction method, relates to the technical field of pathological image intelligent processing, and aims to solve the problem that most methods divide a full-size digital pathological image into small blocks under a certain multiplying power to further train a classification model of the block level due to overlarge pixel size; the information emphasis is different according to different scales of pathological images, the deep neural network is built to concern the spatial scale information, the channel information is also concerned, the pattern block information is fully utilized from multiple dimensions, when the prior knowledge is not available to determine the characteristic expression of the molecular typing in the pathological images, the spatial information of different scales is fully utilized, different channels of model training are emphasized to some extent, so that the model is easier to explore the characteristic expression related to the molecular typing, and the prediction performance is improved.
Description
Technical Field
The invention relates to the technical field of pathological image intelligent processing, in particular to a multi-scale attention-fused deep learning cancer molecular typing prediction method.
Background
The world health organization's affiliated international agency for research on cancer (IARC) data showed that in 2020, the number of new cancer cases in china is about 457 thousands, which exceeds that in other countries of the world. Cancer has become a significant public health problem. Accurate cancer prognosis analysis is helpful for assisting doctors to make diagnosis and treatment judgment, and improves survival opportunity and survival level of patients. Molecular typing provides little help in methods for prognostic prediction, while methods for obtaining molecular typing tend to be expensive and time consuming. Therefore, a more convenient and inexpensive secondary method is needed. The development of deep learning provides the possibility for the deep learning. For example, Hinata M, Ushiku T.Detecting immunological-pathological Subtype using pathological image-based diagnosis [ J ] Science Reports,2021,11:22636. EBV and MSI/dMMR two types highly sensitive to immune checkpoint inhibitors are screened using pathological images in the context of gastric adenocarcinoma patients using transfer Learning, Acwa B, Me C, Jga B, et al.Deep Learning precursors Molecular Subtype of Muscan-innovative vane Cancer from structural diagnostic Slides [ J ]. European Urology,2020,78(2): 256. in the context of pathological image NN network, MDA is predicted for Bladder Cancer using pathological images. Due to the fact that the pixel size of the full-size digital pathological image is too large, most methods divide the full-size digital pathological image into small blocks under a certain multiplying power to further train the classification model at the block level. Because the determination of molecular typing is usually from a molecular method, the characteristics on the image are not clear, so that the related characteristics cannot be fully explored by a fixed scale, and meanwhile, the information of different scales of pathological images is not fully utilized. In the training process of the block, the existing method does not fully utilize the information of the block, such as a channel with the emphasis of the feature, and the like.
Disclosure of Invention
In view of the problems in the prior art, the invention discloses a multi-scale attention-fused deep learning cancer molecular typing prediction method, which adopts the technical scheme that the method comprises the following steps:
s1, acquiring and preprocessing data, acquiring molecular typing of a cancer patient, acquiring a full-size digital pathological section image of the corresponding patient, selecting a proper multiplying power to perform sliding window segmentation on the pathological image according to the characteristics of the cancer species and known related pathological knowledge of the molecular typing, acquiring non-overlapping blocks with fixed size, discarding the blocks if the size of the blocks cut out from the edge of the image is less than the fixed size, evaluating the rest blocks, screening out blocks with effective tissues accounting for less than 50% of the blocks according to pixel values, performing color standardization processing on the rest blocks, and unifying the dyeing space;
s2, screening tumor blocks, wherein cancer molecular classification and tumor correlation are higher, necrosis, pure stroma and tumor-containing blocks are marked, and because the classification characteristics are obvious and the classification difficulty is relatively low, the existing classification network in the computer vision field is used for transfer learning, after a trained model is obtained, all blocks are screened, and the tumor-containing blocks are screened;
s3, constructing a multi-scale attention-fused deep neural network model, introducing a pyramid convolution PyConv module and a channel attention SE module by taking ResNet-50 as a backbone network, and constructing the deep neural network model
S4, training a pattern block classification model, setting a training learning rate, an optimizer and a loss function, randomly cutting (3,224,224) the pattern blocks in a training set, sending the cut pattern blocks into a built neural network, calculating a loss value, updating network parameters after back propagation, iterating for multiple times to complete model training, and realizing classification of pattern block molecular classification;
and S5, block and case prediction, wherein the test set blocks are randomly cut to (3,224,224) and sent to a trained model, forward propagation is carried out to obtain the test value of the final block, and the molecular typing true value is the true value of the case level, so that the predicted values of all the block of the case are averaged to represent the case prediction result, and the case molecular typing prediction is realized.
As a preferred technical scheme of the invention, the deep neural network model with multi-scale fusion attention in S3 is constructed, the size of the graph block sent to the network is required to be (3,224,224), and the network construction specific steps are as follows:
s301, a convolution layer and a batch normalization layer are built at the beginning of the network, and the output size of the convolution layer and the batch normalization layer is (64,112,112) through the ReLU;
s302, acquiring multi-scale information of the pathological image, wherein the pathological image consists of 3 pyramid convolution model blocks which are basically the same;
s303, obtaining attention of the image block channel, and building 4 basically same attention modules;
s304, acquiring multi-scale information of the pathological image, wherein the pathological image consists of 6 pyramid convolution modules;
s305, acquiring pathological image information, wherein the pathological image information consists of 3 pyramid convolution modules;
s306, the preamble feature set passes through the self-adaptive average pooling and full-connection layer at this stage and is finally transmitted to the output, the number of output nodes is the number of types of molecular classification, and the output result is the prediction probability of each type.
As a preferred technical solution of the present invention, in S302, each module is respectively connected in series by a combination of three convolution layers and a batch normalization layer, and is output through a ReLU, wherein a second group of convolution layers is a pyramid convolution, three pyramid convolution modules differ only in that a first module adds a combination of a group of maximum pooling layer, convolution layer, and batch normalization layer at the end to perform downsampling, all pyramid convolutions in this stage are parallel to convolution kernels of four sizes, the sizes of the convolution kernels are 3 × 3, 5 × 5, 7 × 7, and 9 × 9, so as to form a pyramid form, after a preamble feature set is convolved with each convolution kernel, generated features are spliced, and a feature set of (256,56,56) size is finally output in this stage.
As a preferred technical solution of the present invention, in S303, only the first module performs down-sampling on the last combination of a group of maximum pooling layers, convolutional layers, and batch normalization layers, the attention module embeds the SE module into a residual module of ResNet-50 to form an SE-ResNet module, which is based on the residual module, and compresses the spatial dimension of a preamble feature set into a real number as a channel descriptor through global pooling before jump connection, and generates each channel weight to map to each channel of the feature through full connection layer operation, and the stage outputs a feature set with a size of (512,28, 28).
In a preferred embodiment of the present invention, in S304, the parallel convolution kernels in each pyramid convolution have two sizes, 3 × 3 and 5 × 5, respectively, and the feature set with the size (1024,14,14) is output at this stage.
As a preferred embodiment of the present invention, in S305, the difference from S302 is 3 pyramid convolution modules in this stage, and only one convolution kernel with a size of 3 × 3 in each pyramid convolution, and since only one convolution kernel with a size remains in the pyramid convolution, the actual pyramid convolution is reduced to a normal convolution layer with a convolution kernel of 3 × 3, and the output size at this stage is (2048,7, 7).
The invention has the beneficial effects that: according to the method, different information emphasis is introduced according to different scales of pathological images, a deep neural network is built to pay attention to spatial scale information, channel information is paid attention to, image block information is fully utilized from multiple dimensions, when the feature expression of molecular typing in the pathological images is determined without prior knowledge, the spatial information of different scales is fully utilized, different channels of model training are emphasized, so that the model is easier to explore the feature expression related to the molecular typing, and the prediction performance is improved.
Drawings
In order to more clearly illustrate the detailed description of the invention or the technical solutions in the prior art, the drawings that are needed in the detailed description of the invention or the prior art will be briefly described below. In all the drawings, the elements or parts are not necessarily drawn to actual scale.
FIG. 1 is a flow chart of the method for predicting cancer molecular typing according to the present invention;
FIG. 2 is a diagram of a multi-scale attention-fused deep neural network model according to the present invention;
FIG. 3 is a schematic diagram of the pyramid convolution of the present invention;
FIG. 4 is a schematic diagram of the SE-ResNet attention module of the present invention.
Detailed Description
Example 1
As shown in fig. 1 to 4, the invention discloses a multi-scale attention-fused deep learning cancer molecular typing prediction method, which adopts the technical scheme that the method comprises the following steps:
s1, data acquisition and preprocessing, in the specific embodiment, a data set is acquired by a TCGA database, pathological images of muscle-layer invasive bladder cancer in the data set are used, UNC typing is used as a prediction target, 165 examples of basal models and 210 examples of lumen models are used, the acquired case data set is randomly divided into a training set, a verification set and a test set according to 6:2:2, all pathological images are subjected to non-overlapping segmentation to obtain 256 pixel-level image blocks, image blocks with insufficient size are selected and discarded after image edge segmentation, image blocks with insufficient information are screened out according to image block conditions, the image color is darker, RGB image blocks are converted into gray-scale images, the pixel proportion of gray-scale values in the image blocks is calculated, the proportion is larger than 220%, the information is considered to be insufficient, discarding processing is carried out, the rest image blocks are subjected to color standardization, and a Macenko method is adopted in the embodiment;
s2, screening tumor blocks, selecting preprocessed blocks, randomly selecting and labeling necrotic, pure stroma and tumor-containing blocks by experts, labeling 2000 blocks, performing migration learning by using an ImageNet pre-trained increment V3 model, randomly dividing the labeled blocks into a training set, a verification set and a test set according to the ratio of 6:2:2, amplifying the training set to (3,299,299), sending the training set to an increment V3 model for training, performing model verification and test by using the verification set and the test set respectively, training by using random gradient descent as an optimizer, wherein the learning rate is 0.01, the training batch size is 64, training 10000 steps are performed, the test set is sent to a trained network for testing, the evaluation index AUC reaches 0.99, the performance is good, and all blocks are screened by using the model to screen the tumor-containing blocks;
s3, constructing a multi-scale attention-fused deep neural network model;
s301, the network outputs a feature set with the size (64,112,112) through a convolution layer, a batch normalization layer and a ReLU;
s302, obtaining multi-scale information of pathological images, building three basically identical pyramid convolution modules, wherein each module is respectively formed by connecting three groups of convolution layers and a batch normalization layer in series in a combined mode and is output through a ReLU, the second group of convolution layers are pyramid convolutions, the three pyramid convolution modules are different in that only one module is added with a group of maximum pooling layers, convolution layers and a batch normalization layer in the last step for down sampling, all pyramid convolutions in the stage are parallel to convolution kernels with four sizes, the sizes of the convolution kernels are 3 x 3, 5 x 5, 7 x 7 and 9 x 9 respectively, a pyramid form is formed, after convolution of a preorder feature set and each convolution kernel, generated features are spliced, a schematic diagram shown in a figure 3 is a pyramid convolution of the convolution kernels with four sizes, and the final output size of the feature set is (256,56 and 56);
s303, obtaining attention of image block channels, building 4 basically same attention modules, performing down-sampling on a combination of a group of maximum pooling layers, convolution layers and batch normalization layers added at the last of the first module, embedding an SE module into a residual error module of ResNet-50 to form the SE-ResNet module by the attention module, compressing spatial dimensions of a preamble feature set into real numbers as channel descriptors by global pooling before jump connection by taking the residual error module as a basis as shown in FIG. 4, generating channel weights to be mapped to each channel of the features by full connection layer operation, and outputting the feature set with the size of (512,28,28) at the stage;
s304, obtaining multi-scale information of the pathological image, wherein the stage is similar to the stage S302, the difference is that the stage has 6 pyramid convolution modules, parallel convolution kernels in each pyramid convolution have two sizes, namely 3 x 3 and 5 x 5, and the stage outputs a feature set with the size of (1024,14 and 14);
s305, pathological image information is obtained, the stage is similar to the stage S302, the difference is in the stage, 3 pyramid convolution modules are arranged, only one convolution kernel with the size of 3 x 3 is arranged in each pyramid convolution, the actual pyramid convolution is reduced to a common convolution layer with the convolution kernel of 3 x 3 because only one convolution kernel with the size is left in the pyramid convolution, and the stage outputs a feature set with the size of (2048,7, 7);
s306, the pre-order feature set passes through a self-adaptive average pooling layer and a full connection layer at the stage and is finally transmitted to an output, the number of output nodes is 2, and the output nodes are respectively prediction results of a basal type and a lumen type
S4, training a pattern block classification model, reducing pattern blocks of a training set to (3,224,224), sending the pattern blocks into a constructed model, taking cross entropy as a loss function, Adam as an optimizer, beginning learning rate of 0.01, reducing the pattern blocks to one tenth of the original pattern blocks respectively during training of 10 rounds, 35 rounds and 50 rounds, training batch size of 64 rounds, carrying out 60 rounds of training, carrying out back propagation, updating network parameters, completing training, reducing pattern blocks of a verification set to (3,224,224) during training, sending the pattern blocks into the currently trained model for verification, and selecting the model with optimal performance of the verification set as the trained model;
and S5, block diagram and case prediction, namely, reducing block diagram blocks of the test set to (3,224,224), sending the block diagram blocks into a trained network, outputting the block diagram by the network, obtaining predicted values of the block diagram blocks through softmax, obtaining two types of predicted values respectively according to the result of the second classification, and taking the average value of UNC typing predicted values of all the block diagram blocks containing the tumor of the case as the final predicted value of the case.
In this embodiment, a prediction test is performed on a test set, and a comparison experiment is performed on a model and ResNet, PyConvResNet, and SE-ResNet, where PyConvResNet embeds all main modules of ResNet into a pyramid convolution, and SE-ResNet embeds all main modules of ResNet into an SE module, the present invention integrates two types of pyramid convolution and SE module, embeds the pyramid convolution and the SE module into ResNet-50, and the four models all use ResNet-50 as a backbone, and the experimental results are shown in table 1:
TABLE 1 comparative experiment
Through comparison of experimental results, the network constructed by fusing the pyramid convolution module and the attention module SE module can effectively improve the prediction performance of cancer molecule typing predicted by pathological images, the effectiveness of the method is proved, and assistance is provided for a deep learning method in application of cancer molecule typing predicted by pathological images.
Although the present invention has been described in detail with reference to the specific embodiments, the present invention is not limited to the above embodiments, and various changes and modifications without inventive changes may be made within the knowledge of those skilled in the art without departing from the spirit of the present invention.
Claims (6)
1. A multi-scale attention-fused deep learning cancer molecular typing prediction method is characterized by comprising the following steps: the method comprises the following steps:
s1, acquiring and preprocessing data, acquiring molecular typing of a cancer patient, acquiring a full-size digital pathological section image of the corresponding patient, selecting a proper multiplying power to perform sliding window segmentation on the pathological image according to the characteristics of the cancer species and known related pathological knowledge of the molecular typing, acquiring non-overlapping blocks with fixed size, discarding the blocks if the size of the blocks cut out from the edge of the image is less than the fixed size, evaluating the rest blocks, screening out blocks with effective tissues accounting for less than 50% of the blocks according to pixel values, performing color standardization processing on the rest blocks, and unifying the dyeing space;
s2, screening tumor blocks, wherein cancer molecular classification and tumor correlation are higher, necrosis, pure stroma and tumor-containing blocks are marked, and because the classification characteristics are obvious and the classification difficulty is relatively low, the existing classification network in the computer vision field is used for transfer learning, after a trained model is obtained, all blocks are screened, and the tumor-containing blocks are screened;
s3, constructing a multi-scale attention-fused deep neural network model, introducing a pyramid convolution PyConv module and a channel attention SE module by taking ResNet-50 as a backbone network, and constructing the deep neural network model;
s4, training a pattern block classification model, setting a training learning rate, an optimizer and a loss function, randomly cutting pattern blocks of a training set to a length of 3, a width of 224 and a height of 224, sending the pattern blocks into the built neural network, calculating a loss value, updating network parameters after back propagation, iterating for multiple times to complete model training, and realizing classification of pattern block molecular classification;
s5, block and case prediction, randomly cutting the test set blocks to length 3, width 224 and height 224, sending the blocks into a trained model, and transmitting the blocks forward to obtain the test value of the final block, wherein the molecular typing true value is the true value of the case level, so that the predicted values of all blocks of the case are averaged to represent the case prediction result, and the case molecular typing prediction is realized.
2. The method for predicting the molecular typing of cancer based on multi-scale fusion attention as claimed in claim 1, wherein: building the multi-scale attention-fused deep neural network model in S3, wherein the required sizes of the image blocks sent into the network are 3,224 and 224, and the concrete steps of building the network are as follows:
s301, a network starts to build a convolution layer and a batch normalization layer, and the output sizes of the convolution layer and the batch normalization layer are 64 long, 112 wide and 112 high through a ReLU;
s302, acquiring multi-scale information of the pathological image, wherein the pathological image consists of 3 pyramid convolution model blocks which are basically the same;
s303, obtaining attention of the image block channel, and building 4 basically same attention modules;
s304, acquiring multi-scale information of the pathological image, wherein the pathological image consists of 6 pyramid convolution modules;
s305, acquiring pathological image information, wherein the pathological image information consists of 3 pyramid convolution modules;
s306, the preamble feature set passes through the self-adaptive average pooling and full-connection layer at this stage and is finally transmitted to the output, the number of output nodes is the number of types of molecular classification, and the output result is the prediction probability of each type.
3. The method for predicting the molecular typing of the cancer with multi-scale fusion attention according to claim 2, wherein: in S302, each module is connected in series by a combination of three convolution layers and a batch normalization layer, and is output through a ReLU, where the second convolution layer is pyramid convolution, and the difference between the three pyramid convolution modules is only that the first module adds a combination of a maximum pooling layer, a convolution layer, and a batch normalization layer at the end to perform downsampling, all pyramid convolutions in this stage are parallel to convolution kernels of four sizes, the sizes of the convolution kernels are 3 × 3, 5 × 5, 7 × 7, and 9 × 9, respectively, to form a pyramid form, and after the convolution of the feature set and each convolution kernel, the generated features are spliced, and the final output size of the stage is a feature set of 256 length, 56 width, and 56 height.
4. The method for predicting the molecular typing of the cancer with multi-scale fusion attention according to claim 2, wherein: in the step S303, only the last module is provided with a group of maximum pooling layer, convolutional layer and batch normalization layer for down-sampling, the attention module embeds the SE module into a residual module of ResNet-50 to form a SE-ResNet module, the module is based on the residual module, before jump connection, the spatial dimension of a preamble feature set is compressed into a real number as a channel descriptor through global pooling, and each channel weight is generated through operation of a full connection layer and mapped to each channel of the feature, and the output size of the stage is a feature set with a length of 512, a width of 28 and a height of 28.
5. The method for predicting the molecular typing of the cancer with multi-scale fusion attention according to claim 2, wherein: in S304, the parallel convolution kernels in each pyramid convolution have two sizes, 3 × 3 and 5 × 5 respectively, and the output size at this stage is a feature set with a length of 1024, a width of 14, and a height of 14.
6. The method for predicting the molecular typing of the cancer with multi-scale fusion attention according to claim 2, wherein: in S305, the difference from S302 is 3 pyramid convolution modules in this stage, and only one convolution kernel with a size of 3 × 3 is left in each pyramid convolution, and since only one convolution kernel with a size remains in the pyramid convolution, the actual pyramid convolution is reduced to a normal convolution layer with a convolution kernel of 3 × 3, and the output size in this stage is 2048,7, and 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210543763.5A CN114841979A (en) | 2022-05-18 | 2022-05-18 | Multi-scale attention-fused deep learning cancer molecular typing prediction method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210543763.5A CN114841979A (en) | 2022-05-18 | 2022-05-18 | Multi-scale attention-fused deep learning cancer molecular typing prediction method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114841979A true CN114841979A (en) | 2022-08-02 |
Family
ID=82569274
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210543763.5A Pending CN114841979A (en) | 2022-05-18 | 2022-05-18 | Multi-scale attention-fused deep learning cancer molecular typing prediction method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114841979A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115082743A (en) * | 2022-08-16 | 2022-09-20 | 之江实验室 | Full-field digital pathological image classification system considering tumor microenvironment and construction method |
-
2022
- 2022-05-18 CN CN202210543763.5A patent/CN114841979A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115082743A (en) * | 2022-08-16 | 2022-09-20 | 之江实验室 | Full-field digital pathological image classification system considering tumor microenvironment and construction method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110189334B (en) | Medical image segmentation method of residual error type full convolution neural network based on attention mechanism | |
| CN112489061B (en) | Deep learning intestinal polyp segmentation method based on multi-scale information and parallel attention mechanism | |
| CN112418329B (en) | Cervical OCT image classification method and system based on multi-scale textural feature fusion | |
| CN110021425B (en) | Comparison detector, construction method thereof and cervical cancer cell detection method | |
| CN107247971B (en) | Intelligent analysis method and system for ultrasonic thyroid nodule risk index | |
| CN112241766A (en) | Liver CT image multi-lesion classification method based on sample generation and transfer learning | |
| CN108492271A (en) | A kind of automated graphics enhancing system and method for fusion multi-scale information | |
| CN110321968B (en) | Ultrasonic image classification device | |
| CN112446891A (en) | Medical image segmentation method based on U-Net network brain glioma | |
| CN110991254B (en) | Ultrasonic image video classification prediction method and system | |
| CN111524144A (en) | Intelligent pulmonary nodule diagnosis method based on GAN and Unet network | |
| CN112348830B (en) | Multi-organ segmentation method based on improved 3D U-Net | |
| CN111932512B (en) | Intracranial hemorrhage detection method applied to CT image based on CNN and NLSTM neural network | |
| CN114549469A (en) | Deep neural network medical image diagnosis method based on confidence degree calibration | |
| CN111260639A (en) | Multi-view information-collaborative breast benign and malignant tumor classification method | |
| CN111047606A (en) | Pathological full-section image segmentation algorithm based on cascade thought | |
| CN113269799A (en) | Cervical cell segmentation method based on deep learning | |
| CN114445356A (en) | Multi-resolution-based full-field pathological section image tumor rapid positioning method | |
| CN115294075A (en) | OCTA image retinal vessel segmentation method based on attention mechanism | |
| WO2021139351A1 (en) | Image segmentation method, apparatus, medium, and electronic device | |
| CN114912486A (en) | Modulation mode intelligent identification method based on lightweight network | |
| CN114841979A (en) | Multi-scale attention-fused deep learning cancer molecular typing prediction method | |
| CN112396605A (en) | Network training method and device, image recognition method and electronic equipment | |
| CN115100165A (en) | Colorectal cancer T staging method and system based on tumor region CT image | |
| CN116524253A (en) | Thyroid cancer pathological image classification method based on lightweight transducer |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |