CN113313680A - Colorectal cancer pathological image prognosis auxiliary prediction method and system - Google Patents
Colorectal cancer pathological image prognosis auxiliary prediction method and system Download PDFInfo
- Publication number
- CN113313680A CN113313680A CN202110563360.2A CN202110563360A CN113313680A CN 113313680 A CN113313680 A CN 113313680A CN 202110563360 A CN202110563360 A CN 202110563360A CN 113313680 A CN113313680 A CN 113313680A
- Authority
- CN
- China
- Prior art keywords
- image
- colorectal cancer
- survival time
- small
- clustering
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30028—Colon; Small intestine
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medical Informatics (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Public Health (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Databases & Information Systems (AREA)
- Pathology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Epidemiology (AREA)
- Primary Health Care (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a colorectal cancer pathological image prognosis auxiliary prediction method and a colorectal cancer pathological image prognosis auxiliary prediction system, wherein the method comprises the following steps: background separation step: dividing the image into a background area and a tissue area; image small block segmentation step: segmenting the tissue region into small image blocks according to the size of a preset pixel; depth feature extraction: extracting the characteristics of the segmented image small blocks based on a convolutional layer and a pooling layer of a colorectal cancer survival time prediction model to obtain the characteristics of the image small blocks, and encoding the image small blocks into a one-dimensional array form after the convolutional layer processing; clustering: clustering small image features based on K-means clustering, and dividing tumor epithelial tissues, interstitium, mucus, normal tissues and necrotic parts; risk classification step: and dividing the risk range. According to the method, the image small blocks of the whole pathological image are automatically classified through K-means clustering, and then different types of image small blocks are selected for training together, so that the survival time of the patient with the colorectal cancer patient prognosis is predicted more accurately.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a colorectal cancer pathological image prognosis auxiliary prediction method and system.
Background
The existing colorectal cancer prognosis analysis mainly needs the pathological TNM stage of a colorectal cancer patient by a doctor, the survival condition of the patient within 5 years is determined from the stage 1-4, the TNM mainly comprises the tumor infiltration condition, the number of lymph nodes and whether the lymph nodes are transferred to other parts, the TNM stage is continuously developed, the research on the tumor environment in pathological sections at present shows that a lot of information related to prognosis is contained in the pathology, and a large amount of data is waiting to be mined.
In the prior art, a method for predicting prognosis of a pathological image based on deep learning requires an experienced pathologist to delineate a region of interest (ROI) in the pathological image, usually a tumor portion of the pathological image, and then to cut out a small image block from the region of interest, wherein the small image block represents the patient, and a label such as survival time is given to the patient, and the small image block enters a deep convolutional neural network for training to obtain a deep convolutional neural network model, so as to predict prognosis of the pathological image, wherein the deep convolutional neural network comprises 3 convolutional layers, 2 maximal pooling layers and 1 full-link layer.
However, the prognosis prediction of pathological images by medical imaging equipment still has the following problems: (1) after a pathology doctor of a pathology department obtains a pathological image of colorectal cancer, the pathology doctor needs to be experienced to draw a tumor area, and when the medical image is exponentially increased, the manual processing speed is low, the efficiency is low, and missed diagnosis happens. (2) Prognosis prediction is performed only from image patches of the tumor region, lacking sufficient accuracy.
Disclosure of Invention
In order to overcome the defects and shortcomings of the prior art, the invention provides an auxiliary colorectal cancer pathological image prognosis prediction method, which is used for assisting a doctor in predicting the survival time of a patient on a colorectal cancer pathological image, providing auxiliary information of a risk range and reducing the burden of the doctor.
The second purpose of the invention is to provide a colorectal cancer pathological image prognosis auxiliary prediction system.
A third object of the present invention is to provide a computer-readable storage medium.
It is a fourth object of the invention to provide a computer apparatus.
In order to achieve the first purpose, the invention adopts the following technical scheme:
the invention provides a colorectal cancer pathological image prognosis auxiliary prediction method, which comprises the following steps:
background separation step: separating a tissue area and a background area of the pathological image, converting an RGB space into an HSV color space, and performing automatic threshold segmentation on a saturation channel part in the HSV color space to divide the saturation channel part into the background area and the tissue area;
image small block segmentation step: segmenting the tissue region into small image blocks according to the size of a preset pixel;
depth feature extraction: extracting the characteristics of the segmented image small blocks based on a convolutional layer and a pooling layer of a colorectal cancer survival time prediction model to obtain the characteristics of the image small blocks, and encoding the image small blocks into a one-dimensional array form after the convolutional layer processing;
clustering: clustering small image features based on K-means clustering, and dividing tumor epithelial tissues, interstitium, mucus, normal tissues and necrotic parts;
risk classification step: outputting the survival time of the patient based on a full-connection layer of the colorectal cancer survival time prediction model, setting a plurality of preset risk thresholds, and dividing a risk range according to the survival time of the patient and the preset risk thresholds;
the colorectal cancer survival time prediction model is obtained by using multiple groups of data through machine learning training, and each group of data in the multiple groups of data comprises a colorectal cancer pathological image and label information for identifying the survival time of a patient corresponding to the colorectal cancer pathological image.
As a preferred technical solution, the clustering step specifically includes the following steps:
a central point generating step: randomly generating a central point of the image small block characteristic;
image small block classification step: calculating Euclidean distance between each image small block and a first central point, and dividing each image small block into a small block category set of the closest central point according to the Euclidean distance between the characteristics of the image small blocks and each central point;
and (4) classification and inspection: after the Euclidean distances between all the image small blocks and the first central point are calculated and the small block category sets of the first central point are divided, recalculating a second central point of each small block category set, and judging the clustering progress according to a preset distance threshold;
if the distance between the second central point and the previous first central point is smaller than a preset distance threshold, the classification result tends to be stable, and the clustering step is finished;
if the distance change between the new central point and the original central point exceeds a preset distance threshold, the clustering is still in progress, and the image small block classification step and the classification checking step need to be repeated.
As a preferred technical solution, in the risk classification step, the colorectal cancer survival time prediction model is specifically trained by using a VGG16 deep convolutional network, the VGG16 deep convolutional network takes pathological images of colorectal cancer as input data, quantifies the patient survival time for colorectal cancer prognosis and uses the patient survival time as a label, the patient survival time is subjected to convolution processing by a convolutional layer and then transmitted to a pooling layer to reduce the size of a feature map, finally, the patient survival time is output through a full-link layer, a plurality of preset risk thresholds are set, and a risk range is classified according to the patient survival time and the preset risk thresholds.
As a preferred technical scheme, the VGG16 deep convolutional network takes a pathological image of colorectal cancer as input data, after small-block features of the image are obtained, the small-block features of the image are respectively input into a convolutional layer part of the VGG16 deep convolutional network to extract features, the features comprise a plurality of convolutional layers and a plurality of full-link layers, finally, the features of all channels are combined together, the full-link layers are classified, and finally, the obtained image features are output.
In order to achieve the second object, the invention adopts the following technical scheme:
a colorectal cancer pathological image prognosis auxiliary prediction system comprises: the system comprises a background separation module, an image small block segmentation module, a depth feature extraction module, a clustering module and a risk division module;
the background separation module is used for dividing the input pathological image of the colorectal cancer into a tissue area and a background area;
the image small block segmentation module is used for segmenting the pathological image into small image blocks and segmenting the small image blocks in the tissue region according to the preset pixel size;
the depth feature extraction module is used for extracting features of the segmented image small blocks according to a convolutional layer and a pooling layer of the colorectal cancer survival time prediction model to obtain image small block features, and encoding the image small blocks into a one-dimensional array form after convolutional layer processing;
the clustering module is used for clustering the small image features according to K-means clustering and dividing tumor epithelial tissues, interstitium, mucus, normal tissues and necrotic parts;
the risk division module is used for outputting the survival time of the patient according to the full-connection layer of the colorectal cancer survival time prediction model, setting a plurality of preset risk thresholds and dividing the risk range according to the survival time of the patient and the preset risk thresholds.
The colorectal cancer survival time prediction model is composed of a convolutional layer, a pooling layer and a full-link layer, receives pathological images of colorectal cancer as input, performs convolution processing through the convolutional layer, transmits the pathological images to the pooling layer to reduce the size of a characteristic map, and finally outputs the survival time of a patient through the full-link layer.
In order to achieve the third object, the invention adopts the following technical scheme:
a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the above-mentioned colorectal cancer pathology image prognosis auxiliary prediction method.
In order to achieve the fourth object, the invention adopts the following technical scheme:
a computer device comprising a memory for storing a computer program and a processor; the processor is used for realizing the colorectal cancer pathological image prognosis auxiliary prediction method when executing the computer program.
Compared with the prior art, the invention has the following advantages and beneficial effects:
(1) according to the invention, the image small block features are clustered through K-means clustering, and the tumor epithelial tissue, the stroma, the mucus, the normal tissue and the necrotic part are divided, so that the ROI is automatically sketched without an experienced pathologist, the time of the doctor is saved, and the burden of the doctor is reduced.
(2) According to the method, background cut-out image small blocks are automatically removed from pathological images, deep learning characteristics are automatically extracted and classified, pathological tumor epithelium, mucus, necrotic areas, normal tissues and tumor stroma parts are simulated to a certain extent, image small blocks of a plurality of areas are integrated instead of images of only tumor areas to train a colorectal cancer survival time prediction model, the image small blocks of the pathological image tumor areas are not only used during deep learning, but the image small blocks of the whole pathological image are automatically classified through K-means clustering and then different types of image small blocks are selected to train together, so that a network can learn the relation among the different types of small blocks, and the survival time prediction of patients with colorectal cancer prognosis is more accurate.
(3) According to the invention, through a visual interactive interface, the operation and use of doctors are facilitated, and when the prognosis of a colorectal cancer patient is predicted, the survival time of the patient is predicted according to the colorectal cancer survival time prediction model, so that auxiliary information of risk assessment is provided, and the doctors are assisted to make a personalized treatment scheme for the patient.
Drawings
FIG. 1 is a flowchart illustrating the steps of a method for prognosis-assisted prediction of pathological images of colorectal cancer in example 1 of the present invention;
FIG. 2 is a schematic diagram of pathological images of colorectal cancer in example 1 of the present invention;
FIG. 3 is a schematic diagram of background processing of pathological images of colorectal cancer according to example 1 of the present invention;
FIG. 4 is a schematic diagram of the colorectal cancer pathology image being subjected to segmentation processing in embodiment 1 of the present invention;
FIG. 5 is a schematic diagram of the colorectal cancer pathology image subjected to K-means clustering processing in example 1 of the present invention;
FIG. 6 is a schematic diagram of a model for predicting survival time of colorectal cancer according to example 1 of the present invention;
fig. 7(a) is a first state diagram of a visualization interface in embodiment 2 of the present invention;
fig. 7(b) is a second state diagram of the visual interface in embodiment 2 of the present invention;
fig. 7(c) is a third state diagram of the visual interface in embodiment 2 of the present invention;
fig. 7(d) is a fourth state diagram of the visual interface in embodiment 2 of the present invention;
fig. 8 is a schematic diagram of the colorectal cancer pathology image prognosis auxiliary prediction system in embodiment 3 of the present invention.
Detailed Description
In the description of the present disclosure, it should be noted that the terms "first", "second", and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. Also, the use of the terms "a," "an," or "the" and similar referents do not denote a limitation of quantity, but rather denote the presence of at least one. The word "comprising" or "comprises", and the like, means that the element or item appearing before the word covers the element or item listed after the word and its equivalents, but does not exclude other elements or items. The terms "connected" or "coupled" and the like are not restricted to physical or mechanical connections, but may include electrical connections, whether direct or indirect.
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Example 1
As shown in fig. 1, the present embodiment provides a method for auxiliary prognosis prediction of pathological image of colorectal cancer, which includes the following steps:
background separation step: as shown in fig. 2, the pathological image is divided into a tissue region and a background region, the tissue region and the background region of the pathological image are separated first, the pathological image is obtained, the picture format of the pathological image is converted from RGB space to HSV color space, the saturation channel portion in the HSV color space is subjected to automatic threshold segmentation, and the pathological image is divided into the background region and the tissue region. In practical application, the acquired pathological image default picture format is an RGB space, wherein the RGB space includes red (R), green (G), and blue (B), and when the pathological image is converted into an HSV (Hue, Saturation, Value) color space, wherein the HSV color space includes Hue (H), Saturation (S), and lightness (V), and since the background is white and the Saturation is the highest white, only the Saturation channel portion of the pathological image is extracted for automatic threshold segmentation, specifically as shown in fig. 3, the Saturation channel portion is close to white when being greater than a threshold, the background region is a background region, the Saturation channel portion is not close to white when being less than the threshold, and the background region is a tissue region;
image small block segmentation step: as shown in fig. 4, the pathological image is divided into small image blocks, the tissue region is a yellow portion, the background region is a dark purple portion, and the small image blocks are divided into the tissue region according to a preset pixel size. In practical applications, the preset pixel size is set to 512 pixels by 512 pixels, that is, an average pathological image can be divided into 300 image blocks.
Depth feature extraction: and performing feature extraction on the segmented image small blocks based on the convolution layer and the pooling layer of the colorectal cancer survival time prediction model to obtain image small block features, and encoding the image small blocks into a one-dimensional array form after feature tiling. In practical application, the colorectal cancer survival time prediction model is a trained VGG16 network, and a backward propagation mechanism is utilized to enable the colorectal cancer survival time prediction model to learn a proper value and extract proper features, wherein the pooling layer comprises an average pooling layer and a maximum pooling layer which are respectively used for solving the average value and the maximum value in a pooling matrix area, so that the purpose of reducing the size of a feature map is achieved. Because the deep neural network in the embodiment adopts the maximum pooling layer, in the feature information extracted after the convolution operation, the adjacent regions have similar feature information, if the feature information is completely retained, information redundancy may exist, the calculation difficulty is increased, and after the processing, the image small blocks are directly tiled into an array of 1 × 25088 from an array of 512 × 7, so that the unique deep learning feature of each pathological image small block is obtained.
Clustering: as shown in fig. 5, the image patch features are clustered based on K-means clustering, and tumor epithelial tissue, stroma, mucus, normal tissue, and necrotic parts are divided. In practical application, after an experienced pathologist outlines a pathological image ROI, the pathological image ROI can be theoretically divided into 5 types, namely, tumor epithelial tissues, stroma, mucus, normal tissues and necrotic parts, so that the characteristics are subjected to K-means clustering (K mean value clustering processing) to be divided into 5 types.
Risk classification step: the patient survival time is output based on a full-connection layer of the colorectal cancer survival time prediction model, a plurality of preset risk thresholds are set, and a risk range is divided according to the patient survival time and the preset risk thresholds.
In this embodiment, the model for predicting colorectal cancer survival time is obtained by machine learning training using a plurality of sets of data, each set of data in the plurality of sets of data includes a colorectal cancer pathology image and label information for identifying the survival time of a patient corresponding to the colorectal cancer pathology image.
In this embodiment, the clustering step specifically includes the following steps:
a central point generating step: randomly generating 5 points as the central points of the image small block characteristics;
image small block classification step: and calculating Euclidean distances between each image small block and 5 central points, and dividing each image small block into a small block category set of the closest central point according to the Euclidean distance between the characteristics of the image small blocks and each central point. In practical application, the feature of the image patch is separated from which first central point by the minimum Euclidean distance, and the image patch is divided into a patch category set of the first central point.
And (4) classification and inspection: after the Euclidean distances between all the image small blocks and the first central point are calculated and the small block category sets of the first central point are divided, recalculating a second central point of each small block category set, and judging the clustering progress according to a preset distance threshold;
if the distance between the second central point and the previous first central point is smaller than a preset distance threshold, the classification result tends to be stable, at this moment, the clustering can be considered to reach the expected result, the clustering step is finished, and the calculation is finished;
if the distance change between the new central point and the original central point exceeds a preset distance threshold, the clustering is still in progress, and the image small block classification step and the classification checking step need to be repeated.
With reference to fig. 5, the pathological image is divided into small image blocks, each type is endowed with a unique color after the pathological image is subjected to K-means clustering, and the color is set at the position of an original image.
In this embodiment, the colorectal cancer survival time prediction model is trained by specifically adopting a VGG16 deep convolutional network, the VGG16 deep convolutional network specifically includes 13 convolutional layers, 5 pooling layers and 3 full-link layers, a pathological image of colorectal cancer is used as input data, the survival time of a patient with colorectal cancer prognosis is quantified and used as a label, after the convolutional layer is subjected to convolutional processing, the survival time is transmitted to the pooling layers to reduce the size of a feature map, finally, the survival time of the patient is output through the full-link layers, a plurality of preset risk thresholds are set, and a risk range is divided according to the survival time of the patient and the preset risk thresholds, so that a doctor is assisted in predicting the colorectal cancer prognosis.
In practical application, in the convolutional layer, the convolution calculation is performed through a convolution kernel with the size of 3 × 3, the input pathological image of the colorectal cancer is scanned according to the step length (default to 1), the numerical value of the scanned area and the value of the corresponding position of the convolution kernel are multiplied, summed, and the obtained values form a new matrix, and finally, a proper value is learned through a back propagation mechanism of a neural network, so that proper features are extracted.
As shown in fig. 6, after 5 different small block category sets are obtained, features are extracted from convolutional layers of the VGG16 model, 25088 features are obtained from each path, finally, the features of 5 channels are combined together, that is, the features are combined to obtain 125440 features, the whole convolutional neural network is used as a classifier through a self-defined full connection layer, and the purpose of transfer learning is achieved. Wherein the VGG16 model is provided with 13 convolutional layers and 3 full-link layers.
In addition, the deep convolutional network adopted by the colorectal cancer survival time prediction model in the embodiment is not limited, and those skilled in the art may substitute other deep convolutional networks according to actual situations.
Example 2
Through the above description of the embodiments, those skilled in the art will clearly understand that the present invention may be implemented by software plus a necessary hardware platform, and certainly may be implemented by hardware, but in many cases, the former is a better embodiment. With this understanding in mind, all or part of the technical solutions of the present invention that contribute to the background can be embodied in the form of a software product, which can be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of the embodiments or the parts of the embodiments.
The present embodiment provides a storage medium, which may be a storage medium such as ROM, RAM, magnetic disk, optical disk, etc., and the storage medium stores one or more programs, and when the programs are executed by a processor, the method for auxiliary prognosis prediction of pathological image of colorectal cancer according to embodiment 1 above is implemented.
The embodiment also provides a computing device, which may be a desktop computer, a notebook computer, a smart phone, a PDA handheld terminal, a tablet computer, or other terminal devices with a display function, and the computing device includes a processor and a memory, where the memory stores one or more programs, and when the processor executes the programs stored in the memory, the method for auxiliary prediction of colorectal cancer pathological image prognosis according to embodiment 1 above is implemented.
Specifically, as shown in fig. 7(a), after the colorectal cancer survival time prediction model is trained, a convenient operation mode is provided for a user through a visual interactive interface based on a browser or client software, wherein the visual interactive interface comprises image preprocessing, risk calculation, a progress display frame and a picture display area.
As shown in fig. 7(b), after inputting a new pathological image path of a colorectal cancer patient, i.e., acquiring input data, displaying a pathological image of the patient by clicking a start preprocessing button and performing the background separation step in embodiment 1.
As shown in fig. 7(c), the image patch segmentation step, the depth feature extraction step, and the clustering step in embodiment 1 are executed by clicking the display processing result, and the patch category sets and the restored results are displayed, so that in actual application, all the image patches segmented by the original image are divided into different color categories, and the colors represent different categories, thereby differentiating the patch category sets. And (4) returning the classified image small blocks to the original positions in the graph, representing different image small block categories by using different colors, and providing a visual classification result to realize restoration.
As shown in FIG. 7(d), the calculation of the five-year survival risk of colorectal cancer of the patient was started by inputting the model path of the model for predicting colorectal cancer survival time after training in example 1, dividing a risk range according to the survival time of the patient and combining a preset risk threshold value, thereby displaying a risk giving suggestion on the lower left side, and when the method is actually applied, the risk range comprises a low risk range, a medium risk range and a high risk range, specifically, 0-0.33 is the low risk range, can be directly operated to remove tumor with low recurrence rate, 0.33-0.66 is the risk range of apoplexy, besides the operation, the comprehensive treatment means such as chemotherapy and the like are adopted to check the disease condition and prevent acute recurrence, 0.66-1 is a high risk range, the 5-year survival probability of the patients is low, the surgical risks can hardly be borne, and conservative treatment should be adopted.
In addition, in this embodiment, for example, one of the development environments is used, the operating system adopts a linux system, the CPU adopts I7-9800X, the GPU adopts RTX 2080TI, the hard disk adopts SSD 1T, and the development framework is based on pycharm and matlab; the development language adopts matlab and python 3.6.7; in addition, the hardware environment adopted in the present application is not limited to this, and those skilled in the art may change the development environment according to actual situations.
Example 3
As shown in fig. 8, the present embodiment provides a colorectal cancer pathological image prognosis auxiliary prediction system, including: the system comprises a background separation module, an image small block segmentation module, a depth feature extraction module, a clustering module and a risk division module;
the background separation module is used for dividing the input pathological image of the colorectal cancer into a tissue area and a background area;
the image small block segmentation module is used for segmenting the pathological image into small image blocks and segmenting the small image blocks in the tissue region according to the preset pixel size;
the depth feature extraction module is used for extracting features of the segmented image small blocks according to the convolutional layer and the pooling layer of the colorectal cancer survival time prediction model to obtain image small block features, and encoding the image small blocks into a one-dimensional array form after the image small blocks are processed by the convolutional layer;
the clustering module is used for clustering the small image features according to the K-means clustering and dividing tumor epithelial tissues, interstitium, mucus, normal tissues and necrotic parts;
the risk division module is used for outputting the survival time of the patient according to the full-connection layer of the colorectal cancer survival time prediction model, setting a plurality of preset risk thresholds and dividing a risk range according to the survival time of the patient and the preset risk thresholds;
in this embodiment, the colorectal cancer survival time prediction model is composed of a convolutional layer, a pooling layer and a full-link layer, receives a pathological image of colorectal cancer as an input, performs convolution processing through the convolutional layer, transmits the pathological image to the pooling layer to reduce the size of a feature map, and finally outputs the survival time of a patient through the full-link layer.
The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.
Claims (8)
1. The colorectal cancer pathological image prognosis auxiliary prediction method is characterized by comprising the following steps of:
background separation step: separating a tissue area and a background area of the pathological image, converting an RGB space into an HSV color space, and performing automatic threshold segmentation on a saturation channel part in the HSV color space to divide the saturation channel part into the background area and the tissue area;
image small block segmentation step: segmenting the tissue region into small image blocks according to the size of a preset pixel;
depth feature extraction: extracting the characteristics of the segmented image small blocks based on a convolutional layer and a pooling layer of a colorectal cancer survival time prediction model to obtain the characteristics of the image small blocks, and encoding the image small blocks into a one-dimensional array form after the convolutional layer processing;
clustering: clustering small image features based on K-means clustering, and dividing tumor epithelial tissues, interstitium, mucus, normal tissues and necrotic parts;
risk classification step: outputting the survival time of the patient based on a full-connection layer of the colorectal cancer survival time prediction model, setting a plurality of preset risk thresholds, and dividing a risk range according to the survival time of the patient and the preset risk thresholds;
the colorectal cancer survival time prediction model is obtained by using multiple groups of data through machine learning training, and each group of data in the multiple groups of data comprises a colorectal cancer pathological image and label information for identifying the survival time of a patient corresponding to the colorectal cancer pathological image.
2. The colorectal cancer pathology image prognosis auxiliary prediction method according to claim 1, wherein the clustering step specifically comprises the following steps:
a central point generating step: randomly generating a central point of the image small block characteristic;
image small block classification step: calculating Euclidean distance between each image small block and a first central point, and dividing each image small block into a small block category set of the closest central point according to the Euclidean distance between the characteristics of the image small blocks and each central point;
and (4) classification and inspection: after the Euclidean distances between all the image small blocks and the first central point are calculated and the small block category sets of the first central point are divided, recalculating a second central point of each small block category set, and judging the clustering progress according to a preset distance threshold;
if the distance between the second central point and the previous first central point is smaller than a preset distance threshold, the classification result tends to be stable, and the clustering step is finished;
if the distance change between the new central point and the original central point exceeds a preset distance threshold, the clustering is still in progress, and the image small block classification step and the classification checking step need to be repeated.
3. The colorectal cancer pathological image prognosis auxiliary prediction method according to claim 1, wherein in the risk classification step, the colorectal cancer survival time prediction model is specifically trained by using a VGG16 deep convolution network, the VGG16 deep convolution network takes a colorectal cancer pathological image as input data, quantifies and takes the colorectal cancer prognosis patient survival time as a label, performs convolution processing on the convolution layer, transmits the label to a pooling layer to reduce the size of a feature map, finally outputs the patient survival time through a full connection layer, sets a plurality of preset risk thresholds, and classifies a risk range according to the patient survival time and the preset risk thresholds.
4. The colorectal cancer pathological image prognosis auxiliary prediction method according to claim 3, wherein the VGG16 deep convolution network takes a pathological image of colorectal cancer as input data, after small image block features are obtained, the small image block features are respectively input into a convolution layer part of the VGG16 deep convolution network to extract features, the features comprise a plurality of convolution layers and a plurality of full connection layers, finally, the features of all channels are combined together, the full connection layers are classified, and finally obtained image features are output.
5. An auxiliary prognosis prediction system for pathological images of colorectal cancer is characterized by comprising: the system comprises a background separation module, an image small block segmentation module, a depth feature extraction module, a clustering module and a risk division module;
the background separation module is used for dividing the input pathological image of the colorectal cancer into a tissue area and a background area;
the image small block segmentation module is used for segmenting the pathological image into small image blocks and segmenting the small image blocks in the tissue region according to the preset pixel size;
the depth feature extraction module is used for extracting features of the segmented image small blocks according to a convolutional layer and a pooling layer of the colorectal cancer survival time prediction model to obtain image small block features, and encoding the image small blocks into a one-dimensional array form after convolutional layer processing;
the clustering module is used for clustering the small image features according to K-means clustering and dividing tumor epithelial tissues, interstitium, mucus, normal tissues and necrotic parts;
the risk division module is used for outputting the survival time of the patient according to the full-connection layer of the colorectal cancer survival time prediction model, setting a plurality of preset risk thresholds and dividing the risk range according to the survival time of the patient and the preset risk thresholds.
6. The colorectal cancer pathological image prognosis auxiliary prediction system according to claim 5, wherein the colorectal cancer survival time prediction model is composed of a convolutional layer, a pooling layer and a full-link layer, the colorectal cancer survival time prediction model receives the pathological image of colorectal cancer as input, carries out convolution processing through the convolutional layer, transmits to the pooling layer to reduce the size of a feature map, and finally outputs the survival time of the patient through the full-link layer.
7. A computer-readable storage medium, characterized in that the storage medium has stored thereon a computer program which, when being executed by a processor, implements the colorectal cancer pathology image prognosis auxiliary prediction method according to any one of claims 1-4.
8. A computer device comprising a memory and a processor, the memory for storing a computer program;
the processor, when executing the computer program, is configured to implement the method of aided prognosis prediction of pathological image of colorectal cancer according to any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110563360.2A CN113313680B (en) | 2021-05-24 | 2021-05-24 | Colorectal cancer pathological image prognosis auxiliary prediction method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110563360.2A CN113313680B (en) | 2021-05-24 | 2021-05-24 | Colorectal cancer pathological image prognosis auxiliary prediction method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113313680A true CN113313680A (en) | 2021-08-27 |
| CN113313680B CN113313680B (en) | 2023-06-23 |
Family
ID=77374354
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110563360.2A Active CN113313680B (en) | 2021-05-24 | 2021-05-24 | Colorectal cancer pathological image prognosis auxiliary prediction method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113313680B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113782191A (en) * | 2021-09-26 | 2021-12-10 | 萱闱(北京)生物科技有限公司 | Colorectal lesion type prediction device, model construction method, medium, and apparatus |
| CN113947607A (en) * | 2021-09-29 | 2022-01-18 | 电子科技大学 | Cancer pathology image survival prognosis model construction method based on deep learning |
| TWI827043B (en) * | 2022-05-10 | 2023-12-21 | 中山醫學大學 | A method to establish a clinical decision support system for second primary cancers in colorectal cancer using predictive models and visualization methods |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200058125A1 (en) * | 2018-08-14 | 2020-02-20 | Mikhail Teverovskiy | Comparative cancer survival models to assist physicians to choose optimal treatment |
| US10650520B1 (en) * | 2017-06-06 | 2020-05-12 | PathAI, Inc. | Systems and methods for training a statistical model to predict tissue characteristics for a pathology image |
| CN111370128A (en) * | 2020-03-05 | 2020-07-03 | 上海市肺科医院（上海市职业病防治院） | Lung cancer patient prognosis prediction system and method |
| CN112309576A (en) * | 2020-09-22 | 2021-02-02 | 江南大学 | Colorectal cancer survival period prediction method based on deep learning CT (computed tomography) image omics |
| WO2021062904A1 (en) * | 2019-09-30 | 2021-04-08 | 中国科学院计算技术研究所 | Tmb classification method and system based on pathological image, and tmb analysis device based on pathological image |
-
2021
- 2021-05-24 CN CN202110563360.2A patent/CN113313680B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10650520B1 (en) * | 2017-06-06 | 2020-05-12 | PathAI, Inc. | Systems and methods for training a statistical model to predict tissue characteristics for a pathology image |
| US20200058125A1 (en) * | 2018-08-14 | 2020-02-20 | Mikhail Teverovskiy | Comparative cancer survival models to assist physicians to choose optimal treatment |
| WO2021062904A1 (en) * | 2019-09-30 | 2021-04-08 | 中国科学院计算技术研究所 | Tmb classification method and system based on pathological image, and tmb analysis device based on pathological image |
| CN111370128A (en) * | 2020-03-05 | 2020-07-03 | 上海市肺科医院（上海市职业病防治院） | Lung cancer patient prognosis prediction system and method |
| CN112309576A (en) * | 2020-09-22 | 2021-02-02 | 江南大学 | Colorectal cancer survival period prediction method based on deep learning CT (computed tomography) image omics |
Non-Patent Citations (4)
| Title |
|---|
| XINLIANG ZHU 等: "WSISA: Making Survival Prediction from Whole Slide Histopathological Images", 《2017 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 * |
| 孟濬 等: "基于变量聚类的 BP 神经网络术后生存期预测模型", 《浙 江 大 学 学 报(工学版)》 * |
| 徐立: "一种改进的K-Means 算法", 《河北软件职业技术学院学报》 * |
| 杨帆 等: "基于优化后的稀疏ｋ－ｍｅａｎｓ算法的心脏健康状况分类", 《内蒙古师范大学学报（自然科学汉文版）》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113782191A (en) * | 2021-09-26 | 2021-12-10 | 萱闱(北京)生物科技有限公司 | Colorectal lesion type prediction device, model construction method, medium, and apparatus |
| CN113782191B (en) * | 2021-09-26 | 2023-01-13 | 萱闱(北京)生物科技有限公司 | Colorectal lesion type prediction device, model construction method, medium, and device |
| CN113947607A (en) * | 2021-09-29 | 2022-01-18 | 电子科技大学 | Cancer pathology image survival prognosis model construction method based on deep learning |
| CN113947607B (en) * | 2021-09-29 | 2023-04-28 | 电子科技大学 | Cancer pathological image survival prognosis model construction method based on deep learning |
| TWI827043B (en) * | 2022-05-10 | 2023-12-21 | 中山醫學大學 | A method to establish a clinical decision support system for second primary cancers in colorectal cancer using predictive models and visualization methods |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113313680B (en) | 2023-06-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10580137B2 (en) | Systems and methods for detecting an indication of malignancy in a sequence of anatomical images | |
| Xie et al. | Skin lesion segmentation using high-resolution convolutional neural network | |
| US10482633B2 (en) | Systems and methods for automated detection of an indication of malignancy in a mammographic image | |
| US20220189142A1 (en) | Ai-based object classification method and apparatus, and medical imaging device and storage medium | |
| EP3553742A1 (en) | Method and device for identifying pathological picture | |
| CN113313680A (en) | Colorectal cancer pathological image prognosis auxiliary prediction method and system | |
| WO2020151307A1 (en) | Automatic lesion recognition method and device, and computer-readable storage medium | |
| US11967181B2 (en) | Method and device for retinal image recognition, electronic equipment, and storage medium | |
| WO2022088665A1 (en) | Lesion segmentation method and apparatus, and storage medium | |
| CN110689025A (en) | Image recognition method, device and system, and endoscope image recognition method and device | |
| KR102332088B1 (en) | Apparatus and method for polyp segmentation in colonoscopy images through polyp boundary aware using detailed upsampling encoder-decoder networks | |
| CN110570350A (en) | two-dimensional follicle detection method and device, ultrasonic equipment and readable storage medium | |
| WO2019184851A1 (en) | Image processing method and apparatus, and training method for neural network model | |
| CN113012173A (en) | Heart segmentation model and pathology classification model training, heart segmentation and pathology classification method and device based on cardiac MRI | |
| CN112132166A (en) | Intelligent analysis method, system and device for digital cytopathology image | |
| Al-Masni et al. | A deep learning model integrating FrCN and residual convolutional networks for skin lesion segmentation and classification | |
| CN115063592B (en) | Multi-scale-based full-scanning pathological feature fusion extraction method and system | |
| WO2023207743A1 (en) | Image detection method and apparatus, and computer device, storage medium and program product | |
| CN111916206A (en) | CT image auxiliary diagnosis system based on cascade connection | |
| Khattar et al. | Computer assisted diagnosis of skin cancer: a survey and future recommendations | |
| KR20220078495A (en) | Method, apparatus and program to read lesion of small intestine based on capsule endoscopy image | |
| CN107590806B (en) | Detection method and system based on brain medical imaging | |
| CN114283406A (en) | Cell image recognition method, device, equipment, medium and computer program product | |
| CN110992309B (en) | Fundus image segmentation method based on deep information transfer network | |
| CN112801238B (en) | Image classification method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |