CN114511588A - Method and device for judging benign and malignant breast tissue pathological image - Google Patents
Method and device for judging benign and malignant breast tissue pathological image Download PDFInfo
- Publication number
- CN114511588A CN114511588A CN202111599328.6A CN202111599328A CN114511588A CN 114511588 A CN114511588 A CN 114511588A CN 202111599328 A CN202111599328 A CN 202111599328A CN 114511588 A CN114511588 A CN 114511588A
- Authority
- CN
- China
- Prior art keywords
- image
- images
- block
- full
- breast
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 210000000481 breast Anatomy 0.000 title claims abstract description 66
- 238000000034 method Methods 0.000 title claims abstract description 61
- 230000001575 pathological effect Effects 0.000 title claims abstract description 41
- 230000003211 malignant effect Effects 0.000 title description 17
- 230000011218 segmentation Effects 0.000 claims abstract description 39
- 206010028980 Neoplasm Diseases 0.000 claims abstract description 26
- 230000007170 pathology Effects 0.000 claims abstract description 25
- 201000011510 cancer Diseases 0.000 claims abstract description 24
- 230000036210 malignancy Effects 0.000 claims abstract description 20
- 230000004927 fusion Effects 0.000 claims abstract description 8
- 238000012545 processing Methods 0.000 claims abstract description 7
- 238000009499 grossing Methods 0.000 claims abstract description 6
- 238000000638 solvent extraction Methods 0.000 claims abstract description 4
- 230000008569 process Effects 0.000 claims description 12
- 239000011159 matrix material Substances 0.000 claims description 10
- 238000001914 filtration Methods 0.000 claims description 8
- 238000005070 sampling Methods 0.000 claims description 7
- 238000003708 edge detection Methods 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 238000003062 neural network model Methods 0.000 claims description 4
- 238000004891 communication Methods 0.000 claims description 2
- 238000003759 clinical diagnosis Methods 0.000 abstract description 4
- 210000001519 tissue Anatomy 0.000 description 48
- 238000012549 training Methods 0.000 description 16
- 238000013527 convolutional neural network Methods 0.000 description 13
- 206010006187 Breast cancer Diseases 0.000 description 12
- 208000026310 Breast neoplasm Diseases 0.000 description 12
- 210000004027 cell Anatomy 0.000 description 11
- 238000000605 extraction Methods 0.000 description 9
- 238000004422 calculation algorithm Methods 0.000 description 8
- 238000002372 labelling Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000013145 classification model Methods 0.000 description 4
- 238000003745 diagnosis Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000013519 translation Methods 0.000 description 4
- 230000003321 amplification Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 238000003199 nucleic acid amplification method Methods 0.000 description 3
- 238000007637 random forest analysis Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 230000004083 survival effect Effects 0.000 description 3
- 239000011800 void material Substances 0.000 description 3
- 206010060862 Prostate cancer Diseases 0.000 description 2
- 208000000236 Prostatic Neoplasms Diseases 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 206010004146 Basal cell carcinoma Diseases 0.000 description 1
- 241001237259 Campanella <basidiomycete fungus> Species 0.000 description 1
- 201000009030 Carcinoma Diseases 0.000 description 1
- 206010009944 Colon cancer Diseases 0.000 description 1
- 208000001333 Colorectal Neoplasms Diseases 0.000 description 1
- 206010058467 Lung neoplasm malignant Diseases 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 238000012850 discrimination method Methods 0.000 description 1
- 238000001493 electron microscopy Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000008014 freezing Effects 0.000 description 1
- 238000007710 freezing Methods 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 201000005202 lung cancer Diseases 0.000 description 1
- 208000020816 lung neoplasm Diseases 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000010827 pathological analysis Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30068—Mammography; Breast
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Quality & Reliability (AREA)
- Radiology & Medical Imaging (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method for judging the quality and the malignancy of a breast tissue pathological image, which comprises the following steps: partitioning a breast tissue pathology full-section image to be processed to obtain a plurality of partitioned images; performing, separately for each of the plurality of block images: carrying out smoothing processing on the block image to obtain a smoothed image; performing threshold segmentation on each smoothed block image to obtain a block image with the background removed; performing edge segmentation on the block image with the background removed to obtain an image after edge segmentation; obtaining a plurality of images after edge segmentation, wherein each block image respectively corresponds to the image after edge segmentation; and inputting the images after the edge segmentation into a trained VGG19 network to obtain a binary classification result of fusion blocks, and judging the full-slice images to obtain a binary classification result of the full slice. By applying the embodiment of the invention, a pathologist is assisted to carry out clinical diagnosis.
Description
Technical Field
The invention relates to the technical field of medical image diagnosis, in particular to a method and a device for judging the quality and the malignancy of a breast tissue pathological image.
Background
Breast cancer is one of the most common malignancies among women, accounting for 23% of all malignancies women suffer, and statistics show that up to 1/8 women develop breast cancer at some time during their lifetime. In China, the number of the breast cancer attacks of women is 16.9 ten thousand every year (accounting for 12.25 percent of the total attacks of the whole world), about 4.5 ten thousand of people die, the incidence rate is the first of all female malignant tumors, the incidence rate is increased year by year, and the trend of the breast cancer is obvious. The survival rate of the breast cancer is greatly improved as long as the breast cancer is discovered, diagnosed and treated early, the survival rate of the early breast cancer exceeds 90 percent, but the 5-year survival rate of the late breast cancer is only 30 to 40 percent.
As the 'gold standard' for breast cancer diagnosis, in traditional clinical application, pathologists usually need to analyze the samples sent for each patient one by one to give pathological diagnosis reports, and the process fully tests the professional literacy of the pathologists and highly depends on the working experience of the pathologists. The pathological department is in a state of long talent culture period, large talent gap and overload operation for a long time, the method for assisting the pathological doctor in diagnosing by utilizing the modern information technology can relieve the pressure of the pathological department and improve the working efficiency of the pathological doctor.
The accurate classification of the breast tissue pathological images gradually shows extremely high application value in assisting diagnosis of doctors and meeting clinical application requirements. The early benign and malignant judgment of the breast pathological image depends on a classification algorithm based on artificial extraction features and traditional machine learning, the algorithm needs to manually design a feature extractor to extract the features of the pathological image to obtain high-quality global features, and the extracted features are used for classifying the original tissue pathological image by using a machine learning method (such as a support vector machine, a random forest and the like) to obtain a final result. In the algorithm, the manual design of the feature extractor requires professional field knowledge of relevant pathological experts, the feature extraction is complex, the features including color, texture, structure and the like are extracted from the original image and used as the input of the classifier, and finally the benign and malignant classification of the tissue pathological image is obtained. However, the process of manually designing the feature extractor is complex, time-consuming and labor-consuming, is greatly influenced by the subjectivity of a pathologist, and highly depends on the experience of the pathologist. In addition, the feature expression capability of manual extraction is limited, and the comprehensive information of the image cannot be covered, so that the extracted features are not representative, and the final classification effect is influenced. The deep learning algorithm based on data and model driving can automatically extract the abstract features of the images from the original images in a layering mode, and in the histopathology field, the deep convolutional neural network can be applied to the classification task of the histopathology images.
Deep convolutional neural networks have achieved excellent performance in the general class image classification task. With the development of electron microscopy, various histopathology image discrimination methods based on deep convolutional neural networks are applied to clinical diagnosis, help pathologists judge the quality and malignancy of cells of patients, and identification algorithms for histopathology images of cancers such as breast cancer, prostate cancer, lung cancer and colorectal cancer are developed at present. Since the convolutional neural network needs normalized image input, and the size of pathological images usually exceeds one billion pixels, which far exceeds the input requirement of any mainstream convolutional neural network, if the resolution of images is directly reduced by adopting a down-sampling mode, many morphological features in tissue images are lost. At present, researchers only need to cut the full-section histopathology image into blocks with acceptable sizes through a sliding window and a random network according to the input requirements of a convolutional neural network, feature extraction is firstly carried out on the blocks, then the features are integrated to obtain the features of the full-section, and finally the classification of the original histopathology image is obtained.
Currently, mainstream methods for performing benign and malignant determination on a full-slice pathological image include a block sampling method and a multi-example learning method. The block sampling method firstly blocks the full slice, then performs feature extraction on the blocks, and finally aggregates the features of the blocks to obtain the classification of the full slice image, and many scholars have made relevant work on the classification. For example, Wang et al randomly slices a camalyon 16 breast cancer sentinel node into 256 × 256 pixel patches, train google lenet to classify the patches, combine features of all patches to obtain a full-slice thermodynamic diagram, and finally classify the full-slice through a random forest. Xu et al used an improved Alexnet classification model, fine-tuned the pre-trained Alexnet model, then sent 224x224 blocks into the network to get 4096-dimensional feature vectors, and finally computed all block features by p-norm pooling to get full-slice binary results. Dhungel et al directionally extract image features through a feature learning process of an artificial canonical neural network using simple CNN, the performance of the directional features on a random forest classifier is superior to the same kind of features extracted artificially, and the network is directly utilized to make classification prediction on breast pathology images.
In the method, a large amount of manual labeling data is used for model training, and the pixel-level labeling of the histopathology image requires professional field knowledge of a pathologist. In practical clinical application, it is difficult and tedious to obtain high-precision pixel-level labeling, and even for experienced pathologists, there are some ambiguous areas, so the development and application of such classification models are restricted to some extent.
Pathological data sets existing in practical clinical application are all at a full-slice level, and based on the fact that a multi-example learning method is carried out at the same time, relevant scholars conduct research. In the pathology field, a full slice is regarded as a package, all blocks cut out from the full slice are regarded as examples in the package, and if the package is regarded as benign, all the examples in the package are regarded as benign; conversely, if a package is considered malignant, then at least one instance of the package should be considered malignant. Campanella et al propose a multi-example based weakly supervised model using only diagnostic reports as training data. The model is a large-scale classification framework without a pixel-level labeling data set, and can be used for judging the quality and the malignancy of breast cancer, prostate cancer and basal cell carcinoma full sections. Hou et al propose a method for maximizing an expected value, select a region with strong discrimination from the blocks of a full slice as a basis for full-slice image discrimination, optimize a decision fusion strategy, combine the classification results of the block levels, and obtain the benign and malignant discrimination results of the full slice by using the spatial relationship between the blocks. Zhu et al combines a multi-instance learning method with an Alexnet network, obtains a lesion benign and malignant probability distribution map of the whole breast through the AlexNet network and a patient breast original image, and obtains benign and malignant classification of a breast tissue pathological image from the probability map through multi-instance learning based on sparse expression.
The multi-example learning method solves the problem that the fully supervised learning model has high dependence on pixel level data labeling, and can realize classification of whether canceration exists or not only by using full-slice level labeling. The multi-instance learning strategy improves the generalization capability of the model to a certain extent.
In the clinic, the cancerous region of the tissue is usually in the cell matrix, while in the above methods the scholars do not distinguish between the cell matrix and the stroma and generate a large amount of redundant calculations during the model training process. The invention provides a method for distinguishing a breast pathological tissue image by two stages, wherein a cell stroma region which is more likely to become cancerous is extracted before distinguishing, and then good and malignant distinguishing is carried out on the stroma region. Through research of relevant documents and patent patents, research related to the invention does not appear in the existing method for judging the benign and malignant breast tissue pathological images.
Disclosure of Invention
The invention aims to provide a method and a device for judging the quality and the malignancy of a breast tissue pathological image, which can realize the end-to-end quality and malignancy judgment of the breast tissue pathological image, extract a matrix region in a full section by a machine learning method to obtain the matrix region of the section, send the matrix region into an improved VGG network model for quality and malignancy judgment, and realize the classification of whether a breast tissue section has canceration or not.
In order to achieve the above object, the present invention provides a method for discriminating the malignancy or benign of a breast tissue pathology image, including:
partitioning a breast tissue pathology full-section image to be processed to obtain a plurality of partitioned images;
performing, separately for each of the plurality of block images:
carrying out smoothing processing on the block image to obtain a smoothed image;
performing threshold segmentation on the smoothed image to obtain a block image with the background removed;
performing edge segmentation on the block image with the background removed to obtain an image after edge segmentation; obtaining a plurality of images after edge segmentation, wherein each block image respectively corresponds to the image after edge segmentation;
inputting the images after the edge segmentation into a trained neural network model to obtain two classification results of fusion blocks;
and judging the full-slice image based on the obtained two-classification result of the fusion block to obtain the two-classification result of the full slice.
In one implementation, the step of blocking the breast histopathology full-section image to be processed to obtain a plurality of block images includes:
reading a breast histopathology whole-section image by using an Openslide module;
and selecting a window with a preset size to slide along the preset direction of the breast tissue pathology full-section image in a fixed step length to obtain a section.
In one implementation, the step of smoothing each of the block images to obtain a smoothed image includes:
step 31: reading the blocks by using an OpenCV module, and sampling each block image by adopting an observation window with a preset radius;
step 32: sorting the pixel values obtained in the current observation window to obtain a median pixel value;
step 33: updating the pixel value in the center of the observation window by using the median pixel value;
steps 32 and 33 are repeated until each block image is subjected to the filtering process.
In one implementation, the step of performing threshold segmentation on the smoothed segmented image to obtain a background-removed image of the breast tissue pathology full-slice image includes:
for each of the smoothed block images, performing the steps of:
performing threshold segmentation by using an OTSU global threshold segmentation method, automatically selecting an initialization threshold based on the image, and dividing the image into a foreground image and a background image;
and removing a background image, and carrying out Gaussian denoising on the foreground image to obtain a background-removed image of the breast tissue pathology full-section image.
In one implementation, the step of performing edge segmentation on the background-removed block image to obtain an edge-segmented image includes:
step 51: selecting a group of seed pixels of the cell matrix area in each partitioned image after the background is removed, and acquiring the gray value of the seed pixels;
step 52: determining a threshold of difference value of gray value of the seed pixel;
step 53: calculating difference values of pixel gray values around the seed pixels and the seed pixels one by one, and if the difference values are smaller than a threshold, selecting the difference values as elements in a growth area; if the number of the detected signals is larger than the threshold, rejecting the signals;
step 54: repeatedly executing the step 53 on the surrounding pixels, judging the surrounding pixels by using a Canny edge detection operator, and if the surrounding pixels are not edge operators, taking the surrounding pixels as new seed pixels to continue growing; if the operator is an edge operator, the growth is stopped.
The invention also discloses a device for judging the quality and the malignancy of the pathological image of the breast tissue, which comprises a processor and a memory connected with the processor through a communication bus; wherein,
the memory for storing a computer program;
the processor is used for realizing any one of the methods for judging the canceration property of the breast pathology image considering the spatial information correlation when executing a computer program.
By applying the method and the device for judging the benign and malignant breast tissue pathological images provided by the embodiment of the invention, the accuracy of judging the breast tissue pathological images can be effectively improved, a pathologist is assisted in carrying out clinical diagnosis, and the diagnosis efficiency is improved.
Drawings
Fig. 1 is a schematic flow chart of a method for discriminating benign and malignant breast tissue pathological images according to an embodiment of the present invention.
Fig. 2 is a breast histopathology image region growing and edge detection workflow diagram.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention.
Stroma and stroma are two basic tissues in breast pathology images, and about 80% of the carcinomas occur in breast stromal tissue. The Wang et al experiment shows that nearly 82% of the large whole slice images are blank. For full-slice images of the breast, typically each image exceeds one billion pixels, which is a serious challenge for both recognition algorithms and hardware devices. The existing identification algorithm mostly calculates the whole picture, and does not distinguish the cell matrixes and the interstitials in the image, so a large amount of redundant calculation can be generated in the model training process, and the calculation amount of the algorithm is huge, the time consumption is high, and the efficiency is low. In view of this, the invention provides a method for discriminating the benign and malignant breast tissue pathology image, which automatically screens the regions with higher possibility of canceration in the whole slice image, and then discriminates the benign and malignant regions. The invention aims to make the identification algorithm more pertinent to the judgment of the breast pathological image, compress the calculated amount of the model, thereby improving the identification efficiency of the model on the breast tissue pathological image and solving the problems of low accuracy and larger calculation redundancy in the judgment of the breast tissue pathological image in the prior art.
The present invention provides a method for discriminating the benign and malignant breast tissue pathological image as shown in fig. 1, comprising:
and S110, partitioning the breast tissue pathology full-section image to be processed to obtain a plurality of partitioned images.
It should be noted that the acquired breast tissue pathology full-slice image is segmented, and the full-slice image is segmented into blocks of 224 × 224 pixels by using a segmentation technique, and the segmentation technique may adopt a segmentation method such as random segmentation and overlapping sampling segmentation. Specifically, the method comprises the following steps:
step 11: reading a breast histopathology whole-section image I by using an Openslide module;
step 12: sliding a window of size 224x224 pixels in a fixed step s (s <224 pixels) from left to right, top to bottom along image I, acquiring slices;
step 13: and storing the slices for subsequent processing.
And S120-S140 is respectively executed on each block image in the plurality of block images to obtain an image processing result of each block image after being sequentially processed by the steps S120-S140.
And S120, smoothing each block image in the plurality of block images to obtain the smoothed image.
It can be understood that the block image obtained in S110 is smoothed by using a median filtering method to reduce noise and blur generated in the tissue slice preparation process, so as to remove noise signals in the image, and different filtering methods may be selected to perform filtering processing on the blocks, such as mean filtering, maximum filtering, and the like. Specifically, the method comprises the following steps:
step 21: reading the blocks by using an OpenCV module, and sampling the blocks by selecting an observation window with radius r;
step 22: sorting the pixel values obtained in the current observation window to obtain a median pixel value;
step 23: updating the pixel value in the center of the observation window by using the median pixel value;
step 24: and repeating the step 22 and the step 23, and performing filtering processing on the whole block.
And S130, performing threshold segmentation on the smoothed image to obtain a block image with the background removed.
It should be noted that, the smoothed image obtained in S120 is subjected to threshold segmentation by using an OTSU global threshold segmentation method, and the image is automatically selected and initialized based on the image, so that the image is divided into a foreground image and a background image (since only an organization target is screened out and the background is excluded at this stage, the threshold of the target can be quickly found by using this method).
Further, automatically selecting an initialization threshold according to the image, judging whether the image is a foreground image or not, and if not, rejecting a background image; and if so, carrying out Gaussian denoising on the foreground image to obtain a background-removed image of the breast tissue pathology full-section image.
S140, performing edge segmentation on the block image with the background removed to obtain an image after edge segmentation; so as to obtain a plurality of edge-segmented images, wherein each segmented image respectively corresponds to the edge-segmented image.
It can be understood that, by performing edge segmentation on the background-removed block image obtained in S130, the invention adopts a method based on combination of region growing and edge detection to extract a stroma region of breast tissue cells, so as to separate stroma and stroma of the tissue cells and extract a stroma region more likely to become cancerous. Specifically, as shown in fig. 2:
step 51: selecting a group of seed pixels of the cell matrix area in the block image, and acquiring the gray value of the seed pixels; the purpose of grouping is to extract the cell matrix area in the block image, exclude other hollow areas and reduce the calculation amount of the model. Since the binarization process has been performed previously, the pixel of the void region should be in the vicinity of [0 ]. The OpenCV module is used for acquiring pixels of the cell matrix area in the image, wherein the pixel value of the pixels is b, and the pixels serve as seed pixels.
Step 52: determining a threshold T of the difference value of the gray value of the seed pixel; the pixels around the seed pixel and the seed are differenced, and the pixel value difference of the two similar pixels is not too large. The growth starts from the seed pixel in four directions of the upper, lower, left and right until a value near [0] is met (namely, a void region is met), and the growth is stopped when the void is met. The purpose here is to extract the substrate area of the segmented image. The threshold is not well defined as a fixed value, and the specific situation of each batch of block images needs to be seen. If the pixel value of the seed pixel is b, the pixel is encountered during the growth process of the seed pixel, and the difference between the pixel value and the pixel value exceeds T, the pixel is not added into the growth area, and the growth is stopped.
Step 53: calculating difference values of pixel gray values around the seed pixels and the seed pixels one by one, and if the difference values are smaller than a threshold T, selecting the difference values as elements in a growth area; if the threshold T is larger than the threshold T, rejecting;
step 54: and repeating the step 53 for the surrounding pixels, judging the surrounding pixels by using a Canny edge detection operator, taking the pixels as new seed pixels to continue growing if the surrounding pixels are not edge operators, and stopping growing if the surrounding pixels are edge operators.
S150, inputting the images after the edge segmentation into a trained neural network model to obtain a binary classification result of the fusion block.
And inputting the images after the edge segmentation corresponding to all the block images into a trained VGG19 network model for discrimination to obtain two classification results of whether the breast tissue is cancerated or not.
It should be noted that, the VGG19 convolutional neural network model designed and trained in the present invention first includes the specific steps:
step 61: redesigning the classifier of the original VGG model, and converting the output of the model into two types;
step 62: carrying out data amplification by using a data amplification technology; the data enhancement technology comprises the technologies of zooming, rotating, horizontal translation, vertical translation, random center cutting and the like, and the block images obtained by the data enhancement technology are used for data amplification to obtain more training samples. The method aims to train a deep convolutional neural network model with high accuracy and judge whether the breast tissue pathological image is cancerous or not. Carrying out fine tuning training on the improved VGG19 network model by using the data amplified in the step 5.2 and the label file thereof, freezing a feature extraction layer in the training process, and only training a classification layer; and storing the trained model parameters.
And step 63: and training and verifying the model to obtain the optimal model.
Based on the step 61, the VGG network model has 5 different configurations such as A-E, the original network is trained by ImageNet data sets, the VGG19 network model is adopted in the invention, the ImageNet data sets are used for pre-training the feature extraction layer of the model, the softmax function of the classification layer is changed into the classification function suitable for the invention, and the classification task of whether the breast tissue pathological images are cancerated or not is completed.
Based on step 62, the data enhancement techniques include scaling, rotation, horizontal translation, vertical translation, random center clipping, and the like.
Inputting the characteristics of the full-section tissue image into a trained convolutional neural network model for discrimination, aiming at discriminating whether a breast tissue pathological image is cancerated or not by a deep learning method, and comprising the following specific steps of:
building a VGG convolutional neural network model, wherein a pre-trained VGG19 network model is adopted in a feature extraction layer, and a softmax function of two classes is used in a classification layer so as to be suitable for the classification task of the invention;
and (3) training a classification layer of the convolutional neural network by using the processed breast tissue pathology block images and the label files, verifying the model of each round by using a verification set in the training process to obtain a model with the highest accuracy, obtaining a classification model, and storing model parameter files.
The method for judging whether the breast tissue pathological image has canceration or not through a deep learning method comprises the following specific steps:
step 71: building a VGG convolutional neural network, wherein a pre-trained VGG19 network model is adopted by a feature extraction layer, and a softmax function of two classes is used by a classification layer so as to be suitable for the classification task of the invention;
step 72: using the segmented image and the labeled file obtained after the breast tissue pathological image is processed through the steps, training a classification layer of the network, verifying the model of each round by using a verification set in the training process to obtain a model with the highest accuracy, obtaining a classification model, and storing a model parameter file;
step 73: and predicting the tissue picture by using the model obtained in the previous step to obtain two classification results of whether the breast tissue pathological images have canceration or not, and completing classification of the breast tissue pathological images.
And S160, judging the full-slice image based on the obtained two-classification result of the fusion block to obtain the two-classification result of the full slice.
Inputting the block images into a trained VGG19 network to obtain prediction results of the blocks, fusing the two classification results of the blocks, and judging the full-slice images to obtain the two classification results of the full slices.
Specifically, the method adopts a multi-example learning strategy, all the block images cut from one full slice are regarded as one packet, if all the blocks are cancer-free, the full slice is judged to be cancer-free, and if the blocks are cancerated, the full slice is judged to be cancer-free.
Firstly, pre-training a VGG19 network model network on a public data set ImageNet; after pre-training is implemented, the structure of the network model is adjusted, the output layer of the network is changed into 2 types, and the network output is a 2 type prediction result.
Assuming that a full-section image of breast tissue pathology is divided into 10 small images, the 10 small images are classified into two categories by a neural network model. If all 10 small images are judged to be cancer-free, the whole large image is cancer-free; if 1, or even more than one, of the 10 small images is cancerous, then the large image must be cancerous.
According to the method, the breast tissue pathological image is judged to be benign or malignant, so that the accuracy of judging the breast tissue pathological image can be effectively improved, and a pathologist is assisted in clinical diagnosis.
The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.
Claims (6)
1. A method for discriminating the malignancy or benign of a breast tissue pathological image includes:
partitioning a breast tissue pathology full-section image to be processed to obtain a plurality of partitioned images;
performing, separately for each of the plurality of block images:
carrying out smoothing processing on the block image to obtain a smoothed image;
performing threshold segmentation on the smoothed image to obtain a block image with the background removed;
performing edge segmentation on the block image with the background removed to obtain an image after edge segmentation; obtaining a plurality of images after edge segmentation, wherein each block image respectively corresponds to the image after edge segmentation;
inputting the images after the edge segmentation into a trained neural network model to obtain two classification results of fusion blocks;
and judging the full-slice image based on the obtained two-classification result of the fusion block to obtain the two-classification result of the full slice.
2. The method for discriminating the malignancy and the malignancy of a breast pathology image according to claim 1, wherein the step of obtaining a plurality of segmented images by segmenting the breast pathology full-section image to be processed:
reading a breast histopathology whole-section image by using an Openslide module;
and selecting a window with a preset size to slide along the preset direction of the breast tissue pathology full-section image in a fixed step length to obtain a section.
3. The method for discriminating the malignancy and the malignancy of a breast tissue pathology image according to claim 1, wherein the step of smoothing each of the block images to obtain a smoothed image includes:
step 31: reading the blocks by using an OpenCV module, and sampling each block image by adopting an observation window with a preset radius;
step 32: sorting the pixel values obtained in the current observation window to obtain a median pixel value;
step 33: updating the pixel value in the center of the observation window by using the median pixel value;
steps 32 and 33 are repeated until each block image is subjected to the filtering process.
4. The method for discriminating the malignancy and the malignancy of a breast pathology image according to claim 1, wherein the step of obtaining the background-removed image of the breast pathology full-slice image by performing threshold segmentation on the smoothed segmented image:
for each of the smoothed block images, performing the steps of:
performing threshold segmentation by using an OTSU global threshold segmentation method, automatically selecting an initialization threshold based on the image, and dividing the image into a foreground image and a background image;
and removing a background image, and carrying out Gaussian denoising on the foreground image to obtain a background-removed image of the breast tissue pathology full-section image.
5. The method according to claim 1, wherein the step of performing edge segmentation on the background-removed segmented image to obtain an edge-segmented image includes:
step 51: selecting a group of seed pixels of the cell matrix area in each partitioned image after the background is removed, and acquiring the gray value of the seed pixels;
step 52: determining a threshold of difference value of gray value of the seed pixel;
step 53: calculating difference values of pixel gray values around the seed pixels and the seed pixels one by one, and if the difference values are smaller than a threshold, selecting the difference values as elements in a growth area; if the number of the detected signals is larger than the threshold, rejecting the signals;
step 54: repeatedly executing the step 53 on the surrounding pixels, judging the surrounding pixels by using a Canny edge detection operator, and if the surrounding pixels are not the edge operator, taking the surrounding pixels as new seed pixels to continue growing; if the operator is an edge operator, the growth is stopped.
6. The device for judging the quality and the malignancy of the breast tissue pathological image is characterized by comprising a processor and a memory connected with the processor through a communication bus; wherein,
the memory for storing a computer program;
the processor, when executing the computer program, is configured to implement the method for interpreting the cancerous property of a breast pathology image with consideration of spatial information correlation according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111599328.6A CN114511588A (en) | 2021-12-24 | 2021-12-24 | Method and device for judging benign and malignant breast tissue pathological image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111599328.6A CN114511588A (en) | 2021-12-24 | 2021-12-24 | Method and device for judging benign and malignant breast tissue pathological image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114511588A true CN114511588A (en) | 2022-05-17 |
Family
ID=81548772
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111599328.6A Pending CN114511588A (en) | 2021-12-24 | 2021-12-24 | Method and device for judging benign and malignant breast tissue pathological image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114511588A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115909006A (en) * | 2022-10-27 | 2023-04-04 | 武汉兰丁智能医学股份有限公司 | Mammary tissue image classification method and system based on convolution Transformer |
| CN117252893A (en) * | 2023-11-17 | 2023-12-19 | 科普云医疗软件(深圳)有限公司 | Segmentation processing method for breast cancer pathological image |
-
2021
- 2021-12-24 CN CN202111599328.6A patent/CN114511588A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115909006A (en) * | 2022-10-27 | 2023-04-04 | 武汉兰丁智能医学股份有限公司 | Mammary tissue image classification method and system based on convolution Transformer |
| CN115909006B (en) * | 2022-10-27 | 2024-01-19 | 武汉兰丁智能医学股份有限公司 | Mammary tissue image classification method and system based on convolution transducer |
| CN117252893A (en) * | 2023-11-17 | 2023-12-19 | 科普云医疗软件(深圳)有限公司 | Segmentation processing method for breast cancer pathological image |
| CN117252893B (en) * | 2023-11-17 | 2024-02-23 | 科普云医疗软件(深圳)有限公司 | Segmentation processing method for breast cancer pathological image |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107067402B (en) | Medical image processing apparatus and breast image processing method thereof | |
| CN111091527B (en) | Method and system for automatically detecting pathological change area in pathological tissue section image | |
| CN107256558B (en) | Unsupervised automatic cervical cell image segmentation method and system | |
| Chekkoury et al. | Automated malignancy detection in breast histopathological images | |
| CN109064470B (en) | Image segmentation method and device based on self-adaptive fuzzy clustering | |
| Balamurugan et al. | Brain tumor segmentation and classification using hybrid deep CNN with LuNetClassifier | |
| CN114511588A (en) | Method and device for judging benign and malignant breast tissue pathological image | |
| CN110738637B (en) | Automatic classification system for breast cancer pathological sections | |
| CN111402267A (en) | Segmentation method, device and terminal for epithelial cell nucleus in prostate cancer pathological image | |
| CN107492088A (en) | Leucocyte automatic identification and statistical method in a kind of gynaecology's micro-image | |
| Unni et al. | Tumour detection in double threshold segmented mammograms using optimized GLCM features fed SVM | |
| Sarwar et al. | Segmentation of cervical cells for automated screening of cervical cancer: a review | |
| Jiang et al. | Breast cancer detection and classification in mammogram using a three-stage deep learning framework based on PAA algorithm | |
| CN112990214A (en) | Medical image feature recognition prediction model | |
| Reenadevi et al. | Breast cancer histopathological image classification using augmentation based on optimized deep ResNet-152 structure | |
| TWI587844B (en) | Medical image processing apparatus and breast image processing method thereof | |
| Albayrak et al. | A hybrid method of superpixel segmentation algorithm and deep learning method in histopathological image segmentation | |
| Alzubaidi et al. | Multi-class breast cancer classification by a novel two-branch deep convolutional neural network architecture | |
| Sajiv et al. | Automated Breast Cancer Classification based on Modified Deep learning Convolutional Neural Network following Dual Segmentation | |
| CN113870194B (en) | Breast tumor ultrasonic image processing device with fusion of deep layer characteristics and shallow layer LBP characteristics | |
| CN114155399A (en) | Breast pathology whole-section classification method based on multi-feature fusion progressive discrimination | |
| Logambal et al. | Cancer diagnosis using automatic mitotic cell detection and segmentation in histopathological images | |
| Kuo et al. | Lymphatic infiltration detection in breast cancer h&e image prior to lymphadenectomy | |
| Ravishankar et al. | Four novel approaches for detection of region of interest in mammograms—A comparative study | |
| Oprea et al. | A self organizing map approach to breast cancer detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |