CN112330701A - Tissue pathology image cell nucleus segmentation method and system based on polar coordinate representation - Google Patents
Tissue pathology image cell nucleus segmentation method and system based on polar coordinate representation Download PDFInfo
- Publication number
- CN112330701A CN112330701A CN202011351415.5A CN202011351415A CN112330701A CN 112330701 A CN112330701 A CN 112330701A CN 202011351415 A CN202011351415 A CN 202011351415A CN 112330701 A CN112330701 A CN 112330701A
- Authority
- CN
- China
- Prior art keywords
- image
- segmented
- cell nucleus
- segmentation
- polar coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000011218 segmentation Effects 0.000 title claims abstract description 74
- 238000000034 method Methods 0.000 title claims abstract description 73
- 210000003855 cell nucleus Anatomy 0.000 title claims abstract description 47
- 230000007170 pathology Effects 0.000 title claims abstract description 30
- 210000001519 tissue Anatomy 0.000 title claims abstract description 26
- 210000004940 nucleus Anatomy 0.000 claims abstract description 32
- 238000012545 processing Methods 0.000 claims abstract description 14
- 230000006870 function Effects 0.000 claims description 14
- 238000007781 pre-processing Methods 0.000 claims description 12
- 238000003860 storage Methods 0.000 claims description 10
- 238000000605 extraction Methods 0.000 claims description 9
- 238000013527 convolutional neural network Methods 0.000 claims description 8
- 238000010606 normalization Methods 0.000 claims description 6
- 230000001575 pathological effect Effects 0.000 abstract description 24
- 238000005094 computer simulation Methods 0.000 abstract description 3
- 230000000694 effects Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 12
- 238000012549 training Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 9
- 238000012360 testing method Methods 0.000 description 7
- 238000001514 detection method Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000004043 dyeing Methods 0.000 description 3
- 238000003709 image segmentation Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000000903 blocking effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 2
- 238000010191 image analysis Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000012188 paraffin wax Substances 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 238000004445 quantitative analysis Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000004027 cell Anatomy 0.000 description 1
- 238000004195 computer-aided diagnosis Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 239000000975 dye Substances 0.000 description 1
- 210000004907 gland Anatomy 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 210000004072 lung Anatomy 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000007789 sealing Methods 0.000 description 1
- 238000010186 staining Methods 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
Abstract
The utility model provides a tissue pathology image cell nucleus segmentation method and system based on polar coordinate representation, which adopts the image characteristics of the image block to be segmented and obtains the classification data, the central point coordinate and the ray length of the image characteristics based on the polar coordinate system modeling; the method comprises the following steps of performing nucleus segmentation on an image block to be segmented according to classification data, a central point coordinate and a ray length to obtain an image block comprising a central position coordinate and an edge of each nucleus, and in terms of processing effect, firstly providing a method for automatically segmenting the nucleus of a histopathology image based on polar coordinate representation, wherein the method based on the polar coordinate is very suitable for segmentation of a circular object; the pathological image cell nucleus automatic segmentation can be automatically completed, the cell nucleus center and the length of a ray pointing to the target edge with the cell nucleus center as a starting point are obtained, and the cell nucleus is positioned and finely segmented.
Description
Technical Field
The present disclosure relates to the field of image processing, and in particular, to a method and a system for automatically segmenting a cell nucleus of a tissue pathology image based on polar coordinate representation.
Background
Accurate localization and segmentation of nuclei are important steps in pathological image analysis, and are used for quantitatively analyzing the characteristics of tissues in digital pathological images, thereby realizing computer-aided diagnosis. In recent years, artificial intelligence based methods have shown great potential in pathological image analysis tasks.
Many methods have been proposed for the segmentation of cells and nuclei of histopathological images, such as threshold-based image segmentation methods, edge-based image segmentation methods, region growing image segmentation methods, and in addition, nucleus segmentation methods based on specific theories have been proposed, such as: wavelet transform-based methods, level set-based methods, fuzzy mean theory-based methods. However, such techniques for automatic segmentation of pathological image nuclei generally have the limitations of low efficiency, poor results, etc., and it is difficult to achieve fast instance segmentation of pathological image nuclei. An image target instance segmentation method based on artificial intelligence, such as Mask-RCNN, can better solve the problem of pathological image cell nucleus segmentation, but has the limitations of complex model structure, long training time, high hardware equipment requirement and the like.
In summary, the prior art is still lack of an effective solution for the problem of automatic detection of cell nuclei in histopathology images.
Disclosure of Invention
In order to solve the technical problem, the present disclosure provides a tissue pathology image cell nucleus segmentation method and system based on polar coordinate representation.
In a first aspect, the present disclosure provides a method for segmentation of a cell nucleus based on a tissue pathology image represented by polar coordinates, comprising:
acquiring an image to be segmented;
preprocessing an image to be segmented to obtain a plurality of image blocks to be segmented;
extracting image features of image blocks to be segmented, and modeling based on a polar coordinate system to obtain classification data, a central point coordinate and a ray length of the image features; and performing nucleus segmentation of the image block to be segmented according to the classification data, the central point coordinate and the ray length to obtain an image block containing the central position coordinate and the edge of each nucleus.
In a second aspect, the present disclosure provides a system for cellular nucleus segmentation based on a tissue pathology image in polar coordinate representation, comprising:
a data acquisition module: acquiring an image to be segmented;
an image preprocessing module: preprocessing an image to be segmented to obtain a plurality of image blocks to be segmented;
a data processing module: extracting image features of image blocks to be segmented, and modeling based on a polar coordinate system to obtain classification data, a central point coordinate and a ray length of the image features; and performing nucleus segmentation of the image block to be segmented according to the classification data, the central point coordinate and the ray length to obtain an image block containing the central position coordinate and the edge of each nucleus.
In a third aspect, the present disclosure provides a computer-readable storage medium for storing computer instructions which, when executed by a processor, perform the method for tissue pathology image cell nucleus segmentation based on polar coordinate representation according to the first aspect.
In a fourth aspect, the present disclosure provides an electronic device comprising a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein the computer instructions, when executed by the processor, implement the method for segmentation of tissue pathology image nuclei based on polar coordinate representation according to the first aspect.
Compared with the prior art, this disclosure possesses following beneficial effect:
1. the method comprises the steps of extracting image features of image blocks to be segmented, and modeling based on a polar coordinate system to obtain classification data, a central point coordinate and a ray length of the image features; the method comprises the steps of carrying out cell nucleus segmentation on image blocks to be segmented according to classification data, central point coordinates and ray lengths to obtain image blocks containing central position coordinates and edges of each cell nucleus. The model can automatically segment the pathological image cell nucleus to obtain the cell nucleus center and the length of a ray pointing to the target edge with the cell nucleus center as a starting point, and the positioning and the fine segmentation of the cell nucleus are realized.
2. In applicability and expansibility, in one embodiment, the method takes a lung gland histopathology image as an example, and realizes the automatic segmentation of the cell nucleus of the histopathology image. In the method for automatically segmenting the tissue pathological image cell nucleus based on polar coordinate representation, the head network module of the model comprises classification branches, can be used for automatically classifying the cell nucleus, can realize classification of different tissue types, and solves the problems that the automatic segmentation of the pathological image cell nucleus generally has the limitations of low efficiency, poor result and the like, and the rapid example segmentation of the pathological image cell nucleus is difficult to realize.
3. In the aspect of operation speed, the method is based on a deep learning model, based on an FCOS framework, and a one-stage example segmentation method is adopted, so that a backbone network is simple, the calculation complexity is low, and the nucleus segmentation in the pathological image block can be obtained in a very short time. The method based on polar coordinate representation has fast convergence speed and less iteration times in the training process. The method solves the limitation problems of complex model structure, long training time, high requirement on hardware equipment and the like.
4. The method comprises the steps of leading out the acquired full-size digital pathological image according to a certain magnification factor, and carrying out dicing and normalization processing; after an image block which can be directly used for cell nucleus segmentation is obtained, cell nucleus detection and segmentation in the image block are realized by using the cell nucleus automatic segmentation model based on the tissue pathology image represented by the polar coordinates; and displaying the cell nucleus example segmentation result generated by the model, and performing splicing display on the image blocks according to the requirement. The model established by the method can realize the automatic segmentation of the cell nucleus of the histopathology image without manual marking of a doctor, and the automatic segmentation of the cell nucleus of the histopathology image is completely realized.
Advantages of additional aspects of the disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application.
FIG. 1 is a schematic illustration of a tissue pathology image nucleus segmentation method based on polar coordinate representation according to the present disclosure;
fig. 2 is a diagram of an example of a full-size digital pathology image in example 1 of the present disclosure;
fig. 3 is a diagram of an example of a dicing process for a full-size digital pathology image in embodiment 1 of the present disclosure;
FIG. 4 is a model structure diagram of the disclosed automatic cell nucleus segmentation method based on a tissue pathology image represented by polar coordinates;
FIG. 5 is a first tissue pathology image block and nucleus segmentation result display of the present disclosure;
FIG. 6 is a second tissue pathology image block and nucleus segmentation result display of the present disclosure;
fig. 7 is a procedure of nuclear segmentation of a full-size digital pathology image in example 1 of the present disclosure.
The specific implementation mode is as follows:
the present disclosure is further described with reference to the following drawings and examples.
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
Belongs to the explanation:
typical cell nucleus segmentation methods include a threshold-based segmentation method, a region-based segmentation method, an edge-based segmentation method, a particular theory-based segmentation method, and the like, and a deep learning-based segmentation method that has appeared in recent years. Based on a deep learning method, such as Mask R-CNN, YoLACT, Mask screening R-CNN, TensorMask and the like, the deep learning method greatly improves the precision and efficiency of the nuclear segmentation and can realize the nuclear instance segmentation.
Example 1
As shown in fig. 1, the present disclosure provides a method for tissue pathology image nucleus segmentation based on polar coordinate representation, comprising:
acquiring an image to be segmented;
preprocessing an image to be segmented to obtain a plurality of image blocks;
extracting image features of the image blocks, and modeling based on a polar coordinate system to obtain classification data, a central point coordinate and a ray length of the image features; and performing cell nucleus segmentation according to the classification data, the central point coordinates and the ray length to obtain an image block containing the central position coordinates and the edge of each cell nucleus.
Further, the acquiring of the image to be segmented specifically includes obtaining a pathological section through paraffin fixation, slicing, pasting, H & E staining, mounting and the like, and then scanning the pathological section into a full-size digital pathological image (WSI) by using a scanner. In the embodiment, paraffin fixation, slicing, pasting, dyeing, sealing and the like are carried out in a conventional manner without limitation; the specific mode of the scanner for acquiring the full-size digital pathological image is not limited; such as the example graph of the full-scale digital pathology image of fig. 2, and the example graph of the dicing process for the full-scale digital pathology image of fig. 3.
Further, preprocessing the image to be segmented to obtain a plurality of image blocks, wherein the preprocessing comprises the steps of amplifying the image to be segmented and segmenting the image to be segmented into the plurality of image blocks; and carrying out image normalization processing on the obtained image block to obtain a normalized image block. As shown in fig. 7, a procedure for nuclear segmentation of full-size digital pathology images is provided.
The method specifically comprises the following steps: taking the implementation of blocking in python as an example: (1) importing an OpenSlide tool kit, importing OpenSlide, deepzoom and deepZoomGenerator in a grading way, reading by using Openslide, and determining a down-sampling factor; (2) setting a cut size, and cutting the full-size digital pathological image into image blocks which can be directly used for analysis by using a DeepZoomGenerator (slide, tile _ size ═ high, overlap ═ 0, and limit _ boundaries ═ False) command, wherein tile _ size is used for setting the size of the cut, overlap is defined whether or not, and limit _ boundaries represents whether to discard when the size of the edge image is smaller than the set tile _ size value; (3) and storing the cut image blocks in corresponding row and column names.
When the image is diced, the specific blocking algorithm used is not limited, and for example, the image may be diced in any one of programming languages such as Matlab and Python, or another kit for reading pathological images and blocks may be used.
The digital pathological image is influenced by preparation of a glass slide (dyeing agent proportion, dyeing time, dyeing temperature and the like) and image acquisition (an imaging platform, digital noise and the like), the imaging color difference is large, and the difficulty is brought to quantitative analysis. Not only affect nuclear segmentation, but also prognostic prediction and diagnosis. To overcome this problem, a pathology image stain normalization needs to be performed.
When the pathological image is normalized, a specific image normalization method is not limited, and for example, histogram equalization, histogram normalization, Retinex enhancement, or the like may be used, and other methods for predicting parameters through a network may be used.
Further, the image features of the image blocks are extracted, and classification data, central point coordinates and ray lengths of the image features are obtained based on polar coordinate system modeling.
Further, the feature extraction module comprises a backbone convolutional neural network and a feature pyramid network, and features of different scales are obtained through the feature extraction module and are used for subsequent nuclear center positioning and edge regression.
Further, the head network module comprises a first branch and a second branch, wherein the first branch is used for obtaining a target classification and a central point; the second branch is used for regressing the length of the ray emitted by taking the central point as a starting point.
Further, the operation steps of the head network module include:
establishing a polar coordinate of a mask, selecting the center of the mask to diverge outwards from the center point of the mask at intervals of angles through n rays, and obtaining the edge of the mask of a target according to the lengths of the n rays;
preferably, the specific step of obtaining the edge of the mask of the target is to select the center of the mask at first, and the n rays diverge outwards at intervals and angles from the center point of the mask; obtaining the lengths of n rays, wherein the length of the ray is the distance from the center to the edge of the object; starting from 0 degrees, finding control points one by one at intervals of 10 degrees; connecting the n control points to obtain the shade edge of the target;
preferably, 9 to 16 points near the centroid of the image block to be segmented are used as candidate points of the central point; uniformly dispersing each candidate point to the edge of the target object as a starting point to obtain a ray length group, and calculating the weight of the candidate center point according to the ray length group;
preferably, in all the candidate points, the minimum value and the maximum value in the corresponding ray lengths are calculated, and the candidate points of the center point are reweighed according to the difference between the minimum value and the maximum value to find the optimal center point.
Further, the loss function module adopts a polar coordinate intersection ratio loss function for polar coordinate and ray length regression.
Calculating the intersection ratio between the predicted mask and the real mask in an integral mode; the model is converted into a discrete summation form, and the rays used to generate the mask start from a central point and are directed uniformly towards the edges of the object.
As can be seen from the tissue pathology image blocks and the cell nucleus segmentation result display diagrams of fig. 5 and 6, the technical scheme of the disclosure ensures the accuracy of the cell nucleus segmentation of the tissue pathology image. The results of the nuclear segmentation can be used for further quantitative analysis of pathological cytology.
Example 2
A system for cellular nucleus segmentation of histopathology images based on polar coordinate representation, comprising:
a data acquisition module: acquiring an image to be segmented;
an image preprocessing module: preprocessing an image to be segmented to obtain a plurality of image blocks to be segmented;
a data processing module: extracting image features of image blocks to be segmented, and modeling based on a polar coordinate system to obtain classification data, a central point coordinate and a ray length of the image features; and performing nucleus segmentation of the image block to be segmented according to the classification data, the central point coordinate and the ray length to obtain an image block containing the central position coordinate and the edge of each nucleus.
Further, the specific modes of the data acquisition module are configured to respectively correspond to the specific steps of the tissue pathology image cell nucleus segmentation method based on polar coordinate representation in the above embodiment.
Example 3
A computer readable storage medium storing computer instructions which, when executed by a processor, perform the method for segmentation of tissue pathology image cell nuclei based on polar coordinate representation as described in the above embodiments.
Example 4
An electronic device comprising a memory and a processor and computer instructions stored on the memory and executed on the processor, the computer instructions, when executed by the processor, performing the method for segmentation of tissue pathology image nuclei based on polar coordinate representation as described in the above embodiments.
Example 5
A cell nucleus automatic segmentation model of a tissue pathology image based on polar coordinate representation comprises a feature extraction module, a head network module and a loss function module.
The characteristic extraction module is composed of a backbone Convolutional Neural Network (CNN) and a characteristic pyramid network (FPN) and is used for extracting characteristics of images with different scales for subsequent target detection and segmentation tasks;
as shown in fig. 4, the convolutional neural network includes a convolutional layer, a pooling layer, an activation layer, etc., wherein the convolutional layer is composed of a set of learnable convolutional kernels and can be used for extracting features. The full convolutional neural Network (FCN) uses a deconvolution (deconvolution) to gradually reduce the downsampled abstract features to the original size. The feature extraction module used in the present invention is a coding and decoding structure based on a full convolution neural network, as shown in the backbone + FPN structure in fig. 4: (1) in the bottom-up path of encoding, features of different scales are extracted hierarchically through convolution and downsampling. (2) Coding and decoding horizontal connection, taking P4 layer as an example, using C5 feature map upsampling, C4 feature map is added after 1 × 1 convolution to obtain P4 feature layer, and other layers are similar in horizontal connection. (3) In the decoded top-down path, starting from the C5 convolutional layer, the P5 layers are obtained by 1 × 1 convolution, and then the P4 layers are obtained by using the method described in the cross-connect, and further the P3 layers and the P2 layers are obtained. The P5 layer passes through the maximum pooling layer to obtain P6, the same process obtains P7, and the P2-P5 layers are respectively convoluted by 3 multiplied by 3. And (4) obtaining features of different scales by a model through a feature extraction module, and using the features for subsequent nuclear center positioning and edge regression.
When the pyramid feature finally output by the feature pyramid network (hereinafter referred to as a final pyramid feature) is constructed, a construction manner from top to bottom is usually adopted, for example, the feature at the deepest layer of the encoder is firstly utilized to construct the highest-layer feature of the final pyramid feature (i.e., the feature at the first layer with the smallest spatial resolution), then the feature at the second highest layer and the feature at the second deeper layer of the encoder are utilized to construct the next-higher-layer feature (i.e., the feature at the second layer with the smallest spatial resolution), and so on until the feature at the lowest layer of the final pyramid feature (i.e., the feature at the second layer with the largest spatial resolution) is.
The specific steps of the characteristic pyramid network are as follows: acquiring an image to be segmented, and outputting a first pyramid feature aiming at the image to be segmented by a hidden layer used for generating multi-scale features in a feature pyramid network to be trained; acquiring a first interest area characteristic from the first pyramid characteristic according to the interest area; carrying out target object detection on the first interest region characteristic to obtain a first target object detection result; and adjusting the network parameters of the feature pyramid network according to the difference between the first target object detection result and the target object marking information of the sample image.
And the head network module is based on polar coordinate modeling of the example. The method comprises two branches, wherein one branch is used for obtaining a target classification and a central point, and the other branch is used for regressing the length of a ray emitted by taking the central point as a starting point;
the method is based on polar coordinate system modeling, and converts an example segmentation problem into a center point solving problem and a ray length problem. One branch gets the classification result and the center point coordinates, and one branch gets the regression value of the ray length.
Polar coordinate representation of the mask.
For a given example mask, first the mask center (x) is selectedc,yc) N rays diverge outward from the center of the mask at an angle Δ θ (in this method, 36 regression edges of the rays are used, n is 36, and Δ θ is 10 °), and the length { d ] of the n rays is obtained1,d2,…,dnThe length of the ray is the distance from the center to the edge of the object. Starting from 0 DEG, finding control points (x) one by one at intervals of 10 DEGi,yi)：
xi＝cosθi×di+xc
yi＝sinθi×di+yc
The n control points are connected to obtain the mask edge of the target.
And selecting a central point.
In the present disclosure, to ensure the centrality of polar coordinates, 9 to 16 points near the centroid of the sample are used as candidate points for the central point. Each candidate point is uniformly dispersed to the edge of the target object as a starting point, and a group of ray lengths { d } are obtained1,d2,…,dnAnd calculating the weight of the candidate center point by the following formula:
calculating the minimum value d of the corresponding ray lengths in all the candidate pointsminAnd maximum value dmax，dminAnd dmaxThe closer together, the higher the quality of the candidate point is considered. At this time, the higher the weight given to this candidate point, the higher the probability that this point is the center point. By adopting the method, the candidate points of the central point can be reweighed to find the optimal central point.
And the loss function module adopts an Intersection over Union (IoU) loss function for polar coordinate and ray length regression.
IoU denotes the intersection ratio between the predicted mask and the real label. For mask segmentation in polar coordinates, the calculation formula of IoU can be expressed as:
where d is the true center-to-edge distance, d*θ is the angle for the center-to-edge distance predicted by the model. The formula calculates the intersection ratio between the predicted mask and the real mask in an integral mode. For ease of computation, the model is converted to a discrete summation form. The rays for generating the mask start from a central point and are uniformly directed to the edge of the object, and the included angle between two rays d and d +1 is Δ θ:
as N approaches infinity, Δ θ is infinitesimally small and the discretized IoU can be viewed as the sum of the areas of innumerable small triangles, the discretized result being the same as the integrated result. In the method used in the present disclosure, the radiation is emitted uniformly, and therefore,
the same terms in the numerator and the denominator are reduced, the square term is reduced to a non-square term, and the formula can be further reduced to the sum of the short side and the long side:
Polar-coordinate-intersection-ratio-Loss function Polar IoU Loss is the binary cross-entropy Loss of Polar IoU, expressed as:
the polar coordinate intersection-parallel ratio loss function can be micro, and backward propagation can be performed; compared with Smooth-l1 loss, the regression target can be wholly predicted, and the overall performance of the algorithm is improved.
The automatic nucleus segmentation model is used for performing nucleus segmentation processing on the pathological image block and comprises a model training stage and a model testing stage;
and in the model training stage, setting a data path, a pre-training model, a model storage path and the like used by the cell nucleus automatic segmentation model of the histopathology image expressed based on the polar coordinates, and training parameters such as initialization, deviation, regularization, initial learning rate, learning rate reduction mode, optimization algorithm, iteration times, data enhancement mode and the like to realize the training of the cell nucleus automatic segmentation model of the histopathology image expressed based on the polar coordinates.
Alternatively, model training may be done on the initialized model, pre-trained model transfer learning may be used, or training may continue on the trained model. The trained model is used for directly carrying out nucleus segmentation processing on the pathological image block.
A model test phase, setting an input image, using a model, and the like. In one embodiment, the inputs include a test folder path, a test model, a number of test images, a test result output path.
And finally, displaying a test result, displaying a cell nucleus example segmentation result generated by the model, and performing image block splicing display according to the requirement.
As will be appreciated by one skilled in the art, embodiments of the present disclosure may be provided as a method, system, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.
As will be appreciated by one skilled in the art, embodiments of the present disclosure may be provided as a method, system, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.
The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure, and various modifications and changes may be made to the present disclosure by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.
Although the present disclosure has been described with reference to specific embodiments, it should be understood that the scope of the present disclosure is not limited thereto, and those skilled in the art will appreciate that various modifications and changes can be made without departing from the spirit and scope of the present disclosure.
Claims (10)
1. The method for segmenting the cell nucleus of the tissue pathology image based on polar coordinate representation is characterized by comprising the following steps of:
acquiring an image to be segmented;
preprocessing an image to be segmented to obtain a plurality of image blocks to be segmented;
extracting image features of image blocks to be segmented, and modeling based on a polar coordinate system to obtain classification data, a central point coordinate and a ray length of the image features; and performing nucleus segmentation of the image block to be segmented according to the classification data, the central point coordinate and the ray length to obtain an image block containing the central position coordinate and the edge of each nucleus.
2. The method for automatically segmenting the cell nucleus according to claim 1, wherein the step of preprocessing the image to be segmented to obtain a plurality of image blocks comprises the steps of segmenting the image to be segmented into a plurality of image blocks after amplifying the image to be segmented; and carrying out image normalization processing on the obtained image block to obtain an image block to be segmented.
3. The method of claim 1, wherein the step of extracting image features of the image blocks and modeling based on a polar coordinate system to obtain classification data, center point coordinates and ray lengths of the image features comprises establishing a cell nucleus segmentation model, the cell nucleus segmentation model comprising a feature extraction module, a head network module and a loss function module.
4. The method for automatically segmenting the cell nucleus according to claim 1, wherein the feature extraction module comprises a backbone convolutional neural network and a feature pyramid network, and features of different scales are obtained through the feature extraction module and are used for cell nucleus center positioning and edge regression.
5. The method for automatically segmenting a cell nucleus according to claim 4, wherein the head network module comprises a first branch and a second branch, wherein the first branch is used for obtaining a target classification and a central point; the second branch is used for regressing the length of the ray emitted by taking the central point as a starting point.
6. The method for automatically segmenting a cell nucleus according to claim 4, wherein the operation step of the head network module comprises:
establishing a polar coordinate of a mask, selecting the center of the mask to diverge outwards from the center point of the mask at intervals of angles through n rays, and obtaining the edge of the mask of a target according to the lengths of the n rays;
preferably, the specific step of obtaining the edge of the mask of the target is to select the center of the mask at first, and the n rays diverge outwards at intervals and angles from the center point of the mask; obtaining the lengths of n rays, wherein the length of the ray is the distance from the center to the edge of the object; starting from 0 degrees, finding control points one by one at intervals of 10 degrees; connecting the n control points to obtain the shade edge of the target;
preferably, 9 to 16 points near the centroid of the image block to be segmented are used as candidate points of the central point; uniformly dispersing each candidate point to the edge of the target object as a starting point to obtain a ray length group, and calculating the weight of the candidate center point according to the ray length group;
preferably, in all the candidate points, the minimum value and the maximum value in the corresponding ray lengths are calculated, and the candidate points of the center point are reweighed according to the difference between the minimum value and the maximum value to find the optimal center point.
7. The method for automatically segmenting the cell nucleus according to claim 4, wherein the loss function module adopts a polar coordinate intersection ratio loss function and calculates the intersection ratio between the predicted mask and the real mask in an integral mode; the model is converted into a discrete summation form, and the rays used to generate the mask start from a central point and are directed uniformly towards the edges of the object.
8. A system for segmenting a cell nucleus of a tissue pathology image based on polar coordinate representation is characterized by comprising:
a data acquisition module: acquiring an image to be segmented;
an image preprocessing module: preprocessing an image to be segmented to obtain a plurality of image blocks to be segmented;
a data processing module: extracting image features of image blocks to be segmented, and modeling based on a polar coordinate system to obtain classification data, a central point coordinate and a ray length of the image features; and performing nucleus segmentation of the image block to be segmented according to the classification data, the central point coordinate and the ray length to obtain an image block containing the central position coordinate and the edge of each nucleus.
9. A computer-readable storage medium storing computer instructions which, when executed by a processor, perform the method for segmentation of nuclei of histopathology images based on polar coordinate representation according to any one of claims 1-7.
10. An electronic device comprising a memory and a processor, and computer instructions stored on the memory and executed on the processor, wherein the computer instructions, when executed by the processor, perform the method for segmentation of nuclei of histopathology images based on polar coordinate representation according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011351415.5A CN112330701A (en) | 2020-11-26 | 2020-11-26 | Tissue pathology image cell nucleus segmentation method and system based on polar coordinate representation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011351415.5A CN112330701A (en) | 2020-11-26 | 2020-11-26 | Tissue pathology image cell nucleus segmentation method and system based on polar coordinate representation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112330701A true CN112330701A (en) | 2021-02-05 |
Family
ID=74308928
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011351415.5A Pending CN112330701A (en) | 2020-11-26 | 2020-11-26 | Tissue pathology image cell nucleus segmentation method and system based on polar coordinate representation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112330701A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112906551A (en) * | 2021-02-09 | 2021-06-04 | 北京有竹居网络技术有限公司 | Video processing method and device, storage medium and electronic equipment |
| CN113469302A (en) * | 2021-09-06 | 2021-10-01 | 南昌工学院 | Multi-circular target identification method and system for video image |
| WO2022257254A1 (en) * | 2021-06-10 | 2022-12-15 | 腾讯云计算（北京）有限责任公司 | Image data processing method and apparatus, and device and medium |
| CN116821396A (en) * | 2023-08-25 | 2023-09-29 | 神州医疗科技股份有限公司 | Pathological labeling system based on OpenSeadragon framework |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103985119A (en) * | 2014-05-08 | 2014-08-13 | 山东大学 | Method for partitioning cytoplasm and cell nucleuses of white blood cells in color blood cell image |
| CN107977682A (en) * | 2017-12-19 | 2018-05-01 | 南京大学 | Lymph class cell sorting method and its device based on the enhancing of polar coordinate transform data |
| CN108492272A (en) * | 2018-03-26 | 2018-09-04 | 西安交通大学 | Cardiovascular vulnerable plaque recognition methods based on attention model and multitask neural network and system |
| CN110070529A (en) * | 2019-04-19 | 2019-07-30 | 深圳睿心智能医疗科技有限公司 | A kind of Endovascular image division method, system and electronic equipment |
| CN110580699A (en) * | 2019-05-15 | 2019-12-17 | 徐州医科大学 | Pathological image cell nucleus detection method based on improved fast RCNN algorithm |
| CN111027547A (en) * | 2019-12-06 | 2020-04-17 | 南京大学 | Automatic detection method for multi-scale polymorphic target in two-dimensional image |
| CN111311626A (en) * | 2020-05-11 | 2020-06-19 | 南京安科医疗科技有限公司 | Skull fracture automatic detection method based on CT image and electronic medium |
| CN111985488A (en) * | 2020-09-01 | 2020-11-24 | 江苏方天电力技术有限公司 | Target detection segmentation method and system based on offline Gaussian model |
-
2020
- 2020-11-26 CN CN202011351415.5A patent/CN112330701A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103985119A (en) * | 2014-05-08 | 2014-08-13 | 山东大学 | Method for partitioning cytoplasm and cell nucleuses of white blood cells in color blood cell image |
| CN107977682A (en) * | 2017-12-19 | 2018-05-01 | 南京大学 | Lymph class cell sorting method and its device based on the enhancing of polar coordinate transform data |
| CN108492272A (en) * | 2018-03-26 | 2018-09-04 | 西安交通大学 | Cardiovascular vulnerable plaque recognition methods based on attention model and multitask neural network and system |
| CN110070529A (en) * | 2019-04-19 | 2019-07-30 | 深圳睿心智能医疗科技有限公司 | A kind of Endovascular image division method, system and electronic equipment |
| CN110580699A (en) * | 2019-05-15 | 2019-12-17 | 徐州医科大学 | Pathological image cell nucleus detection method based on improved fast RCNN algorithm |
| CN111027547A (en) * | 2019-12-06 | 2020-04-17 | 南京大学 | Automatic detection method for multi-scale polymorphic target in two-dimensional image |
| CN111311626A (en) * | 2020-05-11 | 2020-06-19 | 南京安科医疗科技有限公司 | Skull fracture automatic detection method based on CT image and electronic medium |
| CN111985488A (en) * | 2020-09-01 | 2020-11-24 | 江苏方天电力技术有限公司 | Target detection segmentation method and system based on offline Gaussian model |
Non-Patent Citations (1)
| Title |
|---|
| ENZE XIE ET AL.: "PolarMask:Single Shot Instance Segmentation with Polar Representation", 《ARXIV:1909.13226V4 [CS.CV]》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112906551A (en) * | 2021-02-09 | 2021-06-04 | 北京有竹居网络技术有限公司 | Video processing method and device, storage medium and electronic equipment |
| WO2022257254A1 (en) * | 2021-06-10 | 2022-12-15 | 腾讯云计算（北京）有限责任公司 | Image data processing method and apparatus, and device and medium |
| CN113469302A (en) * | 2021-09-06 | 2021-10-01 | 南昌工学院 | Multi-circular target identification method and system for video image |
| CN116821396A (en) * | 2023-08-25 | 2023-09-29 | 神州医疗科技股份有限公司 | Pathological labeling system based on OpenSeadragon framework |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10127675B2 (en) | Edge-based local adaptive thresholding system and methods for foreground detection | |
| Xie et al. | Multilevel cloud detection in remote sensing images based on deep learning | |
| CN112330701A (en) | Tissue pathology image cell nucleus segmentation method and system based on polar coordinate representation | |
| CN114120102A (en) | Boundary-optimized remote sensing image semantic segmentation method, device, equipment and medium | |
| Khoshelham et al. | Performance evaluation of automated approaches to building detection in multi-source aerial data | |
| US20080101676A1 (en) | System and Method For Segmenting Chambers Of A Heart In A Three Dimensional Image | |
| CN108537751B (en) | Thyroid ultrasound image automatic segmentation method based on radial basis function neural network | |
| CN113780296A (en) | Remote sensing image semantic segmentation method and system based on multi-scale information fusion | |
| Yi et al. | Automatic extraction of cell nuclei from H&E-stained histopathological images | |
| EP2909793A2 (en) | Biological unit identification based on supervised shape ranking | |
| CN114240961A (en) | U-Net + + cell division network system, method, equipment and terminal | |
| Jia et al. | Multi-layer segmentation framework for cell nuclei using improved GVF Snake model, Watershed, and ellipse fitting | |
| WO2018176189A1 (en) | Image segmentation method and system | |
| CN113450396A (en) | Three-dimensional/two-dimensional image registration method and device based on bone features | |
| CN113177957A (en) | Cell image segmentation method and device, electronic equipment and storage medium | |
| CN117197456A (en) | HE dyeing-oriented pathological image cell nucleus simultaneous segmentation classification method | |
| Kalyani et al. | Deep learning-based detection and classification of adenocarcinoma cell nuclei | |
| CN113496148A (en) | Multi-source data fusion method and system | |
| Wu et al. | Semiautomatic segmentation of glioma on mobile devices | |
| He et al. | FRFCM clustering segmentation method for medical MR image feature diagnosis | |
| Parpulov et al. | Convolutional neural network application for cells segmentation in immunocytochemical study | |
| CN114972382A (en) | Brain tumor segmentation algorithm based on lightweight UNet + + network | |
| Jiang et al. | Ship contour extraction from SAR images based on faster R-CNN and chan–vese model | |
| CN109409375B (en) | SAR image semantic segmentation method based on contour structure learning model | |
| Mao et al. | W-net: A two-stage convolutional network for nucleus detection in histopathology image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210205 |