CN116433704A - Cell nucleus segmentation method based on central point and related equipment - Google Patents
Cell nucleus segmentation method based on central point and related equipment Download PDFInfo
- Publication number
- CN116433704A CN116433704A CN202211705880.3A CN202211705880A CN116433704A CN 116433704 A CN116433704 A CN 116433704A CN 202211705880 A CN202211705880 A CN 202211705880A CN 116433704 A CN116433704 A CN 116433704A
- Authority
- CN
- China
- Prior art keywords
- cell nucleus
- segmentation
- nuclear
- central point
- result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 210000003855 cell nucleus Anatomy 0.000 title claims abstract description 221
- 230000011218 segmentation Effects 0.000 title claims abstract description 146
- 238000000034 method Methods 0.000 title claims abstract description 68
- 238000001514 detection method Methods 0.000 claims abstract description 51
- 238000012549 training Methods 0.000 claims abstract description 44
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 37
- 238000001914 filtration Methods 0.000 claims abstract description 14
- 238000012545 processing Methods 0.000 claims abstract description 12
- 230000001419 dependent effect Effects 0.000 claims abstract description 11
- 238000010586 diagram Methods 0.000 claims description 21
- 210000004940 nucleus Anatomy 0.000 claims description 14
- 238000002372 labelling Methods 0.000 claims description 12
- 239000004973 liquid crystal related substance Substances 0.000 claims description 8
- 230000006870 function Effects 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 4
- 239000003550 marker Substances 0.000 claims description 4
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 3
- 238000003064 k means clustering Methods 0.000 claims description 3
- 238000012360 testing method Methods 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 2
- 238000013527 convolutional neural network Methods 0.000 claims description 2
- 230000000877 morphologic effect Effects 0.000 claims description 2
- 238000012805 post-processing Methods 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 4
- 210000004027 cell Anatomy 0.000 description 15
- 230000008569 process Effects 0.000 description 7
- 230000001575 pathological effect Effects 0.000 description 6
- 238000003745 diagnosis Methods 0.000 description 4
- 238000012795 verification Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000007499 fusion processing Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 206010041349 Somnolence Diseases 0.000 description 1
- 238000012896 Statistical algorithm Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000010827 pathological analysis Methods 0.000 description 1
- 230000007170 pathology Effects 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
- G06V10/763—Non-hierarchical techniques, e.g. based on statistics of modelling distributions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20152—Watershed segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Geometry (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a cell nucleus segmentation method based on a central point and related equipment, wherein the method comprises the following steps: clustering the cell nucleus image based on a pseudo tag generation algorithm of the unsupervised clustering to generate an example pixel-level pseudo tag required by training a cell nucleus segmentation network model; after the three-classification pseudo tag instantiation result is obtained, training a cell nucleus segmentation network model by adopting an instance segmentation frame which is not dependent on a boundary frame; training a nuclear central point detection network, and obtaining a predicted nuclear central point by taking a local maximum value and filtering operation; and fusing and cutting the cell nucleus segmentation result and the detection result to obtain a corrected cell nucleus instance segmentation result. According to the invention, the pseudo tag is processed into the classifying chart of three classifications of kernel-contour-background, so that the network can pay attention to the contour of the cell nucleus better, the central point of the cell nucleus is predicted, and the watershed algorithm is used for processing the classifying result, thereby improving the instantiation effect of the adhered cell nucleus.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method, a system, a terminal, and a computer readable storage medium for cell nucleus segmentation based on a center point.
Background
Diagnosis of medical results by means of medical images such as pathological images is one of the diagnostic means frequently used in modern medicine. Under a microscope, a professional pathologist can judge the condition of a patient by observing or counting the number, shape, size, distribution and other characteristics of cells or cell nuclei in a pathological image. However, many thousands of cells are usually found in a common pathological section, and this manual reading and diagnosis process is usually very time-consuming and labor-consuming, and the related observations and statistics are often very susceptible to interference from human factors. In an actual diagnostic scenario, the pathologists are relatively prone to drowsiness, and even error, due to prolonged under-the-lens and excessive attention loss. In addition, manual film reading is also easy to introduce personal subjectivity to doctors due to the influence of film reading experience and other factors, so that the diagnosis result is not objective enough. Therefore, based on technologies such as computer vision (computer vision), a robust and efficient pathological image slice cell related information auxiliary statistical algorithm is researched, and a film reading system for computer auxiliary diagnosis is constructed, so that the method has very important practical significance.
The division of the nucleus (nuclei segmentation) is a very important component of the above-described computer-aided medical diagnostic system. Cell nucleus segmentation can extract many useful features about cells in pathological images, and has important value for nuclear morphology measurement, computational pathological analysis, and the like.
Conventional cell nucleus segmentation algorithms often require time-consuming manual feature extraction, even requiring very specialized domain knowledge, and often have difficulty achieving very desirable results in very complex and varied pathology images. In recent years, with the development of deep learning, a deep network-based automatic cell nucleus segmentation algorithm has been widely studied and developed. However, in order to train a good nucleus segmentation network, various fully supervised segmentation algorithms generally require very fine pixel-level labeling, i.e. require a great deal of manpower and material resources to ask a professional pathologist to make fine manual labeling. Therefore, how to perform cell nucleus segmentation on the basis of a weaker-level labeling has very important research significance.
In addition, in the common pathological image of the cell nucleus, a phenomenon that adjacent cell nuclei are adhered to each other usually occurs, and a problem that the edges of the cell nuclei are blurred often occurs, which also brings great challenges to the model for performing cell verification example level segmentation.
Accordingly, the prior art is still in need of improvement and development.
Disclosure of Invention
The invention mainly aims to provide a cell nucleus segmentation method, a system, a terminal and a computer readable storage medium based on a central point, which aim to solve the problems that a great amount of manpower and material resources are consumed when cell nuclei are segmented in the prior art, a professional pathologist is required to carry out fine manual labeling, and adjacent cell nuclei are adhered to each other.
In order to achieve the above object, the present invention provides a center-point-based cell nucleus segmentation method comprising the steps of:
clustering the cell nucleus image based on a pseudo tag generation algorithm of the unsupervised clustering to generate an example pixel-level pseudo tag required by training a cell nucleus segmentation network model;
after the three-classification pseudo tag instantiation result is obtained, training a cell nucleus segmentation network model by adopting an instance segmentation frame which is not dependent on a boundary frame;
training a nuclear central point detection network, and obtaining a predicted nuclear central point by taking a local maximum value and filtering operation;
and fusing and cutting the cell nucleus segmentation result and the detection result to obtain a corrected cell nucleus instance segmentation result.
Optionally, the center-based cell nucleus segmentation method, wherein,
in addition, to achieve the above object, the present invention also provides a center-point-based cell nucleus segmentation system, wherein the center-point-based cell nucleus segmentation system comprises:
the pseudo tag generation module is used for clustering the cell nucleus image based on a pseudo tag generation algorithm of the unsupervised clustering, and generating example pixel-level pseudo tags required by training of a cell nucleus segmentation network model;
the cell nucleus segmentation module is used for training a cell nucleus segmentation network model by adopting an example segmentation frame which is not dependent on a boundary frame after the three-classification pseudo tag instantiation result is obtained;
the cell nucleus central point detection module is used for training a cell nucleus central point detection network and obtaining a predicted cell nucleus central point through taking a local maximum value and filtering operation;
and the cell nucleus instantiation module is used for fusing and cutting the segmentation result and the detection result of the cell nucleus to obtain a corrected cell nucleus instance segmentation result.
In addition, to achieve the above object, the present invention also provides a terminal, wherein the terminal includes: the cell core segmentation method comprises the steps of a memory, a processor and a cell core segmentation program which is stored on the memory and can run on the processor and is based on the central point, wherein the cell core segmentation program based on the central point realizes the cell core segmentation method based on the central point when being executed by the processor.
In addition, in order to achieve the above object, the present invention also provides a computer-readable storage medium storing a center-point-based cell nucleus segmentation program which, when executed by a processor, implements the steps of the center-point-based cell nucleus segmentation method as described above.
In the invention, a pseudo label generation algorithm based on unsupervised clustering clusters the cell nucleus image to generate an example pixel-level pseudo label required by training a cell nucleus segmentation network model; after the three-classification pseudo tag instantiation result is obtained, training a cell nucleus segmentation network model by adopting an instance segmentation frame which is not dependent on a boundary frame; training a nuclear central point detection network, and obtaining a predicted nuclear central point by taking a local maximum value and filtering operation; and fusing and cutting the cell nucleus segmentation result and the detection result to obtain a corrected cell nucleus instance segmentation result. According to the invention, the pseudo tag is processed into the classifying chart of three classifications of kernel-contour-background, so that the network can pay attention to the contour of the cell nucleus better, different cell nucleus examples can be distinguished, the cell nucleus central point is predicted by using an additional central point detection network, and the classification result is processed by using a watershed algorithm, so that the adhesion cell nucleus instantiation effect is improved.
Drawings
FIG. 1 is a flow chart of a preferred embodiment of the center-point based cell nucleus segmentation method of the present invention;
FIG. 2 is a schematic diagram of a general framework of a center point supervision-based cell nucleus segmentation algorithm in a preferred embodiment of the center point-based cell nucleus segmentation method of the present invention;
FIG. 3 is a schematic flow chart of pseudo tag generation based on unsupervised clustering in a preferred embodiment of the center point-based cell nucleus segmentation method of the present invention;
FIG. 4 is a flow chart of pseudo-tag cell nucleus instantiation based on watershed algorithm in a preferred embodiment of the center-point based cell nucleus segmentation method of the present invention;
FIG. 5 is a flow chart of training a pseudo-tag-based cell nucleus segmentation model in a preferred embodiment of the center-point-based cell nucleus segmentation method of the present invention;
FIG. 6 is a diagram showing training of a cell nucleus center point detection model in a preferred embodiment of the center point-based cell nucleus segmentation method of the present invention;
FIG. 7 is a schematic diagram showing the detection of a nuclear center point detection model in a preferred embodiment of the center point-based nuclear segmentation method of the present invention;
FIG. 8 is a schematic diagram showing a fusion process of a segmentation result and a detection result in a preferred embodiment of the center-point-based cell nucleus segmentation method of the present invention;
FIG. 9 is a schematic diagram of a preferred embodiment of the center-point based nuclear segmentation system of the present invention;
FIG. 10 is a schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more clear and clear, the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
For the core-point supervised cell-core segmentation task, the labels provided by the training stage can only be example-level core point labels of the cell cores, and cannot be pixel-level labels. However, in the test phase, a model is required to be able to predict the results of the cell nuclei at the instance level, pixel level. As shown in fig. 2, the method is based on an unsupervised clustering pseudo tag generation algorithm, and is used for generating an example pixel-level pseudo tag required during cell nucleus segmentation network model training, and then training a cell nucleus example segmentation network by using the generated pseudo tag.
And for the input nuclear original image, performing foreground and background unsupervised clustering by using a K-means clustering algorithm, and judging clusters with more overlapping areas with nuclear center point marks in unsupervised clustered clusters as nuclear prospects. And then, the center point mark of the cell nucleus is used again to instantiate the cell nucleus clustering result. In order to enable the network model to better distinguish different cell examples, the method processes the pseudo tag into a classification chart of three classifications of kernel-contour-background, so that the network can pay attention to the contour of the cell nucleus better and distinguish different cell nucleus examples.
The cell nucleus segmentation network obtained based on the generated pseudo tag training still has more cell nucleus adhesion problems when predicting. In order to further divide some adhered cell nuclei, the final cell nucleus division example level performance is improved, an additional center point detection network is used for predicting the center point of the cell nuclei, and then a watershed algorithm is used for processing the division results again, so that the adhered cell nucleus instantiation effect is improved.
In the center-based cell nucleus segmentation method according to the preferred embodiment of the present invention, as shown in fig. 1, the center-based cell nucleus segmentation method includes the following steps:
step S10, clustering the cell nucleus images by using a pseudo tag generation algorithm based on unsupervised clustering, and generating example pixel-level pseudo tags required by training a cell nucleus segmentation network model.
Specifically, the invention relates to a pseudo tag generation algorithm based on unsupervised clustering, which is used for generating an example pixel-level pseudo tag required by the training of a cell nucleus segmentation network model, and the specific algorithm flow is shown in figure 3. In fig. 3, the visualized graph marked by the center points is thickened, and in the real marking, each center point occupies only one pixel, and the visualized graph marked by the center point is the same.
Firstly, performing unsupervised clustering on the front background of an input nuclear original image by using a K-means clustering algorithm, and judging the cluster with a large number of overlapping areas with nuclear center point marks as a nuclear foreground (such as the upper left corner in fig. 3, two black and white clusters, which overlapping center points are the foreground and the background); at this time, the result of the cell nucleus clustering is only semantic information at the pixel level, and there is no information at the cell nucleus instance level (white is the cell nucleus foreground after clustering, but how many cells in the foreground are unknown, and the instantiation refers to dividing each different cell nucleus in the foreground). And then, the center point mark of the cell nucleus is used for carrying out instantiation processing on the cell nucleus clustering result, so that in order to enable a network model to better distinguish different cell instances, the pseudo tag is processed into a classification chart of three classifications of kernel-contour-background, thereby enabling the network to better pay attention to the contour of the cell nucleus, and finally, distinguishing different cell nucleus instances better.
The invention uses the center point mark of the cell nucleus to segment the result of the cell nucleus foreground clustering, thereby dividing the cell nucleus foreground clustering result with only pixel-level semantic information into independent cell nucleus examples, and the specific flow is shown in figure 4. By means of the method for instantiating the clustering result, the instance-level information in the original nucleus central point labeling can be fused into the processed pseudo tag, and the generated pseudo tag can also contain more rough outline information about adjacent nuclei.
If the nuclear central point labeling chart is C H×W Wherein H and W are the height and width of the nuclear artwork, C H×W Element c of (3) ij (i=1, 2,.; j=1, 2,..w) takes a value of 1 where there is a nuclear center point marker and 0 where there is no nuclear center point marker.
First, labeling a center point of a cell nucleus with a graph C H×W Performing inversion operation to obtain a center point labeling inversion diagram with the same resolution as the processed image:
wherein I is H×W The values of the elements are all 1, for
For example, a pixel with a value of 1 is defined as a foreground, and a pixel with a value of 0 is defined as a background.
Then, the center point is marked and inverted to obtain a corresponding center point, wherein the calculation formula corresponding to each element is as follows:
wherein the method comprises the steps of，
Is->
F (m, n) is the euclidean distance between two pixels.
The center point distance map D obtained above H×W In the method, the pixel values of the areas nearby the cell nucleus central point marks are relatively low, and the pixel values far away from each central point mark are relatively high. Thus, those areas with higher pixel values have a higher likelihood of being the boundary of the nucleus, which may provide some rough nucleus boundary information for the watershed algorithm.
The nuclear prospect binarization map obtained by the clustering is F H×W F is to F H×W Distance map D from center point H×W Fusing to obtain a foreground-distance graph:
wherein, the liquid crystal display device comprises a liquid crystal display device,
representing the multiplication of the matrix corresponding elements.
Then, an instantiation segmentation result W is obtained by using a watershed algorithm H×W ：
W H×W ＝w(D′ H×W ，C H×W )；
Where w (·) represents the watershed algorithm.
And finally, carrying out hole filling post-treatment on each example to obtain a final pseudo tag instantiation result.
And step S20, after the three-classification pseudo tag instantiation result is obtained, training a cell nucleus segmentation network model by adopting an instance segmentation frame which is not dependent on a boundary frame.
Specifically, after the three-class pseudo tag is obtained, training of the cell verification instance segmentation network can be performed. The present invention employs a non-bounding box dependent instance segmentation framework that does not need to rely on a cell nucleus bounding box, but rather performs cell verification instance segmentation directly from a pixel level prediction perspective.
First, a full convolutional neural network architecture U-Net based on ResNet-50 is used to learn the mapping between the input nuclear pictures to three-class pseudo tags, as shown in FIG. 5. For each input nuclear artwork, the output of the nuclear segmentation network model is a three-way diagram with the same spatial resolution as the nuclear artwork, wherein each channel corresponds to each category of the three-category pseudo tag, namely the kernel, the outline and the background. In training the fully convoluted cell nucleus segmentation network described above, the present invention uses cross-entropy loss (cross-entopy loss).
In the prediction stage, predicting the nucleus kernels, the outlines and the backgrounds in the nucleus test image by utilizing the full convolution neural network trained in the mode, and reserving all the predicted nucleus kernels; and carrying out instantiation treatment on the cell nucleus according to whether the areas are communicated, and carrying out morphological expansion operation on the cell nucleus after instantiation to obtain a division result of the cell nucleus at the instance level.
And step S30, training a cell nucleus central point detection network, and obtaining a predicted cell nucleus central point through taking a local maximum value and filtering operation.
Specifically, the present invention uses the concept of regression to perform nuclear central point detection. Since the nuclear central point information is a few relatively sparse coordinate values, it is very difficult to learn it directly using a neural network. Therefore, it is often necessary to pre-process these coordinate values to facilitate learning by the network.
As shown in fig. 6, first, a gaussian convolution kernel is convolved with a center point of a cell nucleus to obtain a distribution probability map of the center point of the cell nucleus. Specifically, map C is labeled on the center point of the cell nucleus H×W Expressed in terms of delta function for pixel x therein:
wherein Q (x) is represented by delta function form of x, x represents each pixel in the nuclear center point label, x i Representing the values in the set of center points a, a= { a 1 ，a 2 ，…，a N The } represents a set of N nuclear center point labels corresponding to the nuclear center point label graph,
gaussian kernel G using fixed propagation parameter (spread parameter) σ σ Convolving with Q (x) to obtain a distribution probability result graph of the center point of the cell nucleus:
where x represents the convolution operation.
The obtained distribution probability result graph is a picture with dimension of H multiplied by W, and is defined as U H×W The distribution probability result graph can be used as a target of network learning for nuclear central point detection.
If the original picture input into the nuclear central point detection network is O H×W The central point distribution prediction probability map obtained after the network N theta (·) is detected through the central point of the cell nucleus is M H×W ＝N θ (O H×W ) Wherein θ is a parameter in the nuclear core central point detection network;
to constrain the learning of the nuclear central point detection network, a Euclidean distance-based loss function is used:
wherein u is ij Represents the predicted value, m ij Representing the actual value.
In the aspects of image feature extraction and network structure, the invention uses U-Net based on ResNet-50 to learn the mapping relation between the input image and the distribution probability map of the center point of the cell nucleus. After training the nuclear central point detection network based on the method, in the detection stage, the predicted nuclear central point can be obtained by taking the local maximum value and filtering operation, and the flow is shown in figure 7.
Specifically, if the center point distribution probability map predicted by the nuclear center point detection network is E H×W For E H×W Taking the maximum value of the local probability to obtain a preliminary cell nucleus central point position diagram:
E′ H×W ＝l Δ （E H×W ）；
wherein l Δ The method is characterized in that the position corresponding to the local maximum value is calculated according to the set minimum pixel distance delta between the maximum values, the probability value of the pixel points of the local maximum value is kept unchanged, and the probability values of the rest pixel points are all set to 0.
Because some noise possibly exists in the network prediction result, a threshold tau is set in the specific implementation process, and the local maximum value with the probability value lower than the threshold tau in the prediction result is filtered, so that a final cell nucleus central point position diagram is obtained: the maximum probability value can be used for obtaining a preliminary cell nucleus central point position diagram:
E″ H×W ＝S τ （E′ H×W ）；
wherein s is τ (. Cndot.) indicates that the corresponding values of all pixels having pixel values greater than τ are set to 1, and the remaining pixel values remain at 0.
And S40, fusing and cutting the cell nucleus segmentation result and the detection result to obtain a corrected cell nucleus instance segmentation result.
Specifically, the cell nucleus segmentation is essentially an instance segmentation task, different target instances need to be paid attention to, post-processing operation is added on the basis of segmentation results and detection results, fusion and cutting processing are carried out on the segmentation results and the detection results, and finally corrected cell nucleus instance segmentation results are obtained.
The invention adopts a watershed algorithm when the adhered cell nucleus is instantiated, and the fusion process of the cell nucleus segmentation result and the detection result is shown in figure 8:
if the cell nucleus is divided into netIn the three channels of the channel output, the predictive probability diagrams of the background, the kernel and the boundary are J respectively H×W 、K H×W And L H×W The method comprises the steps of carrying out a first treatment on the surface of the Pair J H×W 、K H×W And L H×W The three-channel matrix formed by the method is subjected to Argmax operation on the channel dimension to obtain a category matrix of three categories of background, kernel and boundary; taking the coordinates corresponding to the kernel class and forming a kernel class segmentation result graph L H×W Wherein the pixel value corresponding to the nuclear core region is 1, and the pixel value corresponding to the remaining region is 0. In this core segmentation result, there will still be some stuck core cores. Thus, the present invention utilizes the boundary prediction probability map L H×W The boundary probability information in (a) further divides the adhered cell nucleus; specifically, the boundary prediction probability map L H×W And binarized kernel segmentation result S H×W Fusing to obtain a kernel-boundary probability diagram:
probability map L 'after fusion' H×W The method comprises the steps of simultaneously containing segmentation information of cell nuclei and boundary prediction probability information of the cell nuclei, so that a watershed segmentation algorithm is applied to the cell nuclei for further segmentation; in order to further fuse the example level information of the nuclear central point detection, the result E' of the nuclear detection is adopted H×W As a seed for watershed algorithms. Thus, the result of the exemplary cut after the watershed cutting post-treatment can be obtained:
S″ H×W ＝w(S′ H×W ,E″ H×W )；
performing instance-level hole filling on the instantiation cutting result to obtain three-classification pseudo tags:
H H×W ＝h(S″ H×W )；
finally, carrying out post-treatment on the instantiation segmentation result by using an instance-level expansion operation d (-), so as to obtain a final cell nucleus instance segmentation result:
Z H×W ＝d(H H×W )。
according to the invention, on the central point marking information of the cell nucleus, the foreground of the cell nucleus is divided into different cell nucleus examples by using an unsupervised algorithm, so that more cell nucleus examples and outline information are provided for training of a cell nucleus segmentation model; an adhesion cell nucleus instantiation method for further processing the segmentation result by using the predicted cell nucleus central point information is provided.
Further, as shown in fig. 9, based on the above-mentioned method for dividing a cell nucleus based on a central point, the present invention further provides a cell nucleus dividing system based on a central point, where the cell nucleus dividing system based on a central point includes:
the pseudo tag generation module 51 is configured to cluster the cell nucleus image based on a pseudo tag generation algorithm of the unsupervised clustering, and generate an example pixel-level pseudo tag required during training of the cell nucleus segmentation network model;
the cell nucleus segmentation module 52 is configured to perform training of a cell nucleus segmentation network model by using an instance segmentation framework that is not dependent on a bounding box after obtaining a three-class pseudo tag instantiation result;
the cell nucleus central point detection module 53 is used for training a cell nucleus central point detection network, and obtaining a predicted cell nucleus central point by taking a local maximum value and filtering operation;
the cell nucleus instantiation module 54 is configured to fuse and cut the division result and the detection result of the cell nucleus, so as to obtain a corrected division result of the cell nucleus instance.
Further, as shown in fig. 10, based on the above-mentioned method and system for dividing cell nuclei based on a central point, the present invention further provides a terminal correspondingly, which includes a processor 10, a memory 20 and a display 30. Fig. 10 shows only some of the components of the terminal, but it should be understood that not all of the illustrated components are required to be implemented and that more or fewer components may alternatively be implemented.
The memory 20 may in some embodiments be an internal storage unit of the terminal, such as a hard disk or a memory of the terminal. The memory 20 may in other embodiments also be an external storage device of the terminal, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the terminal. Further, the memory 20 may also include both an internal storage unit and an external storage device of the terminal. The memory 20 is used for storing application software installed in the terminal and various data, such as program codes of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, the memory 20 stores a center-point-based cell nucleus segmentation program 40, and the center-point-based cell nucleus segmentation program 40 is executable by the processor 10 to implement the center-point-based cell nucleus segmentation method in the present application.
The processor 10 may in some embodiments be a central processing unit (Central Processing Unit, CPU), microprocessor or other data processing chip for executing program code or processing data stored in the memory 20, for example performing the center point based cell nucleus segmentation method or the like.
The display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like in some embodiments. The display 30 is used for displaying information at the terminal and for displaying a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.
In an embodiment, the step of center point based cell nucleus segmentation is accomplished when the processor 10 executes the center point based cell nucleus segmentation program 40 in the memory 20.
The present invention also provides a computer-readable storage medium storing a center-point based cell nucleus segmentation program which, when executed by a processor, implements the steps of the center-point based cell nucleus segmentation method as described above.
In summary, the present invention provides a method for dividing a nucleus based on a central point and related equipment, the method comprising: clustering the cell nucleus image based on a pseudo tag generation algorithm of the unsupervised clustering to generate an example pixel-level pseudo tag required by training a cell nucleus segmentation network model; after the three-classification pseudo tag instantiation result is obtained, training a cell nucleus segmentation network model by adopting an instance segmentation frame which is not dependent on a boundary frame; training a nuclear central point detection network, and obtaining a predicted nuclear central point by taking a local maximum value and filtering operation; and fusing and cutting the cell nucleus segmentation result and the detection result to obtain a corrected cell nucleus instance segmentation result. According to the invention, the pseudo tag is processed into the classifying chart of three classifications of kernel-contour-background, so that the network can pay attention to the contour of the cell nucleus better, different cell nucleus examples can be distinguished, the cell nucleus central point is predicted by using an additional central point detection network, and the classification result is processed by using a watershed algorithm, so that the adhesion cell nucleus instantiation effect is improved.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or terminal comprising the element.
Of course, those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by a computer program for instructing relevant hardware (e.g., processor, controller, etc.), the program may be stored on a computer readable storage medium, and the program may include the above described methods when executed. The computer readable storage medium may be a memory, a magnetic disk, an optical disk, etc.
It is to be understood that the invention is not limited in its application to the examples described above, but is capable of modification and variation in light of the above teachings by those skilled in the art, and that all such modifications and variations are intended to be included within the scope of the appended claims.
Claims (13)
1. A center-point-based cell nucleus segmentation method, comprising:
clustering the cell nucleus image based on a pseudo tag generation algorithm of the unsupervised clustering to generate an example pixel-level pseudo tag required by training a cell nucleus segmentation network model;
after the three-classification pseudo tag instantiation result is obtained, training a cell nucleus segmentation network model by adopting an instance segmentation frame which is not dependent on a boundary frame;
training a nuclear central point detection network, and obtaining a predicted nuclear central point by taking a local maximum value and filtering operation;
and fusing and cutting the cell nucleus segmentation result and the detection result to obtain a corrected cell nucleus instance segmentation result.
2. The method for cell nucleus segmentation based on the central point according to claim 1, wherein the pseudo tag generation algorithm based on the unsupervised clustering clusters cell nucleus images to generate example pixel-level pseudo tags required for training a cell nucleus segmentation network model, and specifically comprises the following steps:
performing foreground and background unsupervised clustering on the input nuclear original image by using a K-means clustering algorithm, and judging clusters with a large number of overlapping areas with nuclear center point marks in unsupervised clustered clusters as nuclear prospects;
and (3) carrying out instantiation processing on the cell nucleus clustering result by using the center point mark of the cell nucleus, and processing the example pixel-level pseudo tag into a classification chart of three classifications of kernel-contour-background.
3. The method for cell nucleus segmentation based on the center point according to claim 2, wherein the cell nucleus clustering result is instantiated by using the center point label of the cell nucleus, and example pixel-level pseudo tags are processed into a classification chart of three classifications of kernel-contour-background, and the method specifically comprises the following steps:
if the nuclear central point labeling chart is C H×W Wherein H and W are the height and width of the nuclear artwork, C H×W Element c of (3) ij (i=1, 2,.; j=1, 2,..w) takes a value of 1 where there is a nuclear central point marker and 0 where there is no nuclear central point marker;
labeling C for center point of cell nucleus H×W Performing inversion operation to obtain a center point labeling inversion diagram with the same resolution as the processed image:
wherein I is H×W The values of the elements are all 1, for
Defining a pixel with a value of 1 as a foreground and a pixel with a value of 0 as a background;
and (3) marking the center point and inverting to obtain a corresponding center point, wherein the calculation formula corresponding to each element is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,
is->
F (m, n) is the euclidean distance between two pixels;
the nuclear prospect binarization map obtained by clustering is F H×W Will beF H×W Distance map D from center point H×W Fusing to obtain a foreground-distance graph:
wherein, the liquid crystal display device comprises a liquid crystal display device,
representing the multiplication of matrix corresponding elements;
obtaining an instantiation segmentation result W by using a watershed algorithm H×W ：
W H×W ＝w(D' H×W ，C H×W )；
Wherein w (·) represents a watershed algorithm;
and carrying out hole filling post-treatment on each example to obtain a final pseudo tag instantiation result.
4. The method of claim 3, wherein the distance map D is a distance map D at the center point H×W In the method, the pixel value of the center point of the cell nucleus, which is far away from each center point, is higher than that of the surrounding area of the label.
5. A method of center-based nuclear segmentation according to claim 3, wherein the watershed algorithm is used to provide rough nuclear boundary information.
6. The method for cell nucleus segmentation based on the central point according to claim 3, wherein after the three-classification pseudo tag instantiation result is obtained, training a cell nucleus segmentation network model by using an instance segmentation framework which is not dependent on a bounding box, specifically comprising:
mapping between the input cell nucleus picture and the three-class pseudo tag by using a full convolution neural network architecture U-Net based on ResNet-50;
for each input nuclear artwork, the output of the nuclear segmentation network model is a three-way diagram with the same spatial resolution as the nuclear artwork, wherein each channel corresponds to each category of three-category pseudo tags respectively;
in the prediction stage, predicting the nucleus kernels, the outlines and the backgrounds in the nucleus test image by utilizing the fully-convolutional neural network obtained by training, and reserving all the predicted nucleus kernels;
and carrying out instantiation treatment on the cell nucleus according to whether the areas are communicated, and carrying out morphological expansion operation on the cell nucleus after instantiation to obtain a division result of the cell nucleus at the instance level.
7. The center-based nuclear segmentation method according to claim 6, wherein the nuclear segmentation network model is trained using cross entropy loss.
8. The method for cell nucleus segmentation based on the central point according to claim 6, wherein the training of the cell nucleus central point detection network, obtaining the predicted cell nucleus central point by taking the local maximum and the filtering operation, specifically comprises:
convolving the Gaussian convolution kernel with the center point of the cell nucleus to obtain a distribution probability map of the center point of the cell nucleus;
labeling C for center point of cell nucleus H×W Expressed in terms of delta function for pixel x therein:
wherein Q (x) is represented by delta function form of x, x represents each pixel in the nuclear center point label, x i Representing the values in the set of center points a, a= { a 1 ，a 2 ，…，a N The } represents a set of N nuclear center point labels corresponding to the nuclear center point label graph,
gaussian using fixed propagation parameter sigmaCore G σ Convolving with Q (x) to obtain a distribution probability result graph of the center point of the cell nucleus:
wherein, represents convolution operation;
the obtained distribution probability result graph is a picture with dimension H multiplied by W and is defined as U H×W The distribution probability result graph is used as a target for detecting network learning at a cell nucleus central point;
if the original picture input into the nuclear central point detection network is O H×W Network N is detected through nuclear central point θ The central point distribution prediction probability map obtained after (-) is M H×W ＝N θ (O H×W ) Wherein θ is a parameter in the nuclear core central point detection network;
learning of the nuclear central point detection network is constrained using a Euclidean distance-based loss function:
wherein u is ij Represents the predicted value, m ij Representing the actual value;
the U-Net based on ResNet-50 is used for learning the mapping relation between the input image and the distribution probability map of the center point of the cell nucleus;
after training the nuclear central point detection network, obtaining a predicted nuclear central point by taking a local maximum value and filtering operation in a detection stage.
9. The method for cell nucleus segmentation based on the central point according to claim 8, wherein after training the cell nucleus central point detection network, obtaining the predicted cell nucleus central point by taking a local maximum value and a filtering operation in the detection stage, specifically comprising:
if the center point of the cell nucleus is examinedThe central point distribution probability diagram obtained by network prediction is E H×W For E H×W Taking the maximum value of the local probability to obtain a preliminary cell nucleus central point position diagram:
E′ H×W ＝l Δ (E H×W )；
wherein l Δ Calculating the position corresponding to the local maximum value according to the minimum pixel distance delta between the set maximum values, keeping the probability value of the pixel points of the local maximum value unchanged, and setting the probability values of the rest pixel points to be 0;
setting a threshold tau, and filtering out a local maximum value with the probability value lower than the threshold tau in the prediction result to obtain a final nuclear central point position diagram: the maximum probability value can be used for obtaining a preliminary cell nucleus central point position diagram:
E″ H×W =s τ (E′ H×W )；
wherein s is τ (. Cndot.) indicates that the corresponding values of all pixels having pixel values greater than τ are set to 1, and the remaining pixel values remain at 0.
10. The method for dividing cell nuclei based on the center point according to claim 9, wherein the steps of fusing and cutting the division result and the detection result of the cell nuclei to obtain corrected division result of cell nuclei instance comprise:
if the predicted probability diagrams of the background, the kernel and the boundary in the three channels output by the cell nucleus segmentation network are J respectively H×W 、K H×W And L H×W ；
Pair J H×W 、K H×W And L H×W The three-channel matrix formed by the method is subjected to Argmax operation on the channel dimension to obtain a category matrix of three categories of background, kernel and boundary;
taking the coordinates corresponding to the kernel category and forming a kernel category segmentation result graph S H×W Wherein the pixel value corresponding to the nuclear core region is 1, and the pixel value corresponding to the remaining region is 0;
using boundary prediction probability map L H×W Boundary probability information in (a)Dividing the adhered cell nucleus;
predicting the boundary into a probability map L H×W And binarized kernel segmentation result S H×W Fusing to obtain a kernel-boundary probability diagram:
probability map L 'after fusion' H×W The method comprises the steps of simultaneously containing segmentation information of cell nuclei and boundary prediction probability information of the cell nuclei, and carrying out segmentation by applying a watershed segmentation algorithm;
results of nuclear detection E H×W As a seed for the watershed algorithm, an instantiated cut result after watershed cut post-treatment is obtained:
S″ H×W ＝w(S′ H×W ，E″ H×W )；
performing instance-level hole filling on the instance cutting result to obtain three-classification pseudo tags:
H H×W ＝h(S″ H×W )；
post-processing the instantiation segmentation result by using an instance-level expansion operation d (·) to obtain a final cell nucleus instance segmentation result:
Z H×W ＝d(H H×W )。
11. a center-based nuclear segmentation system, the center-based nuclear segmentation system comprising:
the pseudo tag generation module is used for clustering the cell nucleus image based on a pseudo tag generation algorithm of the unsupervised clustering, and generating example pixel-level pseudo tags required by training of a cell nucleus segmentation network model;
the cell nucleus segmentation module is used for training a cell nucleus segmentation network model by adopting an example segmentation frame which is not dependent on a boundary frame after the three-classification pseudo tag instantiation result is obtained;
the cell nucleus central point detection module is used for training a cell nucleus central point detection network and obtaining a predicted cell nucleus central point through taking a local maximum value and filtering operation;
and the cell nucleus instantiation module is used for fusing and cutting the segmentation result and the detection result of the cell nucleus to obtain a corrected cell nucleus instance segmentation result.
12. A terminal, the terminal comprising: a memory, a processor and a center-point based cell nucleus segmentation program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the center-point based cell nucleus segmentation method according to any one of claims 1-10.
13. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a center-point based cell nucleus segmentation program, which when executed by a processor, implements the steps of the center-point based cell nucleus segmentation method according to any of claims 1-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211705880.3A CN116433704A (en) | 2022-12-29 | 2022-12-29 | Cell nucleus segmentation method based on central point and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211705880.3A CN116433704A (en) | 2022-12-29 | 2022-12-29 | Cell nucleus segmentation method based on central point and related equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116433704A true CN116433704A (en) | 2023-07-14 |
Family
ID=87086100
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211705880.3A Pending CN116433704A (en) | 2022-12-29 | 2022-12-29 | Cell nucleus segmentation method based on central point and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116433704A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117291941A (en) * | 2023-10-16 | 2023-12-26 | 齐鲁工业大学(山东省科学院) | Cell nucleus segmentation method based on boundary and central point feature assistance |
| CN117710969A (en) * | 2024-02-05 | 2024-03-15 | 安徽大学 | Cell nucleus segmentation and classification method based on deep neural network |
-
2022
- 2022-12-29 CN CN202211705880.3A patent/CN116433704A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117291941A (en) * | 2023-10-16 | 2023-12-26 | 齐鲁工业大学(山东省科学院) | Cell nucleus segmentation method based on boundary and central point feature assistance |
| CN117710969A (en) * | 2024-02-05 | 2024-03-15 | 安徽大学 | Cell nucleus segmentation and classification method based on deep neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10410353B2 (en) | Multi-label semantic boundary detection system | |
| JP6710135B2 (en) | Cell image automatic analysis method and system | |
| JP5174040B2 (en) | Computer-implemented method for distinguishing between image components and background and system for distinguishing between image components and background | |
| CN111027493B (en) | Pedestrian detection method based on deep learning multi-network soft fusion | |
| Loog et al. | Segmentation of the posterior ribs in chest radiographs using iterated contextual pixel classification | |
| JP6262748B2 (en) | Biological unit identification based on supervised shape ranking | |
| Nguyen et al. | Gaussian-mixture-model-based spatial neighborhood relationships for pixel labeling problem | |
| US10121245B2 (en) | Identification of inflammation in tissue images | |
| CN116433704A (en) | Cell nucleus segmentation method based on central point and related equipment | |
| CN112116599B (en) | Sputum smear tubercle bacillus semantic segmentation method and system based on weak supervised learning | |
| CN110222582B (en) | Image processing method and camera | |
| US20210374455A1 (en) | Utilizing machine learning and image filtering techniques to detect and analyze handwritten text | |
| Dimitropoulos et al. | Automated detection and classification of nuclei in pax5 and H&E-stained tissue sections of follicular lymphoma | |
| WO2024016812A1 (en) | Microscopic image processing method and apparatus, computer device, and storage medium | |
| Liang et al. | A supervised figure-ground segmentation method using genetic programming | |
| Li et al. | Contour extraction of drosophila embryos | |
| CN116844143A (en) | Embryo development stage prediction and quality assessment system based on edge enhancement | |
| CN111627033B (en) | Method, equipment and computer readable storage medium for dividing difficult sample instance | |
| CN114118127A (en) | Visual scene mark detection and identification method and device | |
| Pai et al. | Characterization of errors in deep learning-based brain MRI segmentation | |
| Zhao et al. | Traffic sign detection based on faster R-CNN in scene graph | |
| CN113112475B (en) | Traditional Chinese medicine ear five-organ region segmentation method and device based on machine learning | |
| CN116580397B (en) | Pathological image recognition method, device, equipment and storage medium | |
| Wang et al. | Deep conditional neural network for image segmentation | |
| CN113808068A (en) | Image detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |