CN112446892A - Cell nucleus segmentation method based on attention learning - Google Patents
Cell nucleus segmentation method based on attention learning Download PDFInfo
- Publication number
- CN112446892A CN112446892A CN202011272095.4A CN202011272095A CN112446892A CN 112446892 A CN112446892 A CN 112446892A CN 202011272095 A CN202011272095 A CN 202011272095A CN 112446892 A CN112446892 A CN 112446892A
- Authority
- CN
- China
- Prior art keywords
- attention
- segmentation
- learning
- cell nucleus
- loss
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 210000003855 cell nucleus Anatomy 0.000 title claims abstract description 54
- 230000011218 segmentation Effects 0.000 title claims abstract description 50
- 238000000034 method Methods 0.000 title claims abstract description 44
- 210000004027 cell Anatomy 0.000 claims abstract description 15
- 210000004940 nucleus Anatomy 0.000 claims abstract description 12
- 238000012549 training Methods 0.000 claims description 24
- 239000003550 marker Substances 0.000 claims description 23
- 230000007246 mechanism Effects 0.000 claims description 18
- 239000011159 matrix material Substances 0.000 claims description 14
- 238000010586 diagram Methods 0.000 claims description 10
- 238000005070 sampling Methods 0.000 claims description 8
- 230000006870 function Effects 0.000 claims description 4
- 238000002372 labelling Methods 0.000 claims description 4
- 230000003902 lesion Effects 0.000 claims description 4
- 201000003379 Townes-Brocks syndrome Diseases 0.000 claims description 3
- 239000000654 additive Substances 0.000 claims description 3
- 230000000996 additive effect Effects 0.000 claims description 3
- 201000010099 disease Diseases 0.000 claims description 3
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims description 3
- 238000003745 diagnosis Methods 0.000 claims description 2
- 238000003708 edge detection Methods 0.000 claims 1
- 238000010827 pathological analysis Methods 0.000 abstract description 7
- 238000005516 engineering process Methods 0.000 abstract description 5
- 238000013135 deep learning Methods 0.000 abstract description 3
- 230000002159 abnormal effect Effects 0.000 abstract 1
- 230000001413 cellular effect Effects 0.000 abstract 1
- 230000008901 benefit Effects 0.000 description 6
- 238000003709 image segmentation Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 206010008342 Cervix carcinoma Diseases 0.000 description 4
- 208000006105 Uterine Cervical Neoplasms Diseases 0.000 description 4
- 201000010881 cervical cancer Diseases 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 239000000284 extract Substances 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 210000005260 human cell Anatomy 0.000 description 2
- 238000009595 pap smear Methods 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 230000002441 reversible effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 206010028980 Neoplasm Diseases 0.000 description 1
- 241000270295 Serpentes Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 201000011510 cancer Diseases 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 210000000805 cytoplasm Anatomy 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000013399 early diagnosis Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 208000015181 infectious disease Diseases 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000007788 liquid Substances 0.000 description 1
- 210000004185 liver Anatomy 0.000 description 1
- 230000000813 microbial effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 231100000915 pathological change Toxicity 0.000 description 1
- 230000036285 pathological change Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000010186 staining Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a cell nucleus segmentation method based on attention learning, and relates to a cell nucleus segmentation problem in intelligent pathological diagnosis. The intelligent pathological diagnosis technology utilizes a deep learning technology to segment and identify abnormal cells in the cell image. However, there are fewer models for segmentation of nuclei in cellular images. There are the following problems: (1) the segmentation accuracy is low because the problems of overlapping nuclei and non-obvious boundaries are not considered. (2) Contextual information of the cell nucleus edge is not considered, so that under segmentation or over segmentation is caused, and the result of subsequent classification is influenced. Therefore, a cell nucleus segmentation method based on attention learning is provided. Experiments show that: the model can effectively solve the problems of segmentation of overlapped cells, under-segmentation or over-segmentation of unclear boundaries and the like. The invention is applied to the nucleus segmentation in the intelligent pathological diagnosis.
Description
Technical Field
The invention relates to intelligent pathological diagnosis, in particular to nuclear segmentation
Background
Cervical cancer is the second largest killer threatening the health of women. One female dies of cervical cancer every two minutes worldwide. Early cervical cancer can be completely cured, so early diagnosis and treatment are effective means for dealing with cancer outbreaks. Liquid-based thin-layer cell detection is the most common cytology inspection technology for cervical cancer internationally at present, and can find partial precancerous lesions and microbial infection. However, the traditional pathological diagnosis completely depends on manual operation and manual reading by visual observation of doctors. But the workload is large, the diagnosis rate is low in accuracy, and large-area screening cannot be popularized. With the development of computer image processing and artificial intelligence technology, automatic pathological diagnosis technology is developed. Cell segmentation is the basis of analysis in this technique. The purpose of cell segmentation is to accurately locate and outline cells, providing input for subsequent processing. However, due to the various human cell nuclei, large color difference, uneven staining and pathological changes of the human cell nuclei, part of the cell nuclei are similar to the color of cytoplasm or background, and the boundary area is not obvious. Moreover, a large number of adhered or overlapped cells also cause the problems of high segmentation difficulty, low precision and easy generation of under-segmentation or over-segmentation.
The current segmentation methods are of two types, one is a traditional segmentation method, and the traditional segmentation method comprises a threshold method, region-based segmentation, undirected graph-based segmentation, active contour segmentation, fuzzy clustering and the like. The threshold method divides the pixel points of the image into a plurality of classes by setting different characteristic thresholds, and has the advantages of simple realization, small calculated amount, stable performance and the like. The threshold method does not utilize the spatial information of pixels, cannot segment a target with large internal variance, and is easy to be interfered by noise. Region-based segmentation, typically using a watershed segmentation method, uses spatial information of the image to classify pixels into regions. In order to improve the watershed segmentation performance, researchers provide methods of self-defining mark points, improving the implementation principle of an algorithm, re-segmenting or merging segmentation results and the like. In addition, the region segmentation method also comprises simple linear iterative clustering, mean shift and other super-pixel segmentation methods. Based on undirected graph segmentation, Graphcut is taken as a representative, the basic idea is to establish a graph, image pixels or superpixels are taken as image vertexes, and the optimization aim is to find a cut so that sub-graphs are not connected. Iterative versions of Graphcut achieve better performance, but such methods suffer from high temporal and spatial complexity. The basic idea of the active contour model is to use a continuous curve to express the target contour, and convert the segmentation process into a process of solving the minimum value of the energy functional. Typical examples of the method include a parametric active contour model represented by a snake model and a geometric active contour model represented by a level set method. Most medical images have ambiguity, the fuzzy clustering method introduces the concept of 'membership' in the fuzzy theory, and pixel points are distributed to areas with high membership, so that the segmentation accuracy is improved.
Another method is a deep learning method, UNet first applies a deep learning technique to medical image segmentation, and advances the segmentation of cell images and liver CT images. The jump join in UNet directly combines shallow features with deep features easily creates a semantic gap. UNet + + mirrors the dense connection concept of DenseNet. Dense convolution blocks are added in jump connection, and semantic gap is reduced. Brugger et al, proposes a UNet model with partially reversible sequences. The reversible sequence can realize the back propagation without storing the intermediate result of the previous propagation, thereby reducing the memory consumption in the training process. CE-Net replaces UNet's coding module with a pre-trained model, proposing a context coding network to capture more advanced information and preserve spatial information for 2D medical image segmentation. Mainly comprises three modules: the device comprises a feature coding module, a context extraction module and a feature decoding module. The method performs better than UNet on different 2D medical image segmentation tasks. 3D U2-Net changes the way convolution is done in UNet. The main contribution is to convert the standard convolution in the UNet structure into a domain adapter based on deep separable convolution, and the parameter quantity of the network is reduced. Oktay et al propose a new medical image Attention Gate (AG) model. The model automatically focuses on target structures of different shapes and sizes. The AG-trained model suppresses irrelevant areas in the input image through implicit learning, while highlighting salient features that are useful for a specific task. The AG can be easily integrated into standard convolutional neural network architectures, reducing the time for model prediction.
Most of the above models are for image segmentation in imaging, and few models are for segmentation of cell nuclei in TCT images. There are mainly the following problems: (1) the segmentation accuracy is low because the problems of overlapping nuclei and non-obvious boundaries are not considered. (2) Contextual information at the nuclear boundary is not taken into account, resulting in under-segmentation or over-segmentation. Therefore, a cell nucleus segmentation method based on attention learning is provided. The context coding layer comprises four groups of parallel hole convolutions, and the hole convolutions with different expansion rates can extract the characteristics of the receptive fields with different sizes. The fusion of the different receptive fields may relate to contextual information in the image. The method can effectively solve the problems of under-segmentation and over-segmentation caused by unclear boundaries. The attention learning mechanism comprises a scaling module, a coding and decoding module and attention loss. The scaling module can obtain more image detail information. The coding and decoding module learns the attention weight matrix and solves the problem that the traditional attention mechanism is limited by a specific task. Attention loss can define a remarkable area, and the edge of the cell nucleus is taken as an area needing learning in the text, so that the segmentation problem of the overlapped cell nucleus is solved.
Disclosure of Invention
The invention aims to solve the problem of nucleus segmentation in intelligent pathological diagnosis and provides a nucleus segmentation method based on an attention learning mechanism.
The above object of the invention is mainly achieved by the following technical solutions, and the model structure and the training method are shown in fig. 1:
s1, preparing a training data set comprises four steps:
s11, selecting an image: the nuclear images were obtained from 100 cervical smears and the exfoliated cells were collected from populations of different ages and different disease states, including all lesion levels for the TBS diagnostic criteria.
S12, marking cell nucleus: manually circling the outline of the cell nucleus, and storing the position information of the outline of the cell nucleus in each image into a labeling file.
S13 generates a labeled graph: firstly, creating a completely black single-channel image (the gray value is 0), then reading the position information in the annotation file, and setting the gray value of each cell nucleus area to be 255.
S14, cutting the picture: the cell nucleus image and the label map are cropped simultaneously, with a crop size of 515 x 512. Ensure that the nuclear image and the marker map are consistent.
S2, preparing an attention learning loss label chart, comprising the following steps:
s21, obtaining a loss signature by calculation or image processing operation: and respectively detecting edge images in the X direction and the Y direction of the marker image by using a sobel operator, and fusing the two images to obtain a loss marker image.
S22, loading to a memory after corresponding: after the cell nucleus images, the marker maps and the loss marker maps are in one-to-one correspondence, the whole sequence is disordered, and then the cell nucleus images, the marker maps and the loss marker maps are loaded into the memory in batches.
S3, the attention mechanism is widely applied in the semantic segmentation field, and can be used for emphasizing or selecting important information of a target processing object and suppressing some irrelevant detailed information. The attention weight is obtained through a series of calculations in a general attention mechanism. This attention mechanism can only accomplish a specific task, and the attention is not controllable. Therefore, the present invention provides an attention learning mechanism, which comprises a scaling module, a coding/decoding module and attention loss as shown in fig. 2. Firstly, the input feature map is enlarged to 2 times of the original image, and then a single-channel feature matrix is obtained by using the coding and decoding module. And (4) recovering the original resolution ratio of the feature matrix after the feature matrix is calculated by a Sigmod function, and obtaining the attention matrix. The output of the attention learning structure, o (x), is calculated as follows:
in the formula, f (x) represents an input feature map, and a (x) represents an attention weight matrix.
Note that the zooming module in the mechanics learning mechanism zooms in the edge of the overlapping cell nuclei, so that the network obtains more detailed features. The encoder then encodes the global features of the image and the decoder learns the attention weights. The attention loss makes the attention mechanism not limited by specific tasks, and the advantages are further highlighted.
S4, in the image segmentation, the feature extraction includes convolution and pooling operations. Pooling may increase the receptive field of the convolution kernel and improve computational efficiency. Image segmentation is a classification at the pixel level, so an upsampling operation is required to restore the resolution of the image. However, there is always some information loss in the process of down-sampling and up-sampling. The hole convolution solves this problem to some extent. Hole convolution is a process of adding some spaces or zeros between elements in a convolution kernel to enlarge the convolution kernel. The hole convolution has a relatively important parameter expansion Rate. As shown in fig. 3, when Rate is 1, the convolution kernel has no space between each element. The convolution kernels for different rates have different receptive fields. The convolution fusion of the holes with different expansion rates can extract the context information of the image. With this property and taking advantage of CE-Net we have designed a context coding structure. The convolution of the holes with different expansion rates are operated in series to form four parallel branches. The expansion rate of each branch is increased continuously, the receptive fields are respectively 3, 7, 9 and 19, and the features extracted from the branches are fused in an additive mode. Because the importance degree of the features extracted by different branches is different, an SE channel attention mechanism is added on the basis. The structure extracts context information with different resolutions, and avoids information loss.
And (3) building a network structure, wherein the network adopts a U-Net structure with an attention mechanism, as shown in figure 1. The encoder utilizes a pre-trained model of resnet34 in the ImageNet dataset and inserts a context coding layer before its four-layer coding structure. The decoder employs an upsampling and convolution operation. Each layer of the encoder is connected with an attention learning structure, and the output characteristic diagram is fused with the characteristic diagram with the same resolution of the up-sampling for convolution. The output of the network comprises the up-sampled output and the output r of the attention learning structure. The formula for r is as follows:
wherein C (x) represents 1 × 1 convolution, ups (x, i) represents i upsampling on x, Oi(x) Indicates the ith noteAnd (5) learning the output of the structure through intention mechanics.
S5, training the model and segmenting the cell nucleus. The model designed in S4 is first trained using the data set created in S1, and we use a fixed step method to attenuate the learning rate during training. The loss is then calculated, the formula is as follows:
L＝Lbce(r,g)+Lbce(c,m) (3)
wherein L represents the loss of the model, LbceRepresenting a binary cross entropy loss function, r is shown in equation 2, g is the loss marker in S2, c represents the output of the model, and m represents the marker in S2. After 200 training rounds, the loss is approximately stable, and the training is stopped to obtain parameters of the cell nucleus segmentation model. And finally, segmenting cell nuclei by using a model.
Effects of the invention
The invention provides a cervical cell nucleus segmentation method for attention learning. The existing segmentation method cannot be used for cervical cell nucleus segmentation, and the prior knowledge of the cell nucleus is not utilized, so that the cell nucleus segmentation effect is poor. The attention learning mechanism focuses on the edge position of the cell nucleus, and effectively segments the overlapped cell nucleus. The context coding layer further extracts the context characteristics of cell nucleuses, and the problems of under-segmentation and over-segmentation are avoided. We verified the advantages of our proposed model in our dataset experimentally as shown in figure 5 and table 1. But it is further verified that the present invention has the same advantages in other public data sets.
TABLE 1 test results of different models on the inventive data set
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a model structure and loss calculation method according to an embodiment of the present invention;
FIG. 2 is a diagram of an attention learning mechanism according to an embodiment of the present invention;
FIG. 3 is a convolution of holes of different expansion rates according to an embodiment of the present invention;
FIG. 4 is a multi-scale context coding layer according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating the effect of model segmentation according to an embodiment of the present invention.
Detailed description of the invention
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1:
the invention provides a cell nucleus segmentation method based on an attention learning mechanism, which comprises the following steps:
s1, preparing a training data set;
s2, preparing an attention mechanics learning loss label diagram, and loading data;
s3, building an attention mechanics learning structure;
s4, building a network structure;
s5, training the model and segmenting the cell nucleus.
The following examples illustrate the invention in detail:
the embodiment of the invention is specifically realized.
S1, preparing a training data set comprises four steps:
(1) selecting an image: the nuclear images were obtained from 100 cervical smears and the exfoliated cells were collected from populations of different ages and different disease states, including all lesion levels for the TBS diagnostic criteria.
(2) Marking cell nucleus: manually circling the outline of the cell nucleus, and storing the position information of the outline of the cell nucleus in each image into a labeling file.
(3) Generating a labeled graph: firstly, creating a completely black single-channel image (the gray value is 0), then reading the position information in the annotation file, and setting the gray value of each cell nucleus area to be 255.
(4) Cutting the picture: the cell nucleus image and the label map are cropped simultaneously, with a crop size of 515 x 512. Ensure that the nuclear image and the marker map are consistent.
S2, preparing an attention learning loss label chart, comprising the following steps: firstly, edge images in the X direction and the Y direction of the marker image are respectively detected by using a sobel operator, and a loss marker image is obtained after the two images are fused. And then, after the cell nucleus images, the marker maps and the loss marker maps are in one-to-one correspondence, the whole sequence is disordered, and then the cell nucleus images, the marker maps and the loss marker maps are loaded into the memory in batches.
S3, constructing an attention learning structure as shown in FIG. 2, wherein the attention learning structure comprises a scaling module, a coding and decoding module and attention loss. Firstly, the input feature map is enlarged to 2 times of the original image, and the result is input into the coding and decoding module to obtain a single-channel feature matrix. And performing Sigmod operation on the feature matrix, and recovering the original input resolution ratio of the obtained result to obtain an attention weight matrix. The output of the attention learning structure, o (x), is calculated as follows:
in the formula, f (x) represents an input feature map, and a (x) represents an attention weight matrix.
S4, building a network structure as shown in figure 1, wherein the network adopts a U-Net structure with an attention mechanism. The encoder utilizes the pre-trained model of resnet34 in the ImageNet dataset and inserts the context coding layer before its four-layer coding structure, as shown in fig. 4. The decoder employs an upsampling and convolution operation. Each layer of the encoder is connected with an attention learning structure, and the output characteristic diagram is fused with the characteristic diagram with the same resolution of the up-sampling for convolution. The output of the network comprises the up-sampled output and the output r of the attention learning structure. The formula for r is as follows:
wherein C (x) represents 1 × 1 convolution, ups (x, i) represents i upsampling on x, Oi(x) The output of the ith attention learning structure is shown. In the context coding layer, the hole convolutions with different sampling rates form four branches, the expansion rate of each branch is increased continuously, the receptive fields are respectively 3, 7, 9 and 19, and the features extracted by the branches are fused in an additive mode to form context coding. Then a channel attention mechanism is introduced to make the structure coding capacity stronger.
S5, training the model and segmenting the cell nucleus. The model designed in S4 is first trained using the data set created in S1, and we use a fixed step method to attenuate the learning rate during training. The loss is then calculated, the formula is as follows:
L＝Lbce(r,g)+Lbce(c,m) (3)
wherein L represents the loss of the model, LbceRepresenting a binary cross entropy loss function, r is shown in equation (2), g is the loss label in S2, c represents the output of the model, and m represents the label in S2.
Training and setting: the primary coding structure in the model is based on ResNet pre-trained on ImageNet. The source code implementation is based on a pytorre platform. The training and testing platform is an Ubuntu Server 18.04.2LTS system which is provided with 4 RTX 2080Ti graphics cards and double CPUs. The invention uses 3 display cards, batch is 4, learning rate is 0.00005, training times is 200. And stopping training to obtain parameters of the cell nucleus segmentation model. And finally, segmenting cell nuclei by using a model. The final implementation effect is shown in fig. 5, which shows that the overlapped cell nuclei are well segmented, and the cell nuclei with unclear boundaries do not have over-segmentation phenomenon.
The present invention is capable of other embodiments and its several details are capable of modifications in various obvious respects, all without departing from the spirit and scope of the present invention.
Claims (6)
1. A method for segmenting cell nucleus based on attention learning is characterized by comprising the following steps:
s1, preparing a training data set;
s2, preparing an attention mechanics learning loss label diagram, and loading data;
s3, building an attention mechanics learning structure;
s4, building a network structure;
s5, training the model and segmenting the cell nucleus.
2. The method as claimed in claim 1, wherein the step S1 of preparing the training data set comprises four steps:
(1) selecting an image: the cell image is from 100 cell smears, and the cast-off cells are collected from people of different ages and different disease conditions, wherein the cell image comprises all lesion levels of TBS diagnosis standard;
(2) marking cell nucleus: manually circling the outline of the cell nucleus, and storing the position information of the outline of the cell nucleus in each image into a labeling file;
(3) generating a labeled graph: firstly, creating a full-black single-channel image (the gray value is 0), then reading the position information in the labeling file, and setting the gray value of each cell nucleus area to be 255;
(4) cutting the picture: cropping the cell nucleus image and the label map at the same time, with a crop size of 515 x 512; ensure that the nuclear image and the marker map crop consistently.
3. The method as claimed in claim 1, wherein the step S2 of preparing the attention learning loss label map comprises the following steps: firstly, edge images in the X direction and the Y direction of a marker image are respectively detected by using an edge detection algorithm, and a loss marker image is obtained after two images are fused; and then, after the cell nucleus images, the marker maps and the loss marker maps are in one-to-one correspondence, the whole sequence is disordered, and then the cell nucleus images, the marker maps and the loss marker maps are loaded into the memory in batches.
4. The attention-learning-based nucleus segmentation method as claimed in claim 1, wherein the attention-learning structure is constructed in step S3, and comprises a scaling module, a coding/decoding module and attention loss; firstly, amplifying an input feature map to be 2 times of an original image, and inputting a result into an encoding and decoding module to obtain a single-channel feature matrix; then, carrying out Sigmod operation on the characteristic matrix, and recovering the original input resolution ratio of the obtained result to obtain an attention weight matrix; the output of the attention learning structure, o (x), is calculated as follows:
in the formula, f (x) represents an input feature map, and a (x) represents an attention weight matrix.
5. The attention-learning-based nucleus segmentation method as claimed in claim 1, wherein a network structure is built in step S4, and the network adopts a U-Net structure with an attention mechanism; the encoder is a pre-training model in the ImageNet data set, and a context coding layer is inserted in front of each layer of coding structure of the encoder; the decoder adopts the operations of upsampling and convolution; each layer of the encoder is connected with an attention learning structure, and the output characteristic diagram and the characteristic diagram with the same resolution of the up-sampling are fused together for convolution; the output of the network comprises the output of the up-sampling and the output r of the attention learning structure; the formula for r is as follows:
wherein C (x) represents 1 × 1 convolution, ups (x, i) represents i upsampling on x, Oi(x) Is shown asOutputting the i attention learning structures; in a context coding layer, hole convolutions with different sampling rates form four branches, the expansion rate of each branch is increased continuously, the receptive fields are respectively 3, 7, 9 and 19, and the features extracted by the branches are fused in an additive mode to form context codes; then a channel attention mechanism is introduced to make the structure coding capacity stronger.
6. The attention-learning-based nucleus segmentation method as claimed in claim 1, wherein the model is trained and the nucleus is segmented in step S5; firstly, training a model designed in S4 by using a data set manufactured in S1, and attenuating a learning rate by adopting a fixed step length method in the training; the loss is then calculated, the formula is as follows:
L＝Lbce(r,g)+Lbce(c,m) (3)
wherein L represents the loss of the model, LbceRepresenting a binary cross entropy loss function, r is shown as equation (2), g is a loss label graph in S2, c represents the output of the model, and m represents a label graph in S2; after 200 training rounds, the loss is approximately stable, and the training is stopped to obtain parameters of a cell nucleus segmentation model; and finally, segmenting cell nuclei by using a model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011272095.4A CN112446892A (en) | 2020-11-18 | 2020-11-18 | Cell nucleus segmentation method based on attention learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011272095.4A CN112446892A (en) | 2020-11-18 | 2020-11-18 | Cell nucleus segmentation method based on attention learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112446892A true CN112446892A (en) | 2021-03-05 |
Family
ID=74737315
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011272095.4A Pending CN112446892A (en) | 2020-11-18 | 2020-11-18 | Cell nucleus segmentation method based on attention learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112446892A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113393443A (en) * | 2021-06-17 | 2021-09-14 | 华南理工大学 | HE pathological image cell nucleus segmentation method and system |
| CN113409321A (en) * | 2021-06-09 | 2021-09-17 | 西安电子科技大学 | Cell nucleus image segmentation method based on pixel classification and distance regression |
| CN113658117A (en) * | 2021-08-02 | 2021-11-16 | 浙江大学 | Method for identifying and dividing aggregate boundaries in asphalt mixture based on deep learning |
| CN114022487A (en) * | 2021-11-10 | 2022-02-08 | 哈尔滨理工大学 | Cervical cell nucleus segmentation method and device, electronic equipment and storage medium |
| CN114170483A (en) * | 2022-02-11 | 2022-03-11 | 南京甄视智能科技有限公司 | Training and using method, device, medium and equipment of floater identification model |
| CN114821046A (en) * | 2022-03-28 | 2022-07-29 | 深思考人工智能科技(上海)有限公司 | Method and system for cell detection and cell nucleus segmentation based on cell image |
| CN114943723A (en) * | 2022-06-08 | 2022-08-26 | 北京大学口腔医学院 | Method for segmenting and counting irregular cells and related equipment |
| WO2023062764A1 (en) * | 2021-10-13 | 2023-04-20 | 国立大学法人東北大学 | Biological image processing program, biological image processing device, and biological image processing method |
| CN116152574A (en) * | 2023-04-17 | 2023-05-23 | 厦门大学 | Pathological image classification method based on multi-stage information extraction and memory |
-
2020
- 2020-11-18 CN CN202011272095.4A patent/CN112446892A/en active Pending
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113409321A (en) * | 2021-06-09 | 2021-09-17 | 西安电子科技大学 | Cell nucleus image segmentation method based on pixel classification and distance regression |
| CN113409321B (en) * | 2021-06-09 | 2023-10-27 | 西安电子科技大学 | Cell nucleus image segmentation method based on pixel classification and distance regression |
| CN113393443B (en) * | 2021-06-17 | 2023-02-14 | 华南理工大学 | HE pathological image cell nucleus segmentation method and system |
| CN113393443A (en) * | 2021-06-17 | 2021-09-14 | 华南理工大学 | HE pathological image cell nucleus segmentation method and system |
| CN113658117A (en) * | 2021-08-02 | 2021-11-16 | 浙江大学 | Method for identifying and dividing aggregate boundaries in asphalt mixture based on deep learning |
| CN113658117B (en) * | 2021-08-02 | 2023-09-15 | 安徽省交通控股集团有限公司 | Method for identifying and dividing aggregate boundary in asphalt mixture based on deep learning |
| WO2023062764A1 (en) * | 2021-10-13 | 2023-04-20 | 国立大学法人東北大学 | Biological image processing program, biological image processing device, and biological image processing method |
| CN114022487A (en) * | 2021-11-10 | 2022-02-08 | 哈尔滨理工大学 | Cervical cell nucleus segmentation method and device, electronic equipment and storage medium |
| CN114170483B (en) * | 2022-02-11 | 2022-05-20 | 南京甄视智能科技有限公司 | Training and using method, device, medium and equipment of floater identification model |
| CN114170483A (en) * | 2022-02-11 | 2022-03-11 | 南京甄视智能科技有限公司 | Training and using method, device, medium and equipment of floater identification model |
| CN114821046A (en) * | 2022-03-28 | 2022-07-29 | 深思考人工智能科技(上海)有限公司 | Method and system for cell detection and cell nucleus segmentation based on cell image |
| CN114943723A (en) * | 2022-06-08 | 2022-08-26 | 北京大学口腔医学院 | Method for segmenting and counting irregular cells and related equipment |
| CN114943723B (en) * | 2022-06-08 | 2024-05-28 | 北京大学口腔医学院 | Method for dividing and counting irregular cells and related equipment |
| CN116152574A (en) * | 2023-04-17 | 2023-05-23 | 厦门大学 | Pathological image classification method based on multi-stage information extraction and memory |
| CN116152574B (en) * | 2023-04-17 | 2023-06-30 | 厦门大学 | Pathological image classification method based on multi-stage information extraction and memory |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112446892A (en) | Cell nucleus segmentation method based on attention learning | |
| CN109886986B (en) | Dermatoscope image segmentation method based on multi-branch convolutional neural network | |
| CN111311592B (en) | Three-dimensional medical image automatic segmentation method based on deep learning | |
| CN109410219B (en) | Image segmentation method and device based on pyramid fusion learning and computer readable storage medium | |
| Mirikharaji et al. | Star shape prior in fully convolutional networks for skin lesion segmentation | |
| CN111798462B (en) | Automatic delineation method of nasopharyngeal carcinoma radiotherapy target area based on CT image | |
| CN111369565B (en) | Digital pathological image segmentation and classification method based on graph convolution network | |
| CN111563897B (en) | Breast nuclear magnetic image tumor segmentation method and device based on weak supervision learning | |
| Wan et al. | Robust nuclei segmentation in histopathology using ASPPU-Net and boundary refinement | |
| CN111145181B (en) | Skeleton CT image three-dimensional segmentation method based on multi-view separation convolutional neural network | |
| US11562491B2 (en) | Automatic pancreas CT segmentation method based on a saliency-aware densely connected dilated convolutional neural network | |
| CN111091573B (en) | CT image pulmonary vessel segmentation method and system based on deep learning | |
| CN112150428A (en) | Medical image segmentation method based on deep learning | |
| Liu et al. | Cx22: A new publicly available dataset for deep learning-based segmentation of cervical cytology images | |
| CN113344951A (en) | Liver segment segmentation method based on boundary perception and dual attention guidance | |
| CN114494296A (en) | Brain glioma segmentation method and system based on fusion of Unet and Transformer | |
| CN110689525A (en) | Method and device for recognizing lymph nodes based on neural network | |
| CN114170244A (en) | Brain glioma segmentation method based on cascade neural network structure | |
| CN110782427A (en) | Magnetic resonance brain tumor automatic segmentation method based on separable cavity convolution | |
| CN113065551A (en) | Method for performing image segmentation using a deep neural network model | |
| Shan et al. | SCA-Net: A spatial and channel attention network for medical image segmentation | |
| CN113421240A (en) | Mammary gland classification method and device based on ultrasonic automatic mammary gland full-volume imaging | |
| CN114399510A (en) | Skin lesion segmentation and classification method and system combining image and clinical metadata | |
| CN112488996A (en) | Inhomogeneous three-dimensional esophageal cancer energy spectrum CT (computed tomography) weak supervision automatic labeling method and system | |
| CN116883341A (en) | Liver tumor CT image automatic segmentation method based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |