CN112200801B - Automatic detection method for cell nucleus of digital pathological image - Google Patents

Automatic detection method for cell nucleus of digital pathological image Download PDF

Info

Publication number
CN112200801B
CN112200801B CN202011190195.2A CN202011190195A CN112200801B CN 112200801 B CN112200801 B CN 112200801B CN 202011190195 A CN202011190195 A CN 202011190195A CN 112200801 B CN112200801 B CN 112200801B
Authority
CN
China
Prior art keywords
image
cell nucleus
fragment
images
data set
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011190195.2A
Other languages
Chinese (zh)
Other versions
CN112200801A (en
Inventor
叶丰
步宏
付波
李艳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
West China Hospital of Sichuan University
Original Assignee
West China Hospital of Sichuan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by West China Hospital of Sichuan University filed Critical West China Hospital of Sichuan University
Priority to CN202011190195.2A priority Critical patent/CN112200801B/en
Publication of CN112200801A publication Critical patent/CN112200801A/en
Application granted granted Critical
Publication of CN112200801B publication Critical patent/CN112200801B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30024—Cell structures in vitro; Tissue sections in vitro

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Investigating Or Analysing Biological Materials (AREA)

Abstract

The invention discloses a method for automatically detecting cell nucleuses of a digital pathological image, which comprises the steps of acquiring fragment images of the digital pathological image and carrying out standardized processing on the fragment images; inputting the normalized fragment images into a trained cell nucleus detection model for detection to obtain a cell nucleus set SN1(ii) a Covering the cell nucleuses detected in the fragment images with background colors, and calculating the energy value of each fragment image by adopting an image gradient energy function; judging whether the energy value of each fragment image is larger than a preset threshold value, if so, entering the next step, and otherwise, discarding the corresponding fragment image; inputting all fragment images with energy values larger than a preset threshold value into a trained cell nucleus detection model for detection to obtain a cell nucleus set SN2(ii) a Pool cell nuclear pool SN1And cell nuclear assembly SN2Obtaining the final cell nucleus set SN。

Description

Automatic detection method for cell nucleus of digital pathological image
Technical Field
The invention relates to a technology for detecting cell nucleuses in images, in particular to a method for automatically detecting cell nucleuses of digital pathological images.
Background
The occurrence and development of cancer are the result of the interaction between cancer cells and the tumor microenvironment, and the change of the types, the numbers or the forms of the cells in the tumor stroma has important medical guidance significance. For example, lymphocyte infiltrates in breast cancer generally have a better prognosis, while the presence of tumor-associated fibroblasts suggests a poor prognosis. In routine pathological work, changes in the cellular components and extracellular matrix in the tumor stroma are generally described qualitatively. Based on digital pathological image analysis, different components in the interstitium can be automatically segmented, and quantitative or qualitative research can be carried out.
On quantitative studies, using nuclear morphometric assays, researchers found that there was a significant difference in prognosis between low and high nuclear area patients. To extract a rich set of quantitative features in breast cancer epithelial cells and stroma (6642 features), researchers at Stanford university have developed the C-Path System (Computational Patholoist) for measuring standard morphological descriptors and higher-level context, relationships, and global image features that include image objects.
The cell nucleus detection of the pathology WSI image is the basis of the quantitative analysis of the whole image, and provides reliable support for the quantitative analysis and the biological index judgment required by various medical researches. Although many digital pathology-based methods of nuclear detection are proposed, they remain a very challenging task. The main reasons for this are:
(1) the difference between the WSI images is slight, cells are overlapped, and the color distribution is not uniform;
(2) the lack of large public, labeled data sets presents certain difficulties for algorithm research;
(3) different from other tumors, the size, density, shape and other individual differences of the mammary gland are large, so that the digital imaging is complex.
(4) The manual marking of cell nuclei is very laborious and difficult to form large marking databases.
The cell nucleus detection based on the traditional machine learning needs manual adjustment of a large number of parameters, the accuracy rate is difficult to improve, and the cell nucleus detection method is difficult to adapt to various complex devices and dyeing differences; models based on deep learning methods, in turn, often require a large number of labeled training samples, and careful adjustment of model parameters and training of neural network models, which can be time consuming.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides the automatic cell nucleus detection method for the digital pathological image with high detection precision.
In order to achieve the purpose of the invention, the invention adopts the technical scheme that:
a method for automatically detecting cell nucleuses of a digital pathological image is provided, which comprises the following steps:
s1, acquiring a fragment image of the digital pathological image, and carrying out standardization processing on the fragment image;
s2, inputting the normalized fragment images into a trained cell nucleus detection model for detection to obtain a cell nucleus set SN1；
S3, covering the cell nucleuses detected in the fragment images with background colors, and calculating the energy value of each fragment image by using an image gradient energy function;
s4, judging whether the energy value of each fragment image is larger than a preset threshold value, if so, entering a step S5, and otherwise, discarding the corresponding fragment image;
s5, inputting all fragment images with energy values larger than a preset threshold value into a trained cell nucleus detection model for detection to obtain a cell nucleus set SN2；
S6 merging cell nucleus set SN1And cell nuclear assembly SN2Obtaining the final cell nucleus set SN。
Further, the training method of the cell nucleus detection model comprises the following steps:
downloading MSCOCO data set D1Training a target detection algorithm by adopting an MSCOCO data set to obtain a target detection model M1；
Downloading 2018DSB data set D2And adopting 2018DSB data set to only reserve the target detection model M of the feature extraction layer for the model parameters1Training to obtain a target detection model M2；
Downloading open nuclear annotation data set D of digital pathology images3And for the data set D3Normalizing each fragment image, and then randomly reversing the pixel values of three channels in the normalized fragment images according to a preset proportion;
the data set obtained after the reverse color processing and a data set D are obtained3Target detection model M only retaining feature extraction layer for model parameters after combination3And training to obtain a cell nucleus detection model.
The invention has the beneficial effects that: when detecting cell nucleuses, the scheme performs detection on each fragment image twice, namely the detection is directly performed on the fragment image for the first time, and the detected cell nucleuses are set as background colors on RGB three channels of the original image for the second time; then re-performing the cell nucleus detection; finally, merging the two cell nucleus detections to obtain a final detection result; the condition of missing cell nucleus detection in the image can be greatly reduced by the detection mode.
In addition, when the cell nucleus detection model is trained, the open data set is fully utilized to provide a large amount of labeling information, and a large amount of labeling work can be saved through transfer learning; in addition, the influence of equipment, environment and other objective factors on pathological images can be reduced, and the precision and robustness of cell nucleus detection are greatly improved.
Drawings
Fig. 1 is a flow chart of a method for automatically detecting cell nuclei in a digital pathology image.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate the understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and it will be apparent to those skilled in the art that various changes may be made without departing from the spirit and scope of the invention as defined and defined by the appended claims, and all changes that can be made by the invention using the inventive concept are intended to be protected.
Referring to fig. 1, fig. 1 shows a flow chart of a method for automatically detecting a cell nucleus of a digital pathology image; as shown in fig. 1, the method S includes steps S1 to S6.
In step S1, a patch image of the digital pathology image is acquired and subjected to normalization processing; the method adopted by the standardization treatment is the Vahadane method.
In step S2, the normalized fragment image is input into the trained cell nucleus detection model for detection, and a cell nucleus set S is obtainedN1；
In one embodiment of the invention, the training method of the cell nucleus detection model comprises the following steps:
a1 downloading MSCOCO data set D1Training a target detection algorithm by adopting an MSCOCO data set to obtain a target detection model M1(ii) a The optimal target detection algorithm in the scheme is a Mask-RCNN algorithm.
A2 downloading 2018DSB data set D2And adopting 2018DSB data set to only reserve the target detection model M of the feature extraction layer for the model parameters1Training to obtain a target detection model M2；
A3, downloading the open nuclear labeling data set D of the digital pathology image3And to the data set D3Normalizing each fragment image, and then randomly reversing the pixel values of three channels in the normalized fragment images according to a preset proportion;
wherein, the reverse color processing is to adopt 255 to subtract the original pixel value to update the original pixel value of three channels.
A4, data set obtained by inverse color processing and data set D3Target detection model M only retaining feature extraction layer for model parameters after combination3And training to obtain a cell nucleus detection model.
By adopting the scheme, the cell nucleus detection model is obtained by training, a large amount of labeling information provided by the open data set is fully utilized, and a large amount of labeling work can be saved through transfer learning. After a small amount of digital pathological images are used for data set amplification, the target detection model is trained, and therefore the detection precision of cell nucleuses can be greatly improved.
In implementation, the method for training the optimized cell nucleus detection model further comprises the following steps:
and B1, acquiring a plurality of digital pathological images stored by the user, reading the ultra-high-definition image at the bottommost layer of the digital pathological images, and extracting the fragment images of the ultra-high-definition image in an overlapping or non-overlapping mode to form a fragment set.
When the method is implemented, Openslide pathological image reading software can be adopted to read the bottommost layer image; when extracting fragments, directly reading the fragments according to the position coordinates of image pixelsSlice image Ti(ii) a Can be read in an overlapping manner, then TiIs randomly read, is not overlapped, then TiRead by step n.
B2, calculating the energy value of each fragment image by adopting an image gradient energy function, and selecting the fragment images with the energy values larger than a set threshold value to form an image set ST1；
B3 image set ST1After that, the three-channel pixel values in the normalized fragment image are reversed according to a preset proportion to obtain a new data set ST2；
B4, image set ST1And a data set ST2And training the cell nucleus detection model with model parameters only reserved in the feature extraction layer after combination to obtain the final cell nucleus detection model.
When the scheme is used for model training, a small amount of marking data in practice is introduced, so that the detection accuracy can be greatly improved, the influence of equipment, environment and other objective factors on pathological images is reduced, and the accuracy and robustness of cell nucleus detection are greatly improved.
In step S3, covering the cell nuclei detected in the fragment image with a background color, where the background color is to modify all the pixel values corresponding to the positions of the cell nuclei detected in the fragment image to 255;
then, the energy value of each fragment image is calculated by adopting an image gradient energy function:
Figure BDA0002752559670000061
wherein, TiThe ith fragment image in the data set; g (T)i) Is TiThe energy value of (a); x is the pixel value in the x direction of the image; y is the image y-direction pixel value.
In step S4, determining whether the energy value of each fragment image is greater than a preset threshold, if so, entering step S5, otherwise, discarding the corresponding fragment image;
in step S5, allInputting the fragment image with the energy value larger than the preset threshold value into a trained cell nucleus detection model for detection to obtain a cell nucleus set SN2；
In step S6, the cell nucleus sets S are mergedN1And cell nuclear assembly SN2Obtaining the final cell nucleus set SN。
In conclusion, by adopting the scheme to detect the cell nucleus, the missing detection of the cell nucleus can be reduced, and meanwhile, the precision and the robustness of the cell nucleus detection can be greatly improved.

Claims (7)

1. The automatic detection method of the cell nucleus of the digital pathological image is characterized by comprising the following steps:
s1, acquiring a fragment image of the digital pathological image, and carrying out standardization processing on the fragment image;
s2, inputting the normalized fragment images into a trained cell nucleus detection model for detection to obtain a cell nucleus set SN1；
S3, covering the cell nucleuses detected in the fragment images with background colors, and calculating the energy value of each fragment image by using an image gradient energy function;
s4, judging whether the energy value of each fragment image is larger than a preset threshold value, if so, entering a step S5, and otherwise, discarding the corresponding fragment image;
s5, inputting all fragment images with energy values larger than a preset threshold value into a trained cell nucleus detection model for detection to obtain a cell nucleus set SN2；
S6 merging cell nucleus set SN1And cell nucleus set SN2Obtaining the final cell nucleus set SN。
2. The method for automatically detecting the cell nucleus of the digital pathological image according to claim 1, wherein the training method of the cell nucleus detection model comprises:
downloading MSCOCO data set D1Training a target detection algorithm by adopting an MSCOCO data set to obtain a target detection model M1；
Downloading 2018DSB data set D2And adopting 2018DSB data set to only reserve the target detection model M of the feature extraction layer for the model parameters1Training to obtain a target detection model M2；
Downloading open nuclear annotation data set D of digital pathology images3And to the data set D3Normalizing each fragment image, and then randomly reversing the pixel values of three channels in the normalized fragment images according to a preset proportion;
the data set obtained after the reverse color processing and the data set D are processed3Target detection model M only retaining feature extraction layer for model parameters after combination3And training to obtain a cell nucleus detection model.
3. The method for automatically detecting the cell nucleus of the digital pathological image according to claim 2, further comprising:
acquiring a plurality of digital pathological images stored by a user, reading the ultra-high-definition image at the bottommost layer of the digital pathological images, and extracting fragment images of the ultra-high-definition image in an overlapping or non-overlapping mode to form a fragment set;
calculating the energy value of each fragment image by adopting an image gradient energy function, and selecting the fragment images with the energy values larger than a set threshold value to form an image set ST1；
For image set ST1Normalizing each fragment image, and then randomly reversing the pixel values of three channels in the normalized fragment images according to a preset proportion to obtain a new data set ST2；
Set of images ST1And a data set ST2And training the cell nucleus detection model with model parameters only reserved in the feature extraction layer after combination to obtain the final cell nucleus detection model.
4. The method for automatically detecting the cell nucleus of the digital pathological image according to claim 3, wherein the energy function is calculated by the formula:
Figure FDA0002752559660000021
wherein, TiThe ith fragment image in the data set; g (T)i) Is TiThe energy value of (a); x is the pixel value in the x direction of the image; y is the image y-direction pixel value.
5. The method for automatically detecting the cell nucleus of the digital pathological image according to claim 2, wherein the normalization process is performed by a Vahadane method; the reverse color processing is to adopt 255 to reduce the original pixel value to update the original pixel value of three channels.
6. The method as claimed in claim 1, wherein the background color is overlaid to modify all pixel values corresponding to the detected cell nucleus positions in the fragment image to 255.
7. The method for automatically detecting the cell nucleus of a digital pathological image according to any one of claims 2-5, wherein the target detection algorithm is Mask-RCNN algorithm.
CN202011190195.2A 2020-10-30 2020-10-30 Automatic detection method for cell nucleus of digital pathological image Active CN112200801B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011190195.2A CN112200801B (en) 2020-10-30 2020-10-30 Automatic detection method for cell nucleus of digital pathological image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011190195.2A CN112200801B (en) 2020-10-30 2020-10-30 Automatic detection method for cell nucleus of digital pathological image

Publications (2)

Publication Number Publication Date
CN112200801A CN112200801A (en) 2021-01-08
CN112200801B true CN112200801B (en) 2022-06-17

Family

ID=74011149

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011190195.2A Active CN112200801B (en) 2020-10-30 2020-10-30 Automatic detection method for cell nucleus of digital pathological image

Country Status (1)

Country Link
CN (1) CN112200801B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008146278A (en) * 2006-12-08 2008-06-26 National Institute Of Advanced Industrial & Technology Cell outline extraction device, cell outline extraction method and program
CN107169977A (en) * 2017-04-24 2017-09-15 华南理工大学 Adaptive threshold color image edge detection method based on FPGA and Kirsch
CN108364288A (en) * 2018-03-01 2018-08-03 北京航空航天大学 Dividing method and device for breast cancer pathological image
CN110288582A (en) * 2019-06-26 2019-09-27 电子科技大学 A kind of accurate dividing method of nucleus
CN111275727A (en) * 2020-01-19 2020-06-12 北方工业大学 Cell segmentation method, cell segmentation apparatus, and computer-readable storage medium

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5354842B2 (en) * 2006-05-26 2013-11-27 キヤノン株式会社 Image processing method and image processing apparatus
US10360434B2 (en) * 2016-07-25 2019-07-23 Case Western Reserve University Multi-pass adaptive voting for nuclei detection in histopathological images

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008146278A (en) * 2006-12-08 2008-06-26 National Institute Of Advanced Industrial & Technology Cell outline extraction device, cell outline extraction method and program
CN107169977A (en) * 2017-04-24 2017-09-15 华南理工大学 Adaptive threshold color image edge detection method based on FPGA and Kirsch
CN108364288A (en) * 2018-03-01 2018-08-03 北京航空航天大学 Dividing method and device for breast cancer pathological image
CN110288582A (en) * 2019-06-26 2019-09-27 电子科技大学 A kind of accurate dividing method of nucleus
CN111275727A (en) * 2020-01-19 2020-06-12 北方工业大学 Cell segmentation method, cell segmentation apparatus, and computer-readable storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于卷积神经网络的细胞核图像检测与分类研究;李想;《中国知网硕士学位论文全文数据库电子期刊》;20200615;正文第21-48页 *

Also Published As

Publication number Publication date
CN112200801A (en) 2021-01-08

Similar Documents

Publication Publication Date Title
US11681418B2 (en) Multi-sample whole slide image processing in digital pathology via multi-resolution registration and machine learning
US11205266B2 (en) Systems and methods for detection of structures and/or patterns in images
Kannan et al. Segmentation of glomeruli within trichrome images using deep learning
CN113723573B (en) Tumor tissue pathological classification system and method based on adaptive proportion learning
Chang et al. Nuclear segmentation in H&E sections via multi-reference graph cut (MRGC)
US20220180518A1 (en) Improved histopathology classification through machine self-learning of "tissue fingerprints"
US11972621B2 (en) Systems and methods to label structures of interest in tissue slide images
Xu et al. Using transfer learning on whole slide images to predict tumor mutational burden in bladder cancer patients
CN112308840A (en) Automatic segmentation method for oral cancer epithelial tissue region of pathological image
CN112990214A (en) Medical image feature recognition prediction model
CN109147932B (en) Cancer cell HER2 gene amplification analysis method and system
Thirumal et al. TITAN: An end‐to‐end data analysis environment for the Hyperion™ imaging system
CN112200801B (en) Automatic detection method for cell nucleus of digital pathological image
Ollion et al. Analysis of nuclear organization with TANGO, software for high-throughput quantitative analysis of 3D fluorescence microscopy images
Guo et al. Pathological Detection of Micro and Fuzzy Gastric Cancer Cells Based on Deep Learning.
Salvi et al. cyto‐Knet: An instance segmentation approach for multiple myeloma plasma cells using conditional kernels
Aubreville et al. Augmented mitotic cell count using field of interest proposal
CN109658382A (en) Tongue body localization method based on image clustering and Gray Projection
CN117496276B (en) Lung cancer cell morphology analysis and identification method and computer readable storage medium
Gupta et al. Microscopy cancer cell imaging in b-lineage acute lymphoblastic leukemia
CN114004851B (en) Cell image segmentation method and device and cell counting method
Aalam et al. Deep learning-based identification of esophageal cancer subtypes through analysis of high-resolution histopathology images
CN116500006A (en) Method and device for detecting probe signal for fluorescence in-situ hybridization
CN117809853A (en) Construction method of hepatocellular carcinoma pathological recognition model and electronic equipment
Rittscher et al. Mapping for tissue based cytometry

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant