CN114299044A - Method and device for interpreting lymphocytes - Google Patents

Method and device for interpreting lymphocytes Download PDF

Info

Publication number
CN114299044A
CN114299044A CN202111662822.2A CN202111662822A CN114299044A CN 114299044 A CN114299044 A CN 114299044A CN 202111662822 A CN202111662822 A CN 202111662822A CN 114299044 A CN114299044 A CN 114299044A
Authority
CN
China
Prior art keywords
cell
lymphocytes
segmentation
image
cell segmentation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111662822.2A
Other languages
Chinese (zh)
Inventor
易坚
杨质瑾
王佳茜
苏小凡
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Yutai Antigen Technology Co ltd
Original Assignee
Shenzhen Yutai Antigen Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Yutai Antigen Technology Co ltd filed Critical Shenzhen Yutai Antigen Technology Co ltd
Priority to CN202111662822.2A priority Critical patent/CN114299044A/en
Publication of CN114299044A publication Critical patent/CN114299044A/en
Pending legal-status Critical Current

Links

Images

Abstract

A method and a device for judging lymphocytes are provided, and the method for judging the lymphocytes comprises the following steps: a cell segmentation step, including cell segmentation of the pathological image; and a cell classification step, which comprises the step of identifying lymphocytes from the pathological image after cell segmentation. Cell segmentation and lymphocyte recognition are automatically completed through software, manual participation is not needed, and efficiency and accuracy of reading lymphocytes are effectively improved.

Description

Method and device for interpreting lymphocytes
Technical Field
The invention relates to the field of precise medical treatment, in particular to a method and a device for interpreting lymphocytes.
Background
1. Histopathology
Histopathology (histopathology) refers to the science of histologists examining tissue sections under a microscope and is considered the gold standard for clinical tumor diagnosis. Histopathological subjects are tissue sections, usually tissue samples collected during surgery or biopsy, which are prepared as thin slices of tissue of several microns thickness by the steps of fixing, embedding, sectioning and staining, which are attached to a slide. The pathologist observes these sections through a microscope to make a diagnosis. These diagnoses are highly dependent on the experience of the physician. Staining is an important step in preparing tissue sections, and different stains can reveal different cells and tissue structures. Hematoxylin and eosin (H & E) stains can stain with strong color contrast in the nucleus and cytoplasm, are the most widely used staining reagents, and can be applied to many different tissue types.
2.TILs
One of the essential roles of the immune system is to maintain tissue homeostasis by continuously initiating immune surveillance and inflammatory responses involving activation of innate and adaptive immune cells. Tumors alter the ordered structure of tissues and induce an immune response that can eliminate early stage tumors. However, partial incomplete elimination of tumor cells may escape immune control due to the immune editing mechanisms of the tumor cells.
With the development of clinical studies, morphological assessment of Tumor Infiltrating Lymphocytes (TILs) in breast cancer is of increasing interest. There is increasing evidence to assess the extent of lymphocyte infiltration in tumor tissues by hematoxylin and eosin (H & E) stained tumor sections and to demonstrate that TILs have prognostic and potentially predictive value, especially in triple negative and EGFR receptor 2 overexpressing breast cancers.
3. Digital pathological image analysis (DIA)
Pathological diagnosis is the final diagnosis of a disease, and is widely used in diagnosis and medication guidance of tumors. In practice, pathologists view pathological sections under a microscope, and identifying subtle lesions from complex tissue images is an essential step in diagnosis. The process is time-consuming and has strong subjectivity. Digital pathology based on the whole-slide digital scanning technique (WSI) enables us to obtain pathology images with histological features, mostly captured by high resolution. Histopathological image analysis algorithms are developed to efficiently capture useful information in images. Since the analysis of the algorithm is objective and stable, the result can be verified repeatedly. Therefore, classical cell segmentation algorithms and cell classification algorithms have been widely applied to digital pathology analysis platforms. The digital pathological technology is used for assisting a pathologist to make pathological diagnosis, so that the accuracy and efficiency of pathological physician diagnosis are obviously improved.
In the prior art, the problems include: 1) the efficiency of manual TILs interpretation of HE pathological images is low; 2) human subjective deviation, visual deviation and other reasons can cause poor repeatability between manual interpretation and even interpretation errors.
Disclosure of Invention
According to a first aspect, in an embodiment, there is provided a method of interpreting lymphocytes, comprising:
a cell segmentation step, including cell segmentation of the pathological image;
and a cell classification step, which comprises the step of identifying lymphocytes from the pathological image after cell segmentation.
According to a second aspect, in an embodiment, there is provided an apparatus for interpreting lymphocytes, comprising:
the cell segmentation module is used for carrying out cell segmentation on the pathological image;
and the cell classification module is used for identifying the lymphocytes from the pathological image after cell segmentation.
According to a third aspect, in an embodiment, there is provided an apparatus comprising:
a memory for storing a program;
a processor for implementing the method as described in the first aspect by executing the program stored by the memory.
According to a fourth aspect, in an embodiment, there is provided a computer readable storage medium having a program stored thereon, the program being executable by a processor to implement the method according to the first aspect.
According to the method and the device for judging the lymphocytes, the cell segmentation and the lymphocyte recognition are automatically completed through software without manual participation, and the efficiency and the accuracy of judging the lymphocytes are effectively improved.
Drawings
Fig. 1 is a flowchart of training the lymphocyte recognition algorithm and TILs calculation in example 1.
FIG. 2 is an exemplary illustration of the labeling of lymphocytes manually performed by the pathologist in example 1; (a) HE original pathology image, (b) blue dots are artificially labeled as lymphocytes.
Fig. 3 is a diagram of an example of detecting cells in an HE image by automatic segmentation using the watershed algorithm in example 1. (a) An original pathological image; (b) after cell detection is finished, cell detection result mask images are obtained; (c) area within the detection range, cell number, etc.
FIG. 4 is a graph showing the results of counting the attribute values of each of the detected cells in example 1, in which information on the partial attribute values of 4 exemplary cells is shown.
FIG. 5 is a diagram of an exemplary cell segmentation, lymphocyte detection and TILs calculation in one area of example 1. (a) Original pathological images of the selected area by a pathologist; (b) the algorithm performs the result image of cell segmentation and classification, each ellipse represents a cell, and the dark part represents the cell nucleus. Blue represents lymphocytes; (c) the number of lymphocytes detected in the circled Area (Numimune cells) was calculated from TILs (ImmuneCelarearea%).
FIG. 6 is a diagram showing an example of cell segmentation, lymphocyte detection and TILs calculation in another region of example 1. (a) Original pathological images of the selected area by a pathologist; (b) the algorithm carries out cell segmentation and classification to obtain a result image, each ellipse represents a cell, and the dark part represents a cell nucleus; blue represents lymphocytes. (c) The number of lymphocytes detected in the circled Area (Num immune cells) was calculated from TILs (imminecell Area%).
Fig. 7 is a graph of the correlation performance verification of the software analysis result and the manual interpretation result in example 1.
Detailed Description
The present invention will be described in further detail with reference to the following detailed description and accompanying drawings. Wherein like elements in different embodiments are numbered with like associated elements. In the following description, numerous details are set forth in order to provide a better understanding of the present application. However, those skilled in the art will readily recognize that some of the features may be omitted or replaced with other elements, materials, methods in different instances. In some instances, certain operations related to the present application have not been shown or described in detail in order to avoid obscuring the core of the present application from excessive description, and it is not necessary for those skilled in the art to describe these operations in detail, so that they may be fully understood from the description in the specification and the general knowledge in the art.
Furthermore, the features, operations, or characteristics described in the specification may be combined in any suitable manner to form various embodiments. Also, the various steps or actions in the method descriptions may be transposed or transposed in order, as will be apparent to one of ordinary skill in the art. Thus, the various sequences in the specification and drawings are for the purpose of describing certain embodiments only and are not intended to imply a required sequence unless otherwise indicated where such sequence must be followed.
The numbering of the components as such, e.g., "first", "second", etc., is used herein only to distinguish the objects as described, and does not have any sequential or technical meaning. The term "connected" and "coupled" when used in this application, unless otherwise indicated, includes both direct and indirect connections (couplings).
As used herein, "tumor-infiltrating lymphocytes," also known as tumor-infiltrating lymphocytes (TILs ), are lymphocytes that migrate from the blood to the tumor tissue. In particular to lymphocytes which can migrate into the interior of a tumor or at the tumor margin, and the infiltrated lymphocytes are relevant to the prognosis of the tumor and the treatment effect.
According to a first aspect, in an embodiment, there is provided a method of interpreting lymphocytes, comprising:
a cell segmentation step, including cell segmentation of the pathological image;
and a cell classification step, which comprises the step of identifying lymphocytes from the pathological image after cell segmentation.
In one embodiment, in the cell classifying step, the attribute value of each cell is counted, and the lymphocyte is identified according to the attribute value.
In an embodiment, the property value comprises at least one of area, circumference, roundness, eccentricity, length, solidity, maximum diameter, minimum diameter, hematoxylin optical density value.
In an embodiment, the property values include all of area, perimeter, roundness, eccentricity, length, solidity, maximum diameter, minimum diameter, hematoxylin optical density values.
In one embodiment, the optimal parameter combinations for lymphocyte recognition are as follows: maximum tree depth: minimum number of supported samples per tree (artificially labeled cells): 20, number of features used per tree: 5, maximum tree number: 50.
in one embodiment, the pathology image comprises a region original pathology image circled by a pathologist.
In an embodiment, the pathology image includes a tissue slice treated by stain staining.
In one embodiment, the stain includes hematoxylin and eosin (H & E).
In one embodiment, the method further comprises a step of calculating the lymphocyte proportion according to the lymphocyte proportion [ (the area occupied by the lymphocytes in the identified pathological image)/(the total area of the case image or the total area of the circled areas in the case image) ]. the lymphocyte proportion is 100%.
In one embodiment, the lymphocytes comprise tumor infiltrating lymphocytes.
In one embodiment, in the cell segmentation step, after the pathological image is subjected to cell segmentation, the position information of the cell nucleus and the position information of the cell membrane of each cell are recorded.
In one embodiment, in the cell segmentation step, the cell nucleus and cell membrane contours of each cell are delineated by two contours, wherein the inner contour represents the cell nucleus contour and the outer contour represents the cell membrane contour.
In one embodiment, the algorithm for identifying lymphocytes in the cell classification step includes, but is not limited to, a random forest algorithm.
In one embodiment, the cell segmentation step includes, but is not limited to, a watershed algorithm.
According to a second aspect, in an embodiment, there is provided an apparatus for interpreting lymphocytes, comprising:
the cell segmentation module is used for carrying out cell segmentation on the pathological image;
and the cell classification module is used for identifying the lymphocytes from the pathological image after cell segmentation.
According to a third aspect, in an embodiment, there is provided an apparatus comprising:
a memory for storing a program;
a processor for implementing the method as described in the first aspect by executing the program stored by the memory.
According to a fourth aspect, in an embodiment, there is provided a computer readable storage medium having a program stored thereon, the program being executable by a processor to implement the method according to the first aspect.
In one embodiment, the present invention identifies lymphocytes as follows:
and a cell segmentation step, namely automatically completing the segmentation of the cells by using software based on a watershed algorithm.
And (4) a cell classification step, namely automatically identifying tumor infiltrating lymphocytes by using software and highlighting.
In one embodiment, the present invention develops an analysis process that allows software to automate cell segmentation, lymphocyte recognition and TILs. By providing automatic auxiliary counting, the efficiency and the accuracy of manual interpretation of a pathologist are improved.
In one embodiment, TILs refer to the area fraction of lymphocytes in the circled area.
In one embodiment, the lymphocytes comprise tumor infiltrating lymphocytes.
In one embodiment, the present invention uses Qupath as a digital pathology platform for development of TILs assisted interpretation algorithms.
In one embodiment, the present invention can greatly improve the efficiency and accuracy of TILs interpretation by automatically detecting lymphocytes and calculating the area ratio of the lymphocytes.
Example 1
The lymphocytes in this embodiment are specifically tumor infiltrating lymphocytes.
The embodiment comprises the following steps:
1. lymphocyte labeling and recognition algorithm training
In this embodiment, 7 HE-stained full-slide pathology images are randomly selected, and as shown in fig. 2, a professional pathologist manually marks lymphocytes in HE images by using a marking tool of Qupath, so as to mark more than 1000 lymphocytes.
As shown in fig. 3, the cells in the HE image are segmented using the watershed algorithm, and the position information of the cell nucleus and the cell membrane of each cell is recorded. The cell nucleus and membrane in each cell are outlined by two oval lines, where the inner oval is the nucleus and the outer oval is the membrane.
As shown in fig. 4, the attribute values such as diameter, roundness, area, color depth, etc. of each cell were counted. In order to enable lymphocyte detection to be more accurate and stable, the characteristics of lymphocytes and the HE staining principle are combined, and 9 cell attribute values are screened out, wherein the 9 cell attribute values are as follows: area (Area), circumference (Perimeter), roundness (circulation), Eccentricity (Eccentricity), Length (Length), Solidity (Solidity), maximum diameter (Max diameter), minimum diameter (Min diameter), Hematoxylin optical density value (Hematoxylin OD). Inputting the artificially marked lymphocytes and the screened attribute values into a random forest algorithm, training the random forest algorithm for recognizing the lymphocytes, and obtaining an optimal parameter combination for recognizing the lymphocytes through multi-test iterative optimization and adjustment, wherein the optimal parameter combination is as follows: maximum tree depth: minimum number of supported samples per tree (artificially labeled cells): 20, number of features used per tree: 5, maximum tree number: 50.
as shown in fig. 5 and 6, cell detection, cell attribute value statistics, and lymphocyte recognition are performed on the designated area of the HE pathology image by using a watershed algorithm and a trained random forest algorithm. Each ellipse represents a cell and the dark part represents the nucleus, where the blue label is a lymphocyte, and the TILs (ImmuneCell Area%) are calculated by combining the Area attribute value of each detected cell with the Area and cell number of the designated Area.
2. Lymphocyte detection and TILs (Linear arrays of cells) calculation software development
As shown in FIG. 1, the functions of cell detection, lymphocyte recognition and TILs ratio calculation are integrated into a software through Groovy language, and when a specified Region (ROI) is manually selected by using Qupath, the software can be used for automatically performing the lymphocyte detection and TILs ratio calculation of the ROI.
The TILs calculation formula is as follows:
TILs ═ Immune Cell Area/ROI Area. That is, the total area of the lymphocytes identified in the circled area is divided by the total area of the area, and the value is the TILs ratio.
3. Performance verification of software auto-computed TILs
To verify the performance of the software to calculate TILs, the pathologist randomly selected 32 Regions (ROIs) and manually interpreted the TILs for these regions. We detected TILs for these 32 ROIs using automated software developed earlier. As a result, as shown in FIG. 7, the correlation coefficient between the TILs automatically calculated by the software and the manually interpreted TILs reaches 0.82. We examined the interpretation region where the software interpretation and the manual interpretation have a deviation, and found that the interpretation deviation is mainly because the manual interpretation is affected by visual errors, and the area ratio of lymphocytes is estimated incorrectly. If the auxiliary interpretation software of the embodiment is used, the efficiency and the accuracy of TILs interpretation are improved.
Those skilled in the art will appreciate that all or part of the functions of the various methods in the above embodiments may be implemented by hardware, or may be implemented by computer programs. When all or part of the functions of the above embodiments are implemented by a computer program, the program may be stored in a computer-readable storage medium, and the storage medium may include: a read only memory, a random access memory, a magnetic disk, an optical disk, a hard disk, etc., and the program is executed by a computer to realize the above functions. For example, the program may be stored in a memory of the device, and when the program in the memory is executed by the processor, all or part of the functions described above may be implemented. In addition, when all or part of the functions in the above embodiments are implemented by a computer program, the program may be stored in a storage medium such as a server, another computer, a magnetic disk, an optical disk, a flash disk, or a removable hard disk, and may be downloaded or copied to a memory of a local device, or may be version-updated in a system of the local device, and when the program in the memory is executed by a processor, all or part of the functions in the above embodiments may be implemented.
The present invention has been described in terms of specific examples, which are provided to aid understanding of the invention and are not intended to be limiting. For a person skilled in the art to which the invention pertains, several simple deductions, modifications or substitutions may be made according to the idea of the invention.

Claims (10)

1. A method of interpreting lymphocytes, comprising:
a cell segmentation step, including cell segmentation of the pathological image;
and a cell classification step, which comprises the step of identifying lymphocytes from the pathological image after cell segmentation.
2. The method of claim 1, wherein in the cell classifying step, the attribute value of each cell is counted, and the lymphocytes are identified based on the attribute values.
3. The method of claim 2, wherein the property values include at least one of area, perimeter, roundness, eccentricity, length, solidity, maximum diameter, minimum diameter, hematoxylin optical density values;
preferably, the property values include all of area, circumference, roundness, eccentricity, length, solidity, maximum diameter, minimum diameter, hematoxylin optical density values.
4. The method of claim 1, wherein the optimal parameters for lymphocyte recognition are combined as follows: maximum tree depth: 5, minimum number of supported samples per tree: 20, number of features used per tree: 5, maximum tree number: 50.
5. the method of claim 1, wherein the pathology image comprises a region raw pathology image circled by a pathologist;
and/or, the pathological image comprises a tissue section processed by staining by a staining agent;
and/or, the staining agent comprises hematoxylin and eosin.
6. The method of claim 1, further comprising a step of calculating a lymphocyte proportion, which comprises calculating the lymphocyte proportion according to [ (area occupied by lymphocytes in the identified pathology image)/(total area of case image or total area of circled regions in case image) ] 100%.
7. The method of claim 1, wherein said lymphocytes comprise tumor infiltrating lymphocytes;
and/or in the cell segmentation step, after cell segmentation is carried out on the pathological image, the cell nucleus position information and the cell membrane position information of each cell are recorded;
and/or in the cell segmentation step, the contour of the cell nucleus and the cell membrane in each cell is described through two contour lines, wherein the inner contour line represents the contour of the cell nucleus, and the outer contour line represents the contour of the cell membrane;
and/or in the cell classification step, the algorithm for identifying the lymphocytes comprises a random forest algorithm;
and/or in the cell segmentation step, the algorithm for carrying out cell segmentation on the pathological image comprises a watershed algorithm.
8. An apparatus for interpreting lymphocytes, comprising:
the cell segmentation module is used for carrying out cell segmentation on the pathological image;
and the cell classification module is used for identifying the lymphocytes from the pathological image after cell segmentation.
9. An apparatus, comprising:
a memory for storing a program;
a processor for implementing the method of any one of claims 1 to 7 by executing the program stored in the memory.
10. A computer-readable storage medium, characterized in that the medium has stored thereon a program which is executable by a processor to implement the method according to any one of claims 1 to 7.
CN202111662822.2A 2021-12-31 2021-12-31 Method and device for interpreting lymphocytes Pending CN114299044A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111662822.2A CN114299044A (en) 2021-12-31 2021-12-31 Method and device for interpreting lymphocytes

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111662822.2A CN114299044A (en) 2021-12-31 2021-12-31 Method and device for interpreting lymphocytes

Publications (1)

Publication Number Publication Date
CN114299044A true CN114299044A (en) 2022-04-08

Family

ID=80974217

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111662822.2A Pending CN114299044A (en) 2021-12-31 2021-12-31 Method and device for interpreting lymphocytes

Country Status (1)

Country Link
CN (1) CN114299044A (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106570505A (en) * 2016-11-01 2017-04-19 北京昆仑医云科技有限公司 Method for analyzing histopathologic image and system thereof
CN108288265A (en) * 2018-01-09 2018-07-17 东北大学 A kind of segmentation and sorting technique towards HCC pathological image nucleus
CN109034208A (en) * 2018-07-03 2018-12-18 怀光智能科技(武汉)有限公司 A kind of cervical cell pathological section classification method of high-low resolution combination
CN110632069A (en) * 2019-08-20 2019-12-31 西人马帝言(北京)科技有限公司 Circulating tumor cell detection method, device, equipment and medium
CN113470041A (en) * 2021-05-26 2021-10-01 透彻影像(北京)科技有限公司 Immunohistochemical cell image cell nucleus segmentation and counting method and system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106570505A (en) * 2016-11-01 2017-04-19 北京昆仑医云科技有限公司 Method for analyzing histopathologic image and system thereof
CN108288265A (en) * 2018-01-09 2018-07-17 东北大学 A kind of segmentation and sorting technique towards HCC pathological image nucleus
CN109034208A (en) * 2018-07-03 2018-12-18 怀光智能科技(武汉)有限公司 A kind of cervical cell pathological section classification method of high-low resolution combination
CN110632069A (en) * 2019-08-20 2019-12-31 西人马帝言(北京)科技有限公司 Circulating tumor cell detection method, device, equipment and medium
CN113470041A (en) * 2021-05-26 2021-10-01 透彻影像(北京)科技有限公司 Immunohistochemical cell image cell nucleus segmentation and counting method and system

Similar Documents

Publication Publication Date Title
JP7273215B2 (en) Automated assay evaluation and normalization for image processing
WO2019110583A1 (en) Deep-learning systems and methods for joint cell and region classification in biological images
US20110286654A1 (en) Segmentation of Biological Image Data
US20220351860A1 (en) Federated learning system for training machine learning algorithms and maintaining patient privacy
JP7422235B2 (en) Non-tumor segmentation to aid tumor detection and analysis
WO2014114612A1 (en) Method and systems for cell-level fish dot counting
GB2423150A (en) Distinguishing layers of epithelial tissue
CN107567631B (en) Tissue sample analysis techniques
Kanwal et al. Quantifying the effect of color processing on blood and damaged tissue detection in whole slide images
US20240079116A1 (en) Automated segmentation of artifacts in histopathology images
Scudamore Practical approaches to reviewing and recording pathology data
US9785848B2 (en) Automated staining and segmentation quality control
US10943350B2 (en) Automated segmentation of histological sections for vasculature quantification
Palokangas et al. Segmentation of folds in tissue section images
CN114299044A (en) Method and device for interpreting lymphocytes
Silva et al. Development of a quantitative semi-automated system for intestinal morphology assessment in Atlantic salmon, using image analysis
EP3757872A1 (en) Scanning/pre-scanning quality control of slides
JPH0694706A (en) Pathology image inspection support device
US11263781B2 (en) Process for determining the infiltration of biological cells in a biological object of interest
Satou et al. Automatic analysis for neuron by confocal laser scanning microscope

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20220408

RJ01 Rejection of invention patent application after publication