CN108319815B - Method and system for virtually staining cells - Google Patents
Method and system for virtually staining cells Download PDFInfo
- Publication number
- CN108319815B CN108319815B CN201810109960.XA CN201810109960A CN108319815B CN 108319815 B CN108319815 B CN 108319815B CN 201810109960 A CN201810109960 A CN 201810109960A CN 108319815 B CN108319815 B CN 108319815B
- Authority
- CN
- China
- Prior art keywords
- dyeing
- information
- staining
- image
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16B—BIOINFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR GENETIC OR PROTEIN-RELATED DATA PROCESSING IN COMPUTATIONAL MOLECULAR BIOLOGY
- G16B40/00—ICT specially adapted for biostatistics; ICT specially adapted for bioinformatics-related machine learning or data mining, e.g. knowledge discovery or pattern finding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16B—BIOINFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR GENETIC OR PROTEIN-RELATED DATA PROCESSING IN COMPUTATIONAL MOLECULAR BIOLOGY
- G16B20/00—ICT specially adapted for functional genomics or proteomics, e.g. genotype-phenotype associations
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Medical Informatics (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Molecular Biology (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Biotechnology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Databases & Information Systems (AREA)
- Epidemiology (AREA)
- Bioethics (AREA)
- Public Health (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Mathematical Physics (AREA)
- Biomedical Technology (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Genetics & Genomics (AREA)
- Proteomics, Peptides & Aminoacids (AREA)
- Investigating Or Analysing Biological Materials (AREA)
Abstract
The invention relates to a method and a system for cell virtual staining. The method of the invention establishes a prediction model for predicting second staining information based on first staining information in a machine learning mode, virtually stains a cell staining image according to the prediction result of the prediction model, and superposes the cell staining image on the cell staining image. The system of the invention is a system corresponding to the following method, and can accurately execute the virtual dyeing method. Compared with the conventional dyeing method, the method and the system of the invention obtain the second dyeing information through virtual dyeing, do not need to carry out corresponding actual dyeing on the sample corresponding to the second cell dyeing image, save time and labor, can easily realize multiple virtual dyeing on the same cell dyeing image and the same cell of the same cell dyeing image, better analyze the characteristics of cell types, quantity, distribution, position relation and the like, and are beneficial to clinical treatment and scientific research.
Description
Technical Field
The invention relates to the field of image processing, in particular to a method and a system for cell virtual staining.
Background
Immunohistochemistry (immunohistochemistry) is a technical means widely applied to clinical work and scientific research, and reflects the cell morphology of a sample to be detected and the expression condition of a molecular marker through dyeing, so that important decision information is provided for clinicians and scientific researchers.
With the development of scientific research, more marker species need to be researched in the same sample in order to obtain more useful information to facilitate comprehensive decision. For example, analysis of the tumor microenvironment in which tumor infiltrating lymphocytes reside requires the study of more than 20 molecular markers. However, the existing immunohistochemical staining method has many defects, and is difficult to meet the requirement of increasing the staining index of a marker.
Currently, conventional immunohistochemical staining methods include double immunohistochemical staining, serial section staining, multicolor fluorescence immunohistochemical staining, multiple staining of the same tissue, and RNA-based section staining methods.
The double immunohistochemical staining is to perform twice staining on the same slice, the staining information which can be obtained is limited, the situation is limited today that research indexes are diversified, and for a common staining area, a later staining marker covers the former marker, so that the staining information is difficult to distinguish.
The continuous section staining is to continuously slice the pathological tissue and directly superpose the staining information of the subsequent images on the first image. However, the staining of different cells is actually not accurate in the study of sporadic tumor infiltrating lymphocytes and tumors with high heterogeneity.
The multi-color fluorescence immunohistochemical staining can support 4-7 kinds of staining, but expensive instruments are needed, the staining process is complex, the time consumption is long, and the staining quantity is limited.
The multiple staining of the same tissue is to use color developing agent easy to elute, repeat the staining-eluting process on the same tissue section, and save the result of each staining. The method has the defects of incomplete washing of the coloring agent, pictures, tissue deformation and the like.
RNA-based staining of sections reflects protein expression levels laterally by detecting RNA expression. The method requires a fresh sample, and the expression of RNA and protein is asynchronous, so that the method has high requirements on the sample and the detection result is possibly inaccurate.
As described above, the conventional immunohistochemical staining method is not suitable for performing multiple staining on the same tissue section, or the cost of multiple staining is too high and the number of types of staining indexes is limited. Therefore, there is a need in the art to provide a clinically acceptable and cost-effective method capable of achieving multiple staining on the same tissue section, especially a staining method capable of simultaneously calibrating more than 7 markers.
In view of the above, the present invention is particularly proposed.
Disclosure of Invention
The first purpose of the present invention is to provide a method for virtually staining cells, which can evaluate the degree of correlation between first staining information and second staining information, establish a prediction model, and predict and obtain the second staining information directly according to the first staining information, thereby saving time and labor.
Furthermore, the method can predict and overlay a plurality of staining information on the same cell staining image without repeated elution and staining, expensive detection instruments and complicated staining procedures, and the overlaid staining information can be used for researching complicated co-expression, proximity relation and the like.
Further, the first staining information of the method of the present invention is preferably nuclear staining information, cytoplasmic staining information or cytoskeletal staining information, and the machine learning mode is a full convolution neural network learning mode. The method integrates different staining information into the same cell staining image through a morphology and deep learning method, can better classify the cells and extract cell characteristics, and is beneficial to subsequent research.
Further, the second staining information of the present invention is preferably staining information of a specific marker expressed by a non-atypical cell, which enables cross-patient prediction.
Furthermore, the first staining information is preferably nuclear staining information, and the second staining information is preferably CD3 staining information and/or CD4 staining information, wherein the nuclear staining information has a high degree of association with CD3 staining information and CD4 staining information, and the established model can accurately predict CD3 and CD4 staining information through the nuclear staining information.
The second purpose of the present invention is to provide a system for the foregoing method, which can accurately execute the corresponding operation steps of the foregoing method, and implement prediction and labeling of nuclear staining information.
In order to achieve the above purpose of the present invention, the following technical solutions are adopted:
a method for virtual staining of cells, the method comprising the steps of:
(1) acquiring a first staining image, wherein the first staining image is a cell staining image and comprises first staining information and second staining information;
(2) establishing a prediction model for predicting second dyeing information according to the first dyeing information by taking the first dyeing image as a learning object in a machine learning manner, and verifying whether the prediction model is qualified or not, wherein the qualified prediction model is used for carrying out cell virtual dyeing on the second dyeing image;
(3) obtaining a second stain image, wherein the second stain image is a cell stain image and comprises the first stain information but does not comprise the second stain information;
(4) and inputting the first dyeing information of the second dyeing image into the qualified prediction model, predicting to obtain the second dyeing information of the second dyeing image and superposing the second dyeing information on the second dyeing image, namely obtaining the second virtual dyeing result on the second dyeing image.
In some specific embodiments, the second stain image further comprises additional stain information thereon;
preferably, the other dyeing information is obtained by means of direct dyeing and/or virtual dyeing;
more preferably, one or two of the other staining information is obtained by means of direct staining, and the rest of the other staining information is obtained by means of virtual staining;
most preferably, the virtual staining mode of the other staining information is as follows: and (3) obtaining and performing virtual dyeing by the modes of the step (1) to the step (4), wherein a first dyeing image used by other dyeing information in the virtual dyeing process is the same as or different from a first dyeing image used by second dyeing information in the virtual dyeing process, and the first dyeing information used by the other dyeing information in the virtual dyeing process is the same as or different from the first dyeing information used by the second dyeing information in the virtual dyeing process.
In some specific embodiments, the step (2) specifically comprises the following steps:
a. separating first dyeing information and second dyeing information of the first dyeing image to obtain a first dyeing channel and a second dyeing channel;
b. segmenting the first staining image into a plurality of small images, and acquiring a data set, wherein the data set comprises a first staining channel and a second staining channel of each small image;
c. dividing the data set into a training set and a check set;
d. establishing a machine learning task in the training set by taking the first dyeing channel of each small graph as an input value and the corresponding second dyeing channel as an output value to obtain a model for predicting second dyeing information through the first dyeing information;
e. the specificity and sensitivity of the model are verified using the calibration set, and the qualified specificity and sensitivity model is used to predict second staining information not included in the second staining image.
In some specific embodiments, the sensitivity and specificity of the qualified model is above 85%; preferably, the sensitivity and specificity of the qualified model is above 90%; more preferably, the sensitivity and specificity of the qualifying model is above 95%.
In some specific embodiments, the machine learning is neural network learning or probabilistic graphical model learning; preferably, the machine learning is neural network learning; more preferably, the neural network is learned as a multi-layer full convolution neural network model.
In some specific embodiments, the step (4) specifically comprises the following steps:
f. separating first dyeing information of the second dyeing image to obtain a first dyeing channel;
g. inputting the first dyeing channel into the model, and predicting a second dyeing channel for obtaining the second dyeing image;
i. superimposing the second dye channel in the second dye image.
In some specific embodiments, the first staining information is selected from one or more of nuclear staining information, cytoplasmic staining information, cytoskeletal staining information, or molecular marker staining information;
preferably, the first staining information is nuclear staining information;
more preferably, the nuclear staining information is hematoxylin staining information or 4',6-diamidino-2-phenylindole (4',6-diamidino-2-phenylindole, DAPI) staining information.
In some specific embodiments, the second staining information is selected from nuclear staining information, cytoplasmic staining information, cytoskeletal staining information, or molecular marker staining information;
preferably, the second staining information is molecular marker staining information;
more preferably, the molecular marker is a specific marker expressed by a non-allogeneic cell or a specific marker expressed by a xenogeneic cell;
most preferably, the non-allogeneic cell expresses a specific marker selected from CD3, CD4 or CD8 and the allogeneic cell expresses a specific marker selected from CD34, MUC1 or P53.
In some specific embodiments, the first stained image is obtained from a tissue section, a cell smear, or a cell slide; the second stained image is obtained from a tissue section, a cell smear, or a cell slide.
In some specific embodiments, the sample source of the first stain image and the second stain image is a homogeneous sample or a non-homogeneous sample comprising homogeneous cells;
in some specific embodiments, the sample sources of the first stain image and the second stain image are homogeneous samples, preferably, homogeneous samples of the same subject; more preferably, the homogeneous sample of the same subject is adjacent tissue of the same subject.
In some specific embodiments, the additional staining information is selected from nuclear staining information, cytoplasmic staining information, cytoskeletal staining information, or molecular marker staining information;
preferably, the other staining information is molecular marker staining information;
more preferably, the molecular marker is a specific marker expressed by a non-allogeneic cell or a specific marker expressed by a xenogeneic cell;
most preferably, the non-allogeneic cell expresses a specific marker selected from CD3, CD4 or CD8 and the allogeneic cell expresses a specific marker selected from CD34, MUC1 or P53.
In some specific embodiments, the second staining information and the additional staining information are staining information of co-expressed or proximally expressed molecular markers.
In some specific embodiments, the association of the first staining information and the second staining information is known in advance or unknown.
In some specific embodiments, the method is for non-diagnostic purposes; preferably, the non-diagnostic purpose comprises obtaining an analysis of cell type, number, distribution or positional relationship.
The invention also relates to a system for the aforementioned method, comprising an image acquisition module, a model building module and a virtual staining module, wherein:
the image acquisition module is used for acquiring a first dyeing image and a second dyeing image;
the model establishing module is used for establishing a prediction model for predicting second dyeing information according to the first dyeing information in a machine learning mode and detecting whether the prediction model is qualified or not;
the virtual dyeing module is used for inputting the first dyeing information of the second dyeing image into the prediction model so as to predict and obtain the second dyeing information of the second dyeing image, and the second dyeing information is superposed on the second dyeing image.
In some specific embodiments, the model building module comprises a staining information separation module, an image segmentation module, a data processing module, a training module, and a verification module, wherein:
the dyeing information separation module is used for separating first dyeing information and second dyeing information of the first dyeing image to obtain a first dyeing channel and a second dyeing channel; preferably, the staining information separating module is further used to separate impurities, noise and abnormal tissues, such as defocus, stains and sample folding.
The image segmentation module is used for segmenting the first dyeing image into a plurality of small images;
the data processing module is used for acquiring a first dyeing channel and a second dyeing channel of each small picture to form a data set, and dividing the data set into a training set and a check set;
the training module takes a first dyeing channel of each small graph in the training set as an input value and corresponding second dyeing information as an output value, and obtains a prediction model for predicting the second dyeing information according to the first dyeing information in a machine learning mode;
the verification module is used for verifying the accuracy of the prediction module and using the model with qualified accuracy for predicting the second dyeing information which is not contained in the second dyeing image.
In some specific embodiments, the virtual staining module comprises a staining information separation module, a prediction module, and an overlay module, wherein:
the dyeing information separation module is used for separating the first dyeing marks of the second dyeing image so as to obtain a first dyeing channel;
the prediction module is used for inputting the first dyeing channel of the second dyeing image into the prediction model so as to obtain a second dyeing information prediction result of the second dyeing image;
the superposition module is used for superposing the second dyeing information prediction result in the second dyeing image.
In some specific embodiments, the training module obtains the prediction model by means of neural network learning, preferably, a multilayer full convolution neural network model.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a schematic flow chart of a virtual staining method according to embodiment 1
FIG. 2 is a diagram showing a structure of a full convolution network according to embodiment 1;
FIG. 3 is a schematic flow chart of the virtual staining method according to embodiment 2;
FIG. 4 is a schematic view of the system described in example 3;
FIG. 5 is a first staining pattern as described in Experimental example 1;
FIG. 6 is a schematic view of (a partial) nuclear staining channels isolated from the first staining pattern described in Experimental example 1;
FIG. 7 shows the CD3 staining channel (partial) isolated from the first staining pattern described in Experimental example 1;
FIG. 8 is a graph showing the prediction of the staining result of CD3 after the model was stabilized on the training set in Experimental example 1, which corresponds to FIGS. 4-5;
FIG. 9 is a second staining pattern of Experimental example 1, which includes information on nuclear and MUC1 staining and does not include information on CD3 and CD4 staining;
FIG. 10 shows the CD4 staining channel (partial) isolated from the first staining image described in example 1;
FIG. 11 is a graph showing the prediction of the staining result of CD4 after the model was stabilized on the training set in example 1, which corresponds to FIG. 8;
FIG. 12 is a graph of the result of virtual staining for CD3 in the second staining pattern described in Experimental example 1;
FIG. 13 is a graph showing the results of virtual staining of CD3 and CD4 on the first cell staining pattern described in Experimental example 1.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to examples, but it will be understood by those skilled in the art that the following examples are only illustrative of the present invention and should not be construed as limiting the scope of the present invention. The examples, in which specific conditions are not specified, were conducted under conventional conditions or conditions recommended by the manufacturer. The reagents or instruments used are not indicated by manufacturers, and are all conventional products available on the market.
Example 1
As shown in fig. 1, embodiment 1 of the present invention provides a method for virtual staining of cells, the method including the steps of:
(1) acquiring a first staining image, wherein the first staining image is a cell staining image and comprises first staining information and second staining information;
(2) establishing a prediction model for predicting second dyeing information according to the first dyeing information and verifying whether the prediction model is qualified or not in a machine learning mode by taking the first dyeing image as a learning object, wherein the qualified prediction model is used for virtually dyeing the second dyeing image and comprises the following steps:
a. separating first dyeing information and second dyeing information in the first dyeing image to obtain a first dyeing channel and a second dyeing channel;
b. segmenting the first staining image into a plurality of small images, and acquiring a data set, wherein the data set comprises a first staining channel and a second staining channel of each small image;
c. dividing the data set into a training set and a check set, wherein 60% of the training set and 40% of the check set are training sets and check sets;
d. establishing a machine learning task in the training set by taking the first dyeing channel of each small graph as an input value and the corresponding second dyeing channel as an output value to obtain a model for predicting second dyeing information through the first dyeing information, wherein the machine learning is multilayer full convolution neural network learning (the structure of the full convolution network is shown in fig. 2);
e. verifying an accuracy of the model using the check set, the model having an accuracy greater than 85% for predicting second staining information not included in a second staining image;
(3) obtaining a second stain image, wherein the second stain image is a cell stain image and comprises the first stain information but does not comprise the second stain information;
(4) inputting first dyeing information of the second dyeing image into the model, predicting and obtaining the second dyeing information of the second dyeing image and superimposing the second dyeing information on the second dyeing image, namely obtaining a virtual dyeing result of the second dyeing information on the second dyeing image, wherein the step (4) specifically comprises:
f. separating first dyeing information in the second dyeing image to obtain a first dyeing channel;
g. inputting the first staining channel into the model, and predicting to obtain the second staining channel;
i. superimposing the second dye channel in the second dye image.
Example 2
As shown in fig. 3, embodiment 2 of the present invention provides a method for virtually staining cells, in which the method described in embodiment 1 is performed to virtually stain the second staining information in the second image, and then the method described in embodiment 1 is referred to continue to perform virtual staining of other staining information.
Example 3
As shown in fig. 4, embodiment 3 of the present invention provides a system 100 for the method of embodiment 1 or 2, the system including an image acquisition module 110, a model building module 120, and a virtual staining module 130, wherein:
the image obtaining module 110 is configured to obtain a first dye image and a second dye image;
the model establishing module 120 is configured to establish a prediction model for predicting the second dyeing information according to the first dyeing information in a machine learning manner;
the virtual staining module 130 is configured to input first staining information of the second staining image to the prediction model so as to predict and obtain second staining information of the second staining image, and superimpose the second staining information on the second staining image;
the model building module 120 comprises a staining information separating module 121, an image segmentation module 122, a data processing module 123, a training module 124 and a verification module 125; wherein:
the dyeing information separation module 121 is configured to separate first dyeing information and second dyeing information of the first dyeing image to obtain a first dyeing channel and a second dyeing channel;
the image segmentation module 122 is configured to segment the first staining image into a plurality of small images;
the data processing module 123 is configured to obtain a first staining channel and a second staining channel of each thumbnail to form a data set, and divide the data set into a training set and a check set;
the training module 124 obtains a prediction model for predicting second dyeing information according to the first dyeing information in a machine learning manner by taking the first dyeing channel of each small graph in the training set as an input value and the corresponding second dyeing information as an output value;
the verifying module 125 is configured to verify the accuracy of the predicting module, and use a model with qualified accuracy for predicting the second dyeing information not included in the second dyeing image.
The virtual staining module 130 includes a staining information separating module 131, a predicting module 132, and an overlaying module 133, wherein:
the staining information separating module 131 is configured to separate the first staining marker in the second staining image to obtain a first staining channel;
the prediction module 132 is configured to input the first dyeing channel of the second dyeing image into the prediction model to obtain a second dyeing information prediction result of the second dyeing image;
the superimposing module 133 is configured to superimpose the second dye prediction result in the second dye image.
Experimental example 1
The virtual staining method of example 2 is performed, wherein the first staining image is a tissue section staining image as shown in fig. 5, and the first staining image comprises first staining information (i.e. cell nucleus staining information) and second staining information (i.e. CD3 staining information); the second staining image is a staining image of the tissue section as shown in fig. 11, which includes the first staining information, i.e., the cell nucleus staining information and the fourth staining information (i.e., the MUC1 staining information), and does not include the second staining information (i.e., the CD3 staining information) and the third staining information (i.e., the CD4 staining information); the first staining image and the second staining image are staining images of tumor infiltrating lymphocyte tissue sections.
Specifically, the virtual staining method comprises the following steps:
(1) separating the nuclear staining channel (shown in FIG. 6) and the CD3 staining channel (shown in FIG. 7) for obtaining the first staining pattern; in the training set, a cell nucleus staining channel is used as an input value, a CD3 staining channel is used as an output value, a multilayer complete convolution neural network is trained, and a prediction model for predicting a CD3 staining result according to the cell nucleus staining channel is obtained; in the calibration set, a cell nucleus staining channel is input into the prediction model to obtain a CD3 prediction staining result (shown in figure 8), and the accuracy of the prediction model is 88.9% through statistics;
(2) separating the cell nucleus staining channel of the second staining map, inputting the cell nucleus staining channel into the prediction model in the step (1), predicting to obtain a staining result of CD3, and superposing the predicted staining result of CD3 and the second staining map to obtain a CD3 virtual staining result of the second staining map (as shown in FIG. 12);
(3) establishing a prediction model for predicting the staining result of CD4 according to cell nucleus by referring to the step (1), wherein FIG. 9 is the true staining condition of CD4, FIG. 10 is the staining prediction result of CD4, and the accuracy of the prediction model is 87.2% by statistics;
(4) referring to the step (2), the staining result of CD4 is predicted, and the predicted staining result of CD4 is superimposed on fig. 12, so as to obtain the virtual staining results of CD3 and CD4 of the second staining pattern (as shown in fig. 12-13).
The staining results described in fig. 13 allow analysis of the positional relationship of infiltrating T cells to MUC1 expression: approximately 1/4T cells were recruited around MUC1 positive cells with less helper T cells recruited around MUC1 positive cells.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (27)
1. A method for virtual staining of cells, the method comprising the steps of:
(1) acquiring a first staining image, wherein the first staining image is a cell staining image and comprises first staining information and second staining information;
(2) establishing a prediction model for predicting second dyeing information according to the first dyeing information and verifying whether the prediction model is qualified or not in a machine learning mode by taking the first dyeing image as a learning object, wherein the qualified prediction model is used for carrying out virtual dyeing on the second dyeing image;
(3) obtaining a second stain image, wherein the second stain image is a cell stain image and comprises the first stain information but does not comprise the second stain information;
(4) inputting the first dyeing information of the second dyeing image into the qualified prediction model, predicting to obtain the second dyeing information of the second dyeing image and superposing the second dyeing information on the second dyeing image, namely obtaining a virtual dyeing result of the second dyeing information on the second dyeing image;
the first staining information is selected from one or more of nuclear staining information, cytoplasmic staining information, cytoskeletal staining information, or molecular marker staining information;
the second staining information is different from the first staining information, and the second staining information is selected from nuclear staining information, cytoplasmic staining information, fiber staining information, or molecular marker staining information.
2. The method of claim 1, wherein the second stain image further comprises additional stain information obtained by direct staining and/or virtual staining.
3. The method of claim 2, wherein one or two of the other staining information is obtained by direct staining and the remaining other staining information is obtained by virtual staining.
4. The method according to claim 2, wherein the virtual staining mode of the other staining information is: and (3) acquiring to perform virtual dyeing in the manner of the step (1) to the step (4), wherein a first dyeing image used by other dyeing information in the virtual dyeing process is the same as or different from a first dyeing image used by second dyeing information in the virtual dyeing process, and first dyeing information used by the other dyeing information in the virtual dyeing process is the same as or different from first dyeing information used by the second dyeing information in the virtual dyeing process.
5. The method according to claim 1, characterized in that said step (2) comprises in particular the steps of:
a. separating first dyeing information and second dyeing information of the first dyeing image to obtain a first dyeing channel and a second dyeing channel;
b. segmenting the first staining image into a plurality of small images, and acquiring a data set, wherein the data set comprises a first staining channel and a second staining channel of each small image;
c. dividing the data set into a training set and a check set;
d. establishing a machine learning task in the training set by taking the first dyeing channel of each small graph as an input value and the corresponding second dyeing channel as an output value to obtain a model for predicting second dyeing information through the first dyeing information;
e. the specificity and sensitivity of the model are verified using the calibration set, and the qualified specificity and sensitivity model is used to predict second staining information not included in the second staining image.
6. The method of claim 5, wherein the machine learning is neural network learning or probabilistic graphical model learning.
7. The method of claim 5, wherein the machine learning is neural network learning.
8. The method of claim 7, wherein the neural network is learned as a multi-layer full convolution neural network model.
9. The method according to claim 1, characterized in that said step (4) comprises in particular the steps of:
f. separating first dyeing information of the second dyeing image to obtain a first dyeing channel;
g. inputting the first staining channel into the model, predicting a second staining channel of the second staining image;
i. superimposing the second dye channel on the second dye image.
10. The method of claim 1, wherein the first staining information is nuclear staining information.
11. The method of claim 10, wherein the nuclear staining information is hematoxylin staining information or 4',6-diamidino-2-phenylindole (4',6-diamidino-2-phenylindole, DAPI) staining information.
12. The method of claim 1, wherein the second staining information is molecular marker staining information.
13. The method of claim 12, wherein the molecular marker is a specific marker expressed by a non-allogeneic cell or a specific marker expressed by a xenogeneic cell.
14. The method of claim 13, wherein the specific marker expressed by the non-allogeneic cells is selected from the group consisting of CD3, CD4, and CD8, and the specific marker expressed by the allogeneic cells is selected from the group consisting of CD34, MUC1, and P53.
15. The method of claim 1, wherein the first stained image is obtained from a tissue section, a cell smear, or a cell slide; the second stained image is obtained from a tissue section, a cell smear, or a cell slide.
16. The method of claim 1, wherein the sample sources of the first stain image and the second stain image are homogeneous samples or non-homogeneous samples comprising homogeneous cells.
17. The method of claim 1, wherein the sample sources of the first stain image and the second stain image are homogeneous samples.
18. The method of claim 17, wherein the sample sources of the first stain image and the second stain image are homogeneous samples of the same subject.
19. The method of claim 18, wherein the homogeneous sample of the same subject is adjacent tissue of the same subject.
20. The method of claim 2, wherein the additional staining information is selected from nuclear staining information, cytoplasmic staining information, cytoskeletal staining information, or molecular marker staining information.
21. The method of claim 2, wherein the other staining information is molecular marker staining information.
22. The method of claim 21, wherein the molecular marker is a specific marker expressed by a non-allogeneic cell or a specific marker expressed by a xenogeneic cell.
23. The method of claim 22, wherein the specific marker expressed by the non-allogeneic cells is selected from the group consisting of CD3, CD4, and CD8, and the specific marker expressed by the allogeneic cells is selected from the group consisting of CD34, MUC1, and P53.
24. The method of claim 20, wherein the second staining information and the other staining information are staining information of co-expressed or proximally expressed molecular markers.
25. A system for use in a method for virtual staining of cells according to any of claims 1 to 24, the system comprising an image acquisition module, a model building module and a virtual staining module, wherein:
the image acquisition module is used for acquiring a first dyeing image and a second dyeing image;
the model establishing module is used for establishing a prediction model for predicting second dyeing information according to the first dyeing information in a machine learning mode and detecting whether the prediction model is qualified or not;
the virtual dyeing module is used for inputting the first dyeing information of the second dyeing image into the prediction model so as to predict and obtain the second dyeing information, and the second dyeing information is superposed on the second dyeing image.
26. The system of claim 25, wherein the model building module comprises a staining information separation module, an image segmentation module, a data processing module, a training module, and a verification module; wherein:
the dyeing information separation module is used for separating first dyeing information and second dyeing information of the first dyeing image to obtain a first dyeing channel and a second dyeing channel;
the image segmentation module is used for segmenting the first dyeing image into a plurality of small images;
the data processing module is used for acquiring a first dyeing channel and a second dyeing channel of each small picture to form a data set, and dividing the data set into a training set and a check set;
the training module takes a first dyeing channel of each small graph in the training set as an input value and corresponding second dyeing information as an output value, and obtains a prediction model for predicting the second dyeing information according to the first dyeing information in a machine learning mode;
the checking module is used for verifying the accuracy of the predicting module and using the model with qualified accuracy for predicting the second dyeing information which is not contained in the second dyeing image.
27. The system of claim 25, wherein the virtual staining module comprises a staining information separation module, a prediction module, and an overlay module, wherein:
the dyeing information separation module is used for separating the first dyeing information of the second dyeing image so as to obtain a first dyeing channel;
the prediction module is used for inputting the first dyeing channel of the second dyeing image into the prediction model so as to obtain a second dyeing information prediction result of the second dyeing image;
the superposition module is used for superposing the second dyeing information prediction result in the second dyeing image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810109960.XA CN108319815B (en) | 2018-02-05 | 2018-02-05 | Method and system for virtually staining cells |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810109960.XA CN108319815B (en) | 2018-02-05 | 2018-02-05 | Method and system for virtually staining cells |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108319815A CN108319815A (en) | 2018-07-24 |
| CN108319815B true CN108319815B (en) | 2020-07-24 |
Family
ID=62902441
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810109960.XA Active CN108319815B (en) | 2018-02-05 | 2018-02-05 | Method and system for virtually staining cells |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108319815B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2567155B (en) | 2017-10-02 | 2022-03-02 | Room4 Group Ltd | Histopathological image analysis |
| CN110070547A (en) * | 2019-04-18 | 2019-07-30 | 北京市商汤科技开发有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN110057743A (en) * | 2019-05-06 | 2019-07-26 | 西安交通大学 | The label-free cell three-dimensional Morphology observation method of blood film based on optic virtual dyeing |
| CN113256617B (en) * | 2021-06-23 | 2024-02-20 | 重庆点检生物科技有限公司 | Pathological section virtual immunohistochemical staining method and system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106248559A (en) * | 2016-07-14 | 2016-12-21 | 中国计量大学 | A kind of leukocyte five sorting technique based on degree of depth study |
| CN106650796A (en) * | 2016-12-06 | 2017-05-10 | 国家纳米科学中心 | Artificial intelligence based cell fluorescence image classification method and system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7970718B2 (en) * | 2001-05-18 | 2011-06-28 | Health Discovery Corporation | Method for feature selection and for evaluating features identified as significant for classifying data |
| US7774143B2 (en) * | 2002-04-25 | 2010-08-10 | The United States Of America As Represented By The Secretary, Department Of Health And Human Services | Methods for analyzing high dimensional data for classifying, diagnosing, prognosticating, and/or predicting diseases and other biological states |
-
2018
- 2018-02-05 CN CN201810109960.XA patent/CN108319815B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106248559A (en) * | 2016-07-14 | 2016-12-21 | 中国计量大学 | A kind of leukocyte five sorting technique based on degree of depth study |
| CN106650796A (en) * | 2016-12-06 | 2017-05-10 | 国家纳米科学中心 | Artificial intelligence based cell fluorescence image classification method and system |
Non-Patent Citations (2)
| Title |
|---|
| 局部二维结构描述的 HEp-2 染色模式分类;张东波等;《电子学报》;20161231;第44卷(第12期);第2817-2822页 * |
| 融合纹理与形状特征的 HEp-2 细胞分类;文登伟等;《电子与信息学报》;20170731;第39卷(第7期);第1600-1605页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108319815A (en) | 2018-07-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108319815B (en) | Method and system for virtually staining cells | |
| JP4376058B2 (en) | Quantitative video microscopy and related systems and computer software program products | |
| US8379962B2 (en) | Image classifier training | |
| EP3469548B1 (en) | System for bright field image simulation | |
| EP2697770B1 (en) | Method for preparing quantitative video-microscopy and associated system | |
| AU2003236675A1 (en) | Method for quantitative video-microscopy and associated system and computer software program product | |
| US20100054560A1 (en) | Breast cancer pathological image diagnosis support system, breast cancer pathological image diagnosis support method, and recording medium recording breast cancer pathological image diagnosis support program | |
| BRPI0610115A2 (en) | methods for image analysis based on chromogen separation | |
| WO2003025554A2 (en) | Method quantitative video-microscopy and associated system and computer software program product | |
| AU2002334590A1 (en) | Method quantitative video-microscopy and associated system and computer software program product | |
| CN106462767A (en) | Examining device for processing and analyzing an image | |
| US20180357816A1 (en) | Automated Method of Predicting Efficacy of Immunotherapy Approaches | |
| KR20170007181A (en) | Spatial multiplexing of histological stains | |
| CN111656393A (en) | Histological image analysis | |
| KR20110072098A (en) | Method for normalizing image in digital pathology | |
| Solorzano et al. | Towards automatic protein co-expression quantification in immunohistochemical TMA slides | |
| US9122904B2 (en) | Method for optimization of quantitative video-microscopy and associated system | |
| CN117529750A (en) | Digital synthesis of histological staining using multiple immunofluorescence imaging | |
| CN114898862A (en) | Cervical cancer computer-aided diagnosis method based on convolutional neural network and pathological section image | |
| US11908130B2 (en) | Apparatuses and methods for digital pathology | |
| Baird et al. | Multiplex: From Acquisition to Analysis | |
| JP7215418B2 (en) | Image processing apparatus, image processing method, and pathological diagnosis support system using the same | |
| Liu et al. | A Robust Approach to Locate HER2 and CEN17 Signals in Varied FISH Images | |
| CN116524323A (en) | Construction method of multi-center pathological image cell detection deep learning model | |
| DEMO | The Principles of Colocalization Microscopy and Image Analysis 23 Aug, 2022| Blog posts, Prisma |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |