WO2018143406A1 - Image processing device and program - Google Patents

Abstract

The present invention addresses the problem of providing an image processing device and a program whereby an expression pattern of a biological substance in a tissue specimen can be quantitatively evaluated, and the expression level of the biological substance for each cell can easily be recognized visually. The present invention is provided with an input means for inputting a form image representing the form of a cell and a fluorescence image in the same range as the form image and in which expression of the biological substance is represented by fluorescent bright points in a tissue specimen in which one or a plurality of types of biological substances are stained, a first extraction means for extracting a cell region from the form image, a second extraction means for extracting a fluorescent bright point region from the fluorescence image, a generating means for calculating the expression level of the biological substance from the number of fluorescent bright point regions extracted by the second extraction means and generating expression pattern information 233 including the expression level, and a classifying means for separating cells into classes in accordance with the expression pattern information generated by the generating means.

Description

Image processing apparatus and program

The present invention relates to an image processing apparatus and a program.

In pathological diagnosis, the degree of malignancy of a lesion is determined on the basis of an expression pattern such as an expression level of a specific biological substance in a tissue specimen and a distribution in a cell.
Specifically, for example, the presence or absence of expression of a specific biological material for each cell is determined from a microscopic image obtained by photographing a tissue specimen stained by an immunohistochemical method using an enzyme (DAB) or a fluorescent substance, Used for diagnosis.

For example, overexpression of HER2 protein is observed in many types of cancer such as breast cancer, lung cancer, and colon cancer. The HER2 protein is a glycoprotein present on the cell surface and is involved in cell proliferation and differentiation in normal cells, but when overexpressed, it is involved in malignant cell formation and acts as an oncogene. In order to perform molecular target therapy (anti-HER2 therapy) targeting the HER2 gene, a technique capable of accurately detecting overexpression of HER2 protein in a patient tissue sample is indispensable.

In addition, in order to select an effective treatment method according to the type of lesion, a technique capable of grasping the expression status of a plurality of types of biological substances in a tissue specimen is also required.
For example, breast cancer is classified into five subtypes based on the combination of the presence or absence of expression of hormone receptors (estrogen receptor (ER) and progesterone receptor (PgR)), HER2, and Ki67. Since the properties of cancer cells differ depending on the subtype, it is necessary to select appropriate drug therapies (for example, chemotherapy, hormone therapy, anti-HER2 therapy), so it is necessary to confirm the expression patterns of these biological substances, respectively. There is.

On the other hand, the microenvironment surrounding cancer cells is also known to have a significant effect on the growth of cancer, and has attracted attention in recent years. Macrophages are important cells that form a microenvironment of cancer together with fibroblasts and vascular endothelial cells, and it is known that many macrophages exist around cancer cells. Macrophages are divided into two phenotypes, which are pro-inflammatory M1 type and anti-inflammatory M2 type, which have completely different physiological roles. Macrophages infiltrating tumor tissue are tumor-associated macrophages (Tumor-associated macrophages). , TAM).

TAM is known to consist mainly of M2 macrophage populations, and TAM is known to promote cell proliferation and cancer metastasis by effectively suppressing T cell activity and regulating signal transduction. It has been. Clinical studies have also revealed an association between TAM status and poor prognosis in human tumors, and TAM is now considered a promising target for tumor therapy. Therefore, there is a demand for a technique for detecting a target protein in macrophages with high accuracy.

In response to such a problem, for example, Patent Document 1 discloses a technique in which a plurality of biological materials are dyed with pigments of different colors and the expression pattern of each biological material is observed.
That is, a microscopic image of a tissue specimen stained with cells and biological materials is acquired, and an image stained with cells and an image stained with biological materials are superimposed using an image processing apparatus, and the expression sites of the cells and biological materials are respectively determined. Extract from image. By comparing the dye amount of the stained biological material with a predetermined threshold, the expression pattern is classified for each cell, and the classification result is displayed on the display for observation. According to this method, the expression patterns of a plurality of biological substances can be observed on the same screen, and the diagnostic accuracy can be improved.

JP 2012-37432 A

However, when the biological material is stained with a staining dye using an enzyme such as DAB as in Patent Document 1 and the expression pattern is classified based on the dye strength, the expression pattern is not quantified. It is difficult to subdivide. Alternatively, when dyeing is performed using a fluorescent substance such as an organic fluorescent dye alone, if the light resistance is not sufficient, fading occurs during fluorescence observation, and the expression pattern cannot be quantitatively evaluated. there were.
Furthermore, in the technique described in Patent Document 1, when the information is displayed on the display, the information indicating the expression level of the biological substance is displayed outside the cell morphology image, and thus the expression level is observed together with the cell distribution. Therefore, it is necessary for the operator to observe the cell distribution and the expression level sequentially corresponding to each other. Such an observation method is inconvenient in situations where quick diagnosis is required, for example, quick diagnosis during surgery.

The present invention has been made in view of the above problems, and an object thereof is to provide an image processing apparatus and program capable of quantitatively evaluating an expression pattern of a biological substance in a tissue specimen.

In order to achieve the above object, an image processing apparatus according to claim 1 is provided.
An input means for inputting a morphological image representing the morphology of a cell and a fluorescent image representing the expression of the biological material in the same range as the morphological image with a fluorescent luminescent spot in a tissue specimen stained with a single or plural types of biological materials; ,
First extraction means for extracting a cell region from the morphological image;
Second extraction means for extracting a fluorescent bright spot region from the fluorescent image;
Generating means for calculating an expression level of the biological material extracted from the number of the fluorescent bright spot regions extracted by the second extraction means, and generating expression pattern information including the expression level;
Classification means for classifying cells according to the expression pattern information generated by the generation means.

The invention according to claim 2 is the image processing apparatus according to claim 1,
Display control means for displaying the expression pattern information generated by the generation means on a display means;
The display control means displays the expression pattern information and the morphological image in a superimposed manner.

The invention according to claim 3 is the image processing apparatus according to claim 2,
The display control means changes and displays the expression pattern information display method for each cell class classified by the classification means.

The invention according to claim 4 is the image processing apparatus according to claim 2 or 3,
The display control means displays the expression pattern information so as not to overlap each other.

The invention according to claim 5 is the image processing apparatus according to any one of claims 2 to 4,
The display control means displays the expression pattern information in a color different from the color of the morphological image.

The invention according to claim 6 is the image processing apparatus according to any one of claims 1 to 5,
A specifying means for specifying a cell type based on the feature amount of the cell region extracted by the first extracting means is provided.

The program according to claim 7 is:
The computer of the image processing device
Input means for inputting a morphological image representing cell morphology and a fluorescent image representing the expression of the biological material in the same range as the morphological image with fluorescent luminescent spots in a tissue specimen stained with a single or plural types of biological materials,
First extraction means for extracting a cell region from the morphological image;
Second extraction means for extracting a fluorescent bright spot region from the fluorescent image;
Generating means for calculating an expression level of the biological material extracted from the number of the fluorescent bright spot regions extracted by the second extraction means, and generating expression pattern information including the expression level;
Classification means for classifying cells according to the expression pattern information generated by the generation means,
To function as.

According to the present invention, it is possible to provide an image processing apparatus and program capable of quantitatively evaluating the expression pattern of a biological substance in a tissue specimen.

It is a figure which shows the system configuration | structure of the pathological-diagnosis assistance system using the image processing apparatus of this invention. It is a block diagram which shows the functional structure of the image processing apparatus of FIG. It is a figure which shows an example of a bright field image. It is a figure which shows an example of a fluorescence image. It is a flowchart which shows the image analysis process performed by the control part of FIG. It is a flowchart which shows the detail of the process of step S2 of FIG. It is a figure which shows the image from which the bright field image was extracted. It is a figure which shows the image from which the cell nucleus was extracted. It is a flowchart which shows the detail of the process of step S3 of FIG. It is a flowchart which shows the detail of the process of FIG.5 S5. It is a figure which shows the image from which the fluorescence image was extracted. It is a figure which shows the image from which the luminescent spot area | region was extracted. It is a flowchart which shows the detail of the process of step S7 of FIG. It is a flowchart which shows the detail of the process of step S8 of FIG. It is a figure which shows an example of the display screen in the display part of FIG. It is a figure which shows an example of the setting method of the coordinate of a cell. It is a figure which shows an example of the display position of the expression pattern information before correction | amendment. It is a figure which shows an example of the display position correction method of expression pattern information. It is a figure which shows an example of the display position correction method of expression pattern information.

[Embodiment 1]
Hereinafter, although a 1st embodiment for carrying out the present invention is described with reference to figures, the present invention is not limited to these.

<Configuration of Pathological Diagnosis Support System 100>
FIG. 1 shows an example of the overall configuration of the pathological diagnosis support system 100.
The pathological diagnosis support system 100 acquires a microscopic image of a tissue section of a human body stained with a predetermined staining reagent, and analyzes the acquired microscopic image, thereby expressing the expression of a specific biological material in the tissue section to be observed. This is a system that outputs feature quantities quantitatively.

As shown in FIG. 1, the pathological diagnosis support system 100 is configured by connecting a microscope image acquisition apparatus 1A and an image processing apparatus 2A so that data can be transmitted and received via an interface such as a cable 3A.
The connection method between the microscope image acquisition apparatus 1A and the image processing apparatus 2A is not particularly limited. For example, the microscope image acquisition device 1A and the image processing device 2A may be connected via a LAN (Local Area Network) or may be connected wirelessly.

The microscope image acquisition apparatus 1A is a known camera-equipped microscope, which acquires a microscope image of a tissue section on a slide placed on a slide fixing stage and transmits it to the image processing apparatus 2A.
The microscope image acquisition apparatus 1A includes an irradiation unit, an imaging unit, an imaging unit, a communication I / F, and the like. The irradiating means includes a light source, a filter, and the like, and irradiates the tissue section on the slide placed on the slide fixing stage with light. The imaging means is composed of an eyepiece lens, an objective lens, and the like, and forms an image of transmitted light, reflected light, or fluorescence emitted from the tissue section on the slide by the irradiated light. The image pickup means is a microscope-installed camera that includes a CCD (Charge Coupled Device) sensor and the like, picks up an image formed on the image forming surface by the image forming means, and generates digital image data of the microscope image. The communication I / F transmits image data of the generated microscope image to the image processing apparatus 2A.
The microscope image acquisition apparatus 1A includes a bright field unit that combines an irradiation unit and an imaging unit suitable for bright field observation, and a fluorescence unit that combines an irradiation unit and an imaging unit suitable for fluorescence observation. It is possible to switch between bright field / fluorescence by switching.
In addition, what installed the camera in well-known arbitrary microscopes (For example, a phase-contrast microscope, a differential interference microscope, an electron microscope, etc.) can be used as 1 A of microscope image acquisition apparatuses.

Note that the microscope image acquisition apparatus 1A is not limited to a microscope with a camera. For example, a virtual microscope slide creation apparatus (for example, a special microscope scan apparatus that acquires a microscope image of an entire tissue section by scanning a slide on a microscope slide fixing stage). Table 2002-514319) may be used. According to the virtual microscope slide creation device, it is possible to acquire image data that allows a display unit to view a whole tissue section on a slide at a time.

The image processing apparatus 2A calculates the expression distribution of a specific biological material in the tissue section to be observed by analyzing the microscope image transmitted from the microscope image acquisition apparatus 1A.
FIG. 2 shows a functional configuration example of the image processing apparatus 2A.
As shown in FIG. 2, the image processing apparatus 2 </ b> A includes a control unit 21, an operation unit 22, a display unit 23, a communication I / F 24, a storage unit 25, and the like, and each unit is connected via a bus 26. Yes.

The control unit 21 includes a CPU (Central Processing Unit) and a RAM (Random Access Memory).
) And the like, and various processes are executed in cooperation with various programs stored in the storage unit 25, and the operation of the image processing apparatus 2A is comprehensively controlled.
For example, the control unit 21 executes image analysis processing in cooperation with the image processing program stored in the storage unit 25, and includes a first extraction unit, a second extraction unit, a generation unit, a classification unit, a display control unit, and A function as a specifying means is realized.

The operation unit 22 includes a keyboard having character input keys, numeric input keys, various function keys, and the like, and a pointing device such as a mouse, and includes a key press signal pressed by the keyboard and an operation signal by the mouse. Is output to the control unit 21 as an input signal.

The display unit 23 includes a monitor such as a CRT (Cathode Ray Tube) or an LCD (Liquid Crystal Display), and displays various screens in accordance with display signal instructions input from the control unit 21. A function as a means is realized.

The communication I / F 24 is an interface for transmitting and receiving data to and from external devices such as the microscope image acquisition device 1A. The communication I / F 24 realizes a function as an input unit for fluorescent images and morphological images.

The storage unit 25 is configured by, for example, an HDD (Hard Disk Drive), a semiconductor nonvolatile memory, or the like. As described above, the storage unit 25 stores various programs and various data.
In addition, the image processing apparatus 2A may include a LAN adapter, a router, and the like, and be connected to an external device via a communication network such as a LAN.

<About images>
In the present embodiment, the image processing apparatus 2A is, for example, a fluorescence image that is transmitted from the microscope image acquisition apparatus 1A and that expresses the expression of a specific biological material in a cell with a fluorescent bright spot, and the form of the whole cell, cell nucleus, cell membrane It is preferable to perform the analysis using a morphological image (for example, a bright field image) representing the form of a predetermined structure of the cell.

The “bright field image” refers to, for example, a tissue section stained with a hematoxylin staining reagent (H staining reagent) or a hematoxylin-eosin staining reagent (HE staining reagent) and magnified in a bright field in the microscope image acquisition apparatus 1A. It is a microscope image obtained by image and imaging | photography, Comprising: It is a cell form image showing the form of the cell in the said tissue section. FIG. 3 shows an example of a bright field image. Hematoxylin (H) is a blue-violet pigment that stains cell nuclei, bone tissue, part of cartilage tissue, serous components, etc. (basophilic tissue, etc.). Eodine (E) is a red to pink pigment that stains cytoplasm, connective tissue of soft tissues, erythrocytes, fibrin, endocrine granules (acidophilic tissues, etc.).
As cell morphology images, in addition to bright-field images, tissue sections are stained with a fluorescent staining reagent that can specifically stain the structure to be diagnosed of cells, and the fluorescence emitted by the fluorescent staining reagent used is photographed. Fluorescent images may be used. Examples of the fluorescent staining reagent that can be used for obtaining a morphological image include DAPI staining capable of staining cell nuclei, Papalonikolou staining capable of staining cytoplasm, and the like. Moreover, you may use a phase difference image, a differential interference image, an electron microscope image, etc. as a form image.

A “fluorescence image” that expresses the expression of a specific biological substance in a cell as a fluorescent bright spot is a fluorescence obtained by irradiating a tissue section stained with a fluorescent staining reagent with excitation light having a predetermined wavelength in the microscope image acquisition apparatus 1A. It is a microscope image obtained by emitting a substance and enlarging and photographing this fluorescence. FIG. 4 shows an example of the fluorescence image.
In the present invention, the fluorescent staining reagent refers to fluorescent nanoparticles that specifically bind to and / or react with a specific biological substance. The “fluorescent nanoparticle” is a nano-sized particle that emits fluorescence when irradiated with excitation light, as will be described in detail later, and is sufficient to express a specific biological substance as a bright spot one molecule at a time. It is a particle capable of emitting intense fluorescence.
As the fluorescent nanoparticles, preferably, quantum dots (semiconductor nanoparticles) or fluorescent substance-containing nanoparticles are used. Preferably, nanoparticles having an emission wavelength within the sensitivity range of the imaging means of the microscope image acquisition apparatus 1A, specifically, nanoparticles having an emission wavelength of 400 to 700 nm are used.

<Fluorescent staining reagents and staining methods>
Hereinafter, a fluorescent staining reagent for obtaining a fluorescent image in which the expression of a specific biological substance specifically expressed in cells is expressed by a fluorescent luminescent spot and a staining method of a tissue section using the fluorescent staining reagent will be described.

(1) Fluorescent substance Examples of fluorescent substances used in fluorescent staining reagents include fluorescent organic dyes and quantum dots (semiconductor particles). When excited by ultraviolet to near infrared light having a wavelength in the range of 200 to 700 nm, it preferably emits visible to near infrared light having a wavelength in the range of 400 to 1100 nm.

Examples of fluorescent organic dyes include fluorescein dye molecules, rhodamine dye molecules, Alexa Fluor (Invitrogen) dye molecules, BODIPY (Invitrogen) dye molecules, cascade dye molecules, coumarin dye molecules, and eosin dyes. Examples include molecules, NBD dye molecules, pyrene dye molecules, Texas Red dye molecules, and cyanine dye molecules.
Specifically, 5-carboxy-fluorescein, 6-carboxy-fluorescein, 5,6-dicarboxy-fluorescein, 6-carboxy-2 ′, 4,4 ′, 5 ′, 7,7′-hexachlorofluorescein, 6 -Carboxy-2 ', 4,7,7'-tetrachlorofluorescein, 6-carboxy-4', 5'-dichloro-2 ', 7'-dimethoxyfluorescein, naphthofluorescein, 5-carboxy-rhodamine, 6-carboxy Rhodamine, 5,6-dicarboxy-rhodamine, rhodamine 6G, tetramethylrhodamine, X-rhodamine, and Alexa Fluor 350, Alexa Fluor 405, Alexa Fluor 430, Alexa Fluor 488, Alexa Fluor 500, Alexa Fluor 514, lexa Fluor 532, Alexa Fluor 546, Alexa Fluor 555, Alexa Fluor 568, Alexa Fluor 594, Alexa Fluor 610, Alexa Fluor 633, Alexa Fluor 635, Alexa Fluor 635, Alexa Fluor 635, Alexa Fluor 633 750, BODIPY FL, BODIPY TMR, BODIPY 493/503, BODIPY 530/550, BODIPY 558/568, BODIPY 564/570, BODIPY 576/589, BODIPY 581/591, BODIPY 630/650, BODIP 630/650, BOD Jae ), Methoxycoumarin, eosin, NBD, pyrene, Cy5, Cy5.5, Cy7, and the like. These fluorescent organic dyes may be used alone or as a mixture of plural kinds.

As quantum dots, quantum dots containing II-VI group compounds, III-V group compounds, or group IV elements as components ("II-VI group quantum dots", "III-V group quantum dots", " Or “Group IV quantum dots”). These quantum dots may be used alone or in combination of a plurality of types.
Specific examples include, but are not limited to, CdSe, CdS, CdTe, ZnSe, ZnS, ZnTe, InP, InN, InAs, InGaP, GaP, GaAs, Si, and Ge.

It is also possible to use a quantum dot having the above quantum dot as a core and a shell provided thereon. In the following, as a notation of quantum dots having a shell, when the core is CdSe and the shell is ZnS, it is expressed as CdSe / ZnS.
For example, CdSe / ZnS, CdS / ZnS , InP / ZnS, InGaP / ZnS, Si / SiO 2, Si / ZnS, Ge / GeO 2, Ge / ZnS and the like may be used, but are not limited to.

Quantum dots may be subjected to surface treatment with an organic polymer or the like as necessary. Examples thereof include CdSe / ZnS having a surface carboxy group (manufactured by Invitrogen), CdSe / ZnS having a surface amino group (manufactured by Invitrogen), and the like.

(2) Fluorescent substance-encapsulated nanoparticles “Fluorescent substance-encapsulated nanoparticles” are nanoparticles encapsulating the fluorescent substance as described above, and more specifically, those in which the fluorescent substance is dispersed inside the nanoparticles, The fluorescent substance and the nanoparticles themselves may be chemically bonded or may not be bonded.
The material constituting the nanoparticles is not particularly limited, and examples thereof include silica, polystyrene, polylactic acid, and melamine.

The fluorescent substance-containing nanoparticles can be produced by a known method.
For example, silica nanoparticles encapsulating a fluorescent organic dye can be synthesized with reference to the synthesis of FITC-encapsulated silica particles described in Langmuir 8, Vol. 2921 (1992). Various fluorescent organic dye-containing silica nanoparticles can be synthesized by using a desired fluorescent organic dye in place of FITC.
Silica nanoparticles encapsulating quantum dots can be synthesized with reference to the synthesis of CdTe-encapsulated silica nanoparticles described in New Journal of Chemistry, Vol. 33, p. 561 (2009).
Polystyrene nanoparticles encapsulating a fluorescent organic dye may be copolymerized using an organic dye having a polymerizable functional group described in US Pat. No. 4,326,008 (1982) or polystyrene described in US Pat. No. 5,326,692 (1992). It can be produced using a method of impregnating nanoparticles with a fluorescent organic dye.
The polymer nanoparticles encapsulating the quantum dots can be prepared by using the method of impregnating the quantum nanoparticles into polystyrene nanoparticles described in Nature Biotechnology, Vol. 19, page 631 (2001).

The average particle diameter of the fluorescent substance-containing nanoparticles is not particularly limited, but those having a size of about 30 to 800 nm can be used. Further, the coefficient of variation (= (standard deviation / average value) × 100%) indicating the variation in particle diameter is not particularly limited, but it is preferable to use a coefficient of 20% or less.
The average particle diameter is obtained by taking an electron micrograph using a scanning electron microscope (SEM), measuring the cross-sectional area of a sufficient number of particles, and taking each measured value as the area of the circle. It is the value calculated as. In this embodiment, the arithmetic average of the particle diameters of 1000 particles is defined as the average particle diameter. The coefficient of variation is also a value calculated from the particle size distribution of 1000 particles.

(3) Binding of biological material recognition site and fluorescent nanoparticle In this embodiment, fluorescent nanoparticle and biological material recognition site are directly bonded in advance as a fluorescent staining reagent that specifically binds and / or reacts with a specific biological material. An example of using the above will be described. A “biological substance recognition site” is a site that specifically binds and / or reacts with a specific biological material.
The specific biological substance is not particularly limited as long as a substance that specifically binds to the specific biological substance exists, but typically includes proteins (peptides) and nucleic acids (oligonucleotides, polynucleotides). It is done.
Therefore, examples of the biological substance recognition site include an antibody that recognizes the protein as an antigen, another protein that specifically binds to the protein, and a nucleic acid having a base sequence that hybridizes to the nucleic acid.
Specific biological substance recognition sites include anti-HER2 antibody that specifically binds to HER2, which is a protein present on the cell surface, anti-ER antibody that specifically binds to estrogen receptor (ER) present in the cell nucleus, cells An anti-actin antibody that specifically binds to actin forming the skeleton is exemplified.
Of these, anti-HER2 antibody and anti-ER antibody combined with fluorescent nanoparticles (fluorescent staining reagent) are preferable because they can be used for breast cancer medication selection.

The mode of binding between the biological substance recognition site and the fluorescent nanoparticle is not particularly limited, and examples thereof include covalent bonding, ionic bonding, hydrogen bonding, coordination bonding, physical adsorption, and chemical adsorption. A bond having a strong bonding force such as a covalent bond is preferable from the viewpoint of bond stability.

There may be an organic molecule that connects the biological substance recognition site and the fluorescent nanoparticle. For example, in order to suppress non-specific adsorption with a biological substance, a polyethylene glycol chain can be used, and SM (PEG) 12 manufactured by Thermo Scientific can be used.

When the biological substance recognition site is bound to the fluorescent substance-encapsulating silica nanoparticles, the same procedure can be applied regardless of whether the fluorescent substance is a fluorescent organic dye or a quantum dot.
For example, a silane coupling agent that is a compound widely used for bonding an inorganic substance and an organic substance can be used. This silane coupling agent is a compound having an alkoxysilyl group that gives a silanol group by hydrolysis at one end of the molecule and a functional group such as a carboxyl group, an amino group, an epoxy group, an aldehyde group at the other end, Bonding with an inorganic substance through an oxygen atom of the silanol group.
Specifically, mercaptopropyltriethoxysilane, glycidoxypropyltriethoxysilane, aminopropyltriethoxysilane, a silane coupling agent having a polyethylene glycol chain (for example, PEG-silane no. SIM6492.7 manufactured by Gelest), etc. Is mentioned.
When using a silane coupling agent, you may use 2 or more types together.

A known procedure can be used for the reaction procedure of the fluorescent organic dye-encapsulated silica nanoparticles and the silane coupling agent.
For example, the obtained fluorescent organic dye-encapsulated silica nanoparticles are dispersed in pure water, aminopropyltriethoxysilane is added, and the mixture is reacted at room temperature for 12 hours. After completion of the reaction, fluorescent organic dye-encapsulated silica nanoparticles whose surface is modified with an aminopropyl group can be obtained by centrifugation or filtration. Subsequently, by reacting an amino group with a carboxyl group in the antibody, the antibody can be bound to the fluorescent organic dye-encapsulated silica nanoparticles via an amide bond. If necessary, a condensing agent such as EDC (1-Ethyl-3- [3-Dimethylaminopropyl] carbohydrate Hydrochloride: Pierce (registered trademark)) can also be used.

If necessary, a linker compound having a site that can be directly bonded to the fluorescent organic dye-encapsulated silica nanoparticles modified with organic molecules and a site that can be bonded to the molecular target substance can be used. As a specific example, sulfo-SMCC (Sulfosuccinimidyl 4 [N-maleimidomethyl] -cyclohexane-1-carboxylate: manufactured by Pierce) having both a site that selectively reacts with an amino group and a site that reacts selectively with a mercapto group When used, by binding the amino group of the fluorescent organic dye-encapsulated silica nanoparticles modified with aminopropyltriethoxysilane and the mercapto group in the antibody, antibody-bound fluorescent organic dye-encapsulated silica nanoparticles can be produced.

When binding a biological material recognition site to fluorescent substance-encapsulated polystyrene nanoparticles, the same procedure can be applied regardless of whether the fluorescent substance is a fluorescent organic dye or a quantum dot. That is, by impregnating a fluorescent nanopigment or quantum dot into polystyrene nanoparticles having a functional group such as an amino group, fluorescent substance-containing polystyrene nanoparticles having a functional group can be obtained, and thereafter using EDC or sulfo-SMCC. Thus, antibody-bound fluorescent substance-encapsulated polystyrene nanoparticles can be produced.

Examples of biological substance recognition sites include M. actin, MS actin, SM actin, ACTH, Alk-1, α1-antichymotrypsin, α1-antitrypsin, AFP, bcl-2, bcl-6, β-catenin, BCA 225 , CA19-9, CA125, calcitonin, calretinin, CD1a, CD3, CD4, CD5, CD8, CD10, CD15, CD20, CD21, CD23, CD30, CD31, CD34, CD43, CD45, CD45R, CD56, CD57, CD61, CD68 , CD79a, "CD99, MIC2", CD138, chromogranin, c-KIT, c-MET, collagen type IV, Cox-2, cyclin D1, keratin, cytokeratin (high molecular weight), pankeratin, pankeratin, cytokeratin 5 / 6, Cytokeratin 7, Cytokeratin 8, Cytokeratin 8/18, Cytokeratin 14, Cytokeratin 19, Cytokeratin 20, CMV, E-cadherin, EGFR, ER, EMA, EBV, Factor VIII related antigen, fascin , FSH, galectin-3, gas Phosphorus, GFAP, glucagon, glycophorin A, granzyme B, hCG, hGH, Helicobacter pylori, HBc antigen, HBs antigen, hepatocyte specific antigen, HER2, HSV-I, HSV-II, HHV-8, IgA, IgG, IgM, IGF-1R, Inhibin, Insulin, Kappa L chain, Ki67, Lambda L chain, LH, Lysozyme, Macrophage, Melan A, MLH-1, MSH-2, Myeloperoxidase, Myogenin, Myoglobin, Myosin, Neurofilament, NSE, p27 (Kip1), p53, P63, PAX 5, PLAP, Pneumocystis carini, podoplanin (D2-40), PGR, prolactin, PSA, prostatic acid phosphatase, Renal Cell Carcinoma, S100, somatostatin, spectrin, synaptophysin, TAG -72, TdT, thyroglobulin, TSH, TTF-1, TRAcP, tryptase, villin, vimentin, WT1, Zap-70, etc. It is done.

The fluorescent nanoparticles may be used by directly binding to the biological material recognition site in advance as described above, or may be indirectly bound to the biological material recognition site in the staining step as in a known indirect method in immunostaining. good. Specifically, for example, a tissue specimen is reacted with a biotinylated primary antibody having a specific biological substance as an antigen, and then further reacted with a staining reagent to which fluorescent nanoparticles modified with streptavidin are bound. Alternatively, staining may be performed using the fact that streptavidin and biotin specifically bind to form a complex. In addition, for example, a fluorescent nanoparticle modified with streptavidin after reacting a tissue sample with a primary antibody having a specific protein as an antigen and further reacting with a biotinylated secondary antibody having the primary antibody as an antigen. You may make it react and dye | stain.

(4) Staining method The method for preparing tissue sections is not particularly limited, and those prepared by known methods can be used. The following staining method is not limited to a pathological tissue section, but can also be applied to cultured cells.

(4.1) Deparaffinization process A tissue section is immersed in a container containing xylene to remove paraffin. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, xylene may be exchanged during the immersion.
Next, the tissue section is immersed in a container containing ethanol to remove xylene. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, ethanol may be exchanged during the immersion.
Next, the tissue section is immersed in a container containing water to remove ethanol. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, water may be exchanged during the immersion.

(4.2) Activation process The activation process of the biological material of a tissue section is performed according to a known method.
The activation conditions are not particularly defined, but as the activation liquid, 0.01 M citrate buffer (pH 6.0), 1 mM EDTA solution (pH 8.0), 5% urea, 0.1 M Tris-HCl buffer, etc. are used. be able to. An autoclave, a microwave, a pressure cooker, a water bath, etc. can be used for a heating apparatus. The temperature is not particularly limited, but can be performed at room temperature. The temperature can be 50 to 130 ° C. and the time can be 5 to 30 minutes.
Next, the tissue section after the activation treatment is immersed in a container containing PBS (Phosphate Buffered Saline) and washed. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, PBS may be exchanged during immersion.

(4.3) Staining using a fluorescent staining reagent A PBS dispersion liquid of a fluorescent staining reagent is placed on a tissue section and reacted with a biological material in the tissue section.
By changing the biological material recognition site of the fluorescent staining reagent, staining corresponding to various biological materials becomes possible. When using fluorescent nanoparticles combined with several kinds of biological material recognition sites as fluorescent staining reagents, each fluorescent nanoparticle PBS dispersion may be mixed in advance or separately placed on the tissue section separately. May be. The temperature is not particularly limited, but can be performed at room temperature. The reaction time is preferably 30 minutes or more and 24 hours or less.
It is preferable to drop a known blocking agent such as BSA-containing PBS before staining with a fluorescent staining reagent.
Next, the stained tissue section is immersed in a container containing PBS to remove unreacted fluorescent nanoparticles. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, PBS may be exchanged during immersion. A cover glass is placed on the tissue section and sealed. A commercially available encapsulant may be used as necessary.

In addition, when carrying out HE dyeing | staining etc. in order to obtain a form image, it carries out in the arbitrary steps before enclosure with a cover glass.

(5) Acquisition of fluorescence image A microscope image (fluorescence image) is acquired from the stained tissue section using the microscope image acquisition device 1A. As the excitation light source and the fluorescence detection optical filter, those corresponding to the absorption maximum wavelength and the fluorescence wavelength of the fluorescent material used in the fluorescent staining reagent are appropriately selected.

<Operation of Pathological Diagnosis Support System 100>
Hereinafter, an operation of acquiring and analyzing the above-described fluorescence image and bright field image in the pathological diagnosis support system 100 will be described.
Here, staining was performed using a staining reagent containing fluorescent substance-containing nanoparticles bound to a biological substance recognition site that recognizes a specific protein (for example, Ki67 protein in breast cancer tissue, hereinafter referred to as a specific protein). A case where a tissue specimen is an observation target will be described as an example. However, the present invention is not limited to this, and in the present invention, a plurality of types of biological substances can be dyed using fluorescent nanoparticles having different light emission characteristics and observed on the same screen.

First, the operator uses two types of staining reagents, a HE staining reagent and a staining reagent using fluorescent substance-encapsulated nanoparticles bound with a biological substance recognition site that recognizes a specific protein as a fluorescent labeling material. Stain.
Thereafter, in the microscope image acquisition apparatus 1A, a bright field image and a fluorescence image are acquired by the procedures (a1) to (a5).
(A1) An operator places a tissue specimen stained with a hematoxylin staining reagent and a staining reagent containing fluorescent substance-containing nanoparticles on a slide, and places the slide on a slide fixing stage of the microscope image acquisition apparatus 1A.
(A2) Set to the bright field unit, adjust the imaging magnification and focus, and place the observation target area on the tissue in the field of view.
(A3) Shooting is performed by the imaging unit to generate bright field image data, and the image data is transmitted to the image processing apparatus 2A.
(A4) Change the unit to a fluorescent unit.
(A5) Shooting is performed by the imaging means without changing the field of view and the shooting magnification to generate image data of a fluorescent image, and the image data is transmitted to the image processing apparatus 2A.

Note that the above step (a5) is repeated when a plurality of types of biological substances are stained. As the excitation light and the filter used for acquiring each fluorescent image, a combination suitable for the emission characteristics is appropriately selected.

In the image processing apparatus 2A, image analysis processing is executed based on the bright field image and the fluorescence image.
FIG. 5 shows a flowchart of image analysis processing in the image processing apparatus 2A. The image analysis processing shown in FIG. 5 is executed in cooperation with the control unit 21 and the program stored in the storage unit 25.

First, when a bright field image is input from the microscope image acquisition device 1A through the communication I / F 24 (step S1), a cell region is extracted from the bright field image (step S2).
FIG. 6 shows a detailed flow of the process in step S2. The process of step S2 is executed in cooperation with the control unit 21 and the program stored in the storage unit 25.
FIG. 7 shows an extracted image of cell nuclei as an example of the processing in step 2. In the present invention, this is applied to extract a cell region.

In step S2, first, a bright-field image is converted into a monochrome image (step S201). FIG. 7A shows an example of a bright field image.
Next, threshold processing is performed on the monochrome image using a predetermined threshold, and the value of each pixel is binarized (step S202).

Next, noise processing is performed (step S203). Specifically, the noise process can be performed by performing a closing process on the binary image. The closing process is a process in which the contraction process is performed the same number of times after the expansion process is performed. The expansion process is a process of replacing a target pixel with white when at least one pixel in the range of n × n pixels (n is an integer of 2 or more) from the target pixel is white. The contraction process is a process of replacing a target pixel with black when at least one pixel in the range of n × n pixels from the target pixel contains black. By the closing process, a small area such as noise can be removed. FIG. 7B shows an example of an image after noise processing. As shown in FIG. 7B, after noise processing, an image (cell image) from which cells are extracted is generated.

Next, a labeling process is performed on the image after the noise process, and a label is assigned to each of the extracted cells (step S204). The labeling process is a process for identifying an object in an image by assigning the same label (number) to connected pixels. By the labeling process, each cell can be identified from the image after the noise process and a label can be applied.

In step S3, the type of the extracted cell is specified.
FIG. 8 shows a detailed flow of the process in step S3.
In step S301, first, for all cells in the cell image extracted in step S2, from the cell image, the cell area A, the average cell density B, the pixel luminance variation (σ value) C in the cell region, the cell The “cell feature amount” such as the circularity D and the flatness E of the cell is calculated.
For the area A of the cell, the size of the pixel is calculated by measuring the reference length corresponding to the cell image in advance, and the number of pixels in each cell extracted in step S2 is integrated. Thus, the area A of the cell is determined.
The average density B of the cells is determined by obtaining the luminance signal value converted into the gray scale of each pixel (pixel) in the cell and calculating the average value.
The pixel luminance variation C is determined by calculating the standard deviation of the luminance signal value of each pixel (pixel) in the cell.
The circularity D and the flatness E of the cells are determined by applying a constant value obtained from the cell image to the following formulas (d) and (e) for each cell extracted in step S2.
(Circularity D) = 4πS / L2 (d)
(Flat ratio E) = (ab) / a (e)
In the formula (d), “S” represents a cell area (cell area A), and “L” represents a cell outer peripheral length. In formula (e), “a” represents the major radius and “b” represents the minor radius.

Next, in step S302, a threshold value process is performed on the cell feature amount obtained in step S301 using a predetermined threshold value, and a cell classification process is performed. For example, in the case of nerve cells having an elongated shape, the flatness E value is large, and in the case of spherical lymphocytes, the circularity D value is large. Therefore, by setting an appropriate threshold value for identifying these, Classification according to the characteristics of the cell shape can be performed. Each threshold of the cell classification item and the cell feature amount can be set based on the statistical value, and is tabulated in advance and stored in the storage unit 25.

The identification of the cell type in step 3 is basically automatically performed in cooperation with the program stored in the control unit 21 and the storage unit 25. For such processing, auxiliary work by an observer is performed. May be accompanied. The auxiliary work by the observer includes, for example, adjusting each threshold value of the cell feature amount in a stepwise manner with respect to the program stored in the storage unit 25 and visually confirming the specified cell type.
Each factor of the cell feature amount (A to E in the above case) may be arbitrarily selected and appropriately changed. Of course, another factor different from the above may be used as a factor of the cell feature amount.

On the other hand, when a fluorescent image is input from the microscope image acquisition device 1A through the communication I / F 24 (step S4), a bright spot region is extracted from the fluorescent image (step S5).
FIG. 9 shows a detailed flow of the process in step S5. The process of step S5 is executed in cooperation with the control unit 21 and the program stored in the storage unit 25.

In step S5, first, a color component corresponding to the wavelength of the fluorescent bright spot is extracted from the fluorescent image (step S501).
FIG. 10A shows an example of a fluorescence image.
In step S501, for example, when the emission wavelength of the fluorescent particles is 550 nm, only the fluorescent bright spot having the wavelength component is extracted as an image.

Next, threshold processing is performed on the extracted image, a binarized image is generated, and a bright spot region is extracted (step S502).
Note that noise removal processing such as cell autofluorescence and other unnecessary signal components may be performed before the threshold processing, and a low-pass filter such as a Gaussian filter or a high-pass filter such as a second derivative is preferably used.
FIG. 10B shows an example of an image from which the bright spot region is extracted. As shown in FIG. 10B, in such an image, a bright spot region centered on the fluorescent bright spot is extracted.

Next, a labeling process is performed on the bright spot area, and a label is assigned to each of the extracted bright spot areas (step S503).

After the process of step S3 and step S5 is completed, the process returns to the process of FIG. 5 and the addition process of the cell image and the bright spot area image is performed (step S6), and the distribution of the bright spot area on the cell is This is displayed on the display unit 23, and then the biological material expression pattern is classified. In the present embodiment, the expression level is classified as a biological material expression pattern (step S7).

FIG. 11 shows a detailed flow of the process in step S7.
In step S701, the number of bright spots per cell is calculated based on the cell image and bright spot area image added in step S6.
Next, threshold processing is performed on the obtained number of bright spots using a predetermined threshold (step S702), and cell classification processing based on the expression level is performed. Each cell is classified into a plurality of stages according to the expression level. The expression level can be determined based on the number of fluorescent bright spots of PID. That is, since the fluorescent luminescent spots of PID have high luminance and can be detected individually, it is possible to specify the expression level of the biological substance by the number of luminescent spots.
The threshold may be set for each type of cell specified in step S3, and the biological material expression level may be classified for each cell type.

The classification of biological substance expression levels in step 7 is basically performed automatically in cooperation with the program stored in the control unit 21 and the storage unit 25, but such processing is performed by an observer. Auxiliary work may be involved. The auxiliary work by the observer includes, for example, work for adjusting each threshold value of the expression level of the biological material in stages with respect to the program stored in the storage unit 25.

Next, returning to the process of FIG. 5, the biological material expression pattern is drawn (step S8). Here, the expression level obtained in step S7 is drawn.
FIG. 12 shows a detailed flow of the process in step S8, and FIG. 13 shows an example of the expression level drawing displayed on the display unit.
As shown in FIG. 13, an information box 231 and a drawing box 232 are displayed on the drawing screen. Further, the information box 231 displays information such as the cell type identified in step S3 and the corresponding biological material expression level threshold, and the drawing box 232 displays the cell image and bright spot area in step S6. An image to which the images are added (hereinafter referred to as a cell distribution image 234) and expression pattern information 233 in which the expression level is expressed as a numerical value are displayed.

First, the threshold value of the expression level for each cell type is displayed in the information box 231 as the biological material expression pattern classification information set in step S7 (step S801). Here, classification is performed in three stages of high, medium and low according to the expression level.

Next, the display method of the expression pattern information 233 of the biological material in the drawing box 232 is determined (step S802). Here, the method for displaying the numerical value of the expression level is different for each class of cells classified based on the expression level of the biological material.
As will be described later, as shown in FIG. 13, a numerical value as the expression pattern information 233 is displayed on the cell distribution image 234. For example, the numerical value has a different size, thickness, etc. It is possible to easily grasp the level of expression level by visual recognition. In addition, the expression pattern information 233 may be displayed in a color corresponding to the threshold set in step S801, and the level of expression is expressed not only by the expression pattern information 233 but also by the thickness and color of the frame surrounding the cell. May be. Here, numerical colors are classified into four colors of red, yellow, green, and blue, for example, from those having a high expression level.
When the display method of the expression pattern information 233 is determined, the cell distribution image 234 and the expression pattern information 233 are added in step S803.

Next, the display position of the expression pattern information 233 is determined.
First, in step S804, the coordinates (XY coordinates) of each cell are obtained. Here, as the cell coordinates, the center coordinates and the vertex coordinates of a rectangle surrounding the cell so as to be in contact with the outer edge of the cell are obtained. As shown in FIG. 14A, when there are two cells Ca and Cb, center coordinates Sa and Sb and vertex coordinates Va1 to Va4 and Vb1 to Vb4 of rectangles surrounding the cells are calculated, respectively.
Next, in step S805, the center coordinates of the expression pattern information 233 are set to be the center coordinates of each cell. In step S806, the vertex coordinates of the expression pattern information 233 are calculated. Here, the vertex coordinates of the expression pattern information 233 are the coordinates of the rectangular vertices surrounding the numerical value, and are calculated based on the center coordinates and the size of the expression pattern information 233 set in step S802.

Next, it is determined whether or not duplication occurs when the expression pattern information 233 is drawn in the drawing box 232 (step S807). As shown in FIG. 14B, when the expression pattern information 233a and the expression pattern information 233b are displayed for the cells C1 and C2, duplication may occur. At this time, since it becomes difficult to discriminate because the numerical values overlap, it is necessary to correct the display position so as not to overlap.
When it is determined that there is an overlap in the expression pattern information 233 (step S807: Yes), the display position of the expression pattern information 233 is corrected (step S808). As a correction method, as shown in FIG. 14C, the coordinates of the vertices included in the region where the expression pattern information 233 overlaps each other up to the position in contact with each other's vertex or side, in the X axis direction and / or the Y axis direction. Move to. Alternatively, as shown in FIG. 14D, the vertex coordinates of the expression pattern information 233 are moved so as to coincide with the vertices (in this case, Va1 and Vb4) that are sufficiently distant from other cells among the square vertices. Also good.

As described above, when the correction of the display position of the expression pattern information 233 is completed, or when it is determined in the step S807 that the expression pattern information 233 is not overlapped with each other (step S807: No), the expression amount drawing process is terminated.

Drawing of the expression pattern in step 8 is basically automatically performed in cooperation with the program stored in the control unit 21 and the storage unit 25. For such processing, auxiliary work by the observer is performed. It may be accompanied. The auxiliary work by the observer is, for example, arbitrarily setting the display method of the expression pattern information 233 in step S802 for the program stored in the storage unit 25, and the display position of the expression pattern information 233 in step S808. The correction method is set and visual confirmation of the corrected expression pattern information 233 is included.

As described above, in the image processing apparatus according to the present invention, the expression pattern of the biological material is quantified using the fluorescent nanoparticles, and the cells are classified according to the expression pattern. In other words, the use of fluorescent nanoparticles makes it possible to quantitatively analyze biological materials, which was impossible with the conventional staining method, so that the accuracy of diagnosis can be improved.

Further, in the image processing apparatus according to the present invention, a display method of expression pattern information is set according to cell classification and displayed together with a cell distribution image. This eliminates the need for the observer to observe the cell distribution and the amount of expression of the biological material while sequentially associating them, and makes it possible to grasp the expression pattern at a glance.

Also, if the expression pattern information overlaps with each other, rearrange the expression pattern information so that the expression pattern information does not overlap. Furthermore, the expression pattern information is displayed in a color different from the color of the cell distribution image. Thereby, the discriminability of each expression pattern information can be maintained.

Also, it is possible to specify the cell type based on the cell feature and perform image processing for each cell type. That is, visibility can be further improved by setting a display method for each cell type.

[Second Embodiment]
Hereinafter, a second embodiment for carrying out the present invention will be described. The <configuration of the pathological diagnosis support system 100>, <fluorescent staining reagent and staining method>, and <operation of the pathological diagnosis support system 100> in the present embodiment are the same as those in the first embodiment, and thus detailed description thereof is omitted. To do.

In the first embodiment, a tissue section stained with an HE staining reagent is used as the bright field image. However, in the second embodiment, a protein specifically expressed in macrophages (hereinafter referred to as a bright field image). It is different in that a tissue section stained with a dye described later is used with a macrophage protein) as a target. There are various proteins that are specifically expressed in cells present in the microenvironment surrounding cancer cells, and staining with a dye is possible as appropriate. The cells forming the microenvironment include stromal cells (fibroblasts, endothelial cells, leukocytes (lymphocytes (B cells, T cell NK cells, T-reg etc.)), monocytes, neutrophils, eosinophils. Spheres, basophils, etc.)) dead cells, glandular cells, fat cells, epithelial cells, etc., and macrophages are classified as monocytes.
Furthermore, dyeing using two or more different dyes is also possible, and dyeing combining dye dyeing and H dyeing E dyeing can also be used.

Specifically, it includes a step (A) of staining macrophage protein, a step (B) of staining the target protein, and a step (C) of quantitatively evaluating a signal derived from the target protein. Steps (A) and (B) are steps performed on the same specimen. In the information acquisition method of the present invention, the order of the steps (A) and (B) is not particularly limited, but it is usually preferable to carry out the step (A) → step (B) in that order, and then carry out the step (C).

In the present embodiment, in addition to the steps (A) to (C), the step (D) is preferably further included, and the steps (D) and (E) are more preferably included. Here, the step (D) is a step of specifying the position and number of macrophages by staining in the step (A). The step (E) specifies information on the expression state of the target protein based on the signal derived from the target protein measured in the step (C) and the position and number of macrophages specified in the step (D). It is a process to do.

The information that can be acquired in this embodiment preferably includes information based on the signal derived from the target protein measured in the step (C), and the position and number of macrophages identified in the step (D), and Those based on information on the expression state of the target protein identified in the step (E) are more preferred.

Such information is not particularly limited. For example, the expression amount of the target protein per unit area of the specimen, for example, the number of macrophages per unit area of the specimen, the ratio of TAM to the total number of macrophages contained in the specimen, Among the target proteins per unit area, the amount expressed in tumor cells and the amount expressed in macrophages (TAM), and their ratio, the morphology of tissues and cells contained in the specimen, etc. It is possible by staining with more than one type. In particular, it preferably includes at least one of the position and expression level of the target protein in macrophages, and more preferably includes the position and expression level.

The staining performed in the step (A) is preferably dye staining, the staining performed in the step (B) is preferably fluorescent staining, and the “signal derived from the target protein” is applied to the fluorescently stained target protein. It is preferably based on the number of derived bright spots.

In the staining performed in the steps (A) and (B), a labeling substance is directly or indirectly bound to a macrophage protein and a target protein to be stained by bringing a specimen and a labeling substance, which will be described later, into contact with each other. For example, immunostaining is preferably performed by reacting a labeled antibody obtained by binding a labeling substance to an antibody that directly or indirectly binds to a macrophage protein or a target protein with a specimen.

<Macrophage protein and target protein>
Hereinafter, the macrophage protein and target protein used for the analysis of the second embodiment will be described.

(1) Macrophage protein The macrophage protein stained in the step (A) in the information acquisition method of the present invention can be arbitrarily selected from proteins specifically expressed in macrophages, for example, CD163, CD204, CD68. , Iba1, CD11c, CD206, CD80, CD86, CD163, CD181, CD197, iNOS, Arginase1, CD38, Egr2, etc., in particular, CD68, CD163, and CD204.
It is preferable to select from.

The macrophage protein is preferably a protein that is specifically expressed in M2 macrophages, and is also preferably a protein that is expressed in tumor-associated macrophages (TAM).

CD163 and CD204 are preferable as proteins specifically expressed in M2 macrophages.

(2) Dye staining In the step (A), it is preferable that dye staining is performed on the macrophage protein. The dye staining is not particularly limited as long as it is a technique for staining macrophage protein with a dye capable of bright-field observation. For example, a labeling substance (enzyme) is bound to macrophage protein to be stained by an arbitrary method, and the enzyme substrate A method of staining a target substance by depositing a dye on a specimen by adding a dye (substrate) that develops color by reaction is widely used. For example, immunostaining can be performed by adding a dye that is a substrate of the enzyme to a sample that has been reacted in advance with a labeled antibody in which the enzyme is bound to an antibody that binds directly or indirectly to the target protein. Preferably there is. Examples of the enzyme include peroxidase and alkaline phosphatase, and examples of the dye include 3,3′-diaminobenzidine (DAB), Histogreen, TMB, Betazoid DAB, Cardassian DAB, Bajoran Purple, VinaGreen, Romulin AEC, Can be mentioned.

(3) Target protein The target protein stained in the step (B) in the information acquisition method of the present invention is at least one kind of protein contained in the specimen, and is not particularly limited. Examples of the target protein include CSF- Used as a biomarker in pathological diagnosis of colony-stimulating factor receptors such as 1R, PD-L1 (Programmed cell death1 ligand 1), B7-1 / 2, CD8, CD30, CD48, CD59 And a protein involved in immune cell metabolism such as IDO (Indoleamine-2,3-dioxygenase-1).

The target protein is preferably a protein (antigen) expressed in macrophages, more preferably a protein specifically expressed in macrophages, and particularly preferably a protein specifically expressed in M2 macrophages. When using a tumor tissue as a specimen, the target protein is preferably a protein expressed in TAM, and more preferably a protein specifically expressed in TAM. Specific target proteins are preferably CSF-1R, IDO, PDL1, B7-1 / 2, CD8, CD30, CD48, and CD59, and more preferably CSF-1R, IDO, or PDL1.

Hereinafter, an example in which CD68 is cited as the macrophage protein and CSF-1R is cited as the target protein will be described for the staining and the bright spot count results according to the second embodiment. As shown in the table, it was clarified from the results of the bright spot count that cells having different bright spot numbers exist, and it was possible to obtain an indication that different cell types or different cell environments exist. Further, if two-color bright-field images are obtained by simultaneously staining CD204 together with CD68 as a macrophage protein, macrophages can be identified as M2 macrophages, so that it is considered possible to classify cells in more detail.

[Example]
(1) Pre-staining treatment (1-1) Deparaffinization treatment Deparaffinization treatment was performed on lung adenocarcinoma tissue array slides (HLug-Ade150Sur-02: US Biomax) according to the following procedure. The tissue array slide was left in a 65 ° C. incubator for 15 minutes to melt the paraffin in the slide. Each was immersed in three containers containing xylene for 5 minutes, washed with dehydrated ethanol (Kanto Chemical; 14599-95), and further immersed in dehydrated ethanol for 5 minutes x 2 times. Thereafter, it was further dehydrated with 99.5% ethanol (Kanto Chemical; 14033-70) and washed by flowing in pure water for 10 minutes.

(1-2) Activation treatment The deparaffinized tissue array slide is immersed in an activation solution (10 mM Tris buffer (pH 9.0)) preliminarily heated to 95 ° C. and left for 45 minutes. After leaving it to reach room temperature, it is washed by exposing it to pure water flowing for 10 minutes, and the section slide is immersed in a staining vat containing PBS and washed 5 times × 3 times.

(1-3) Endogenous Peroxidase Block Endogenous peroxidase block was performed by immersing the activated tissue array slide in 3% hydrogen peroxide for 15 minutes.

(1-4) Blocking The tissue array slide subjected to the above treatment was immersed in PBS containing 1% BSA, and subjected to blocking treatment.
(2) CD68 staining step (2-1) Primary antibody reaction Anti-CD68 mouse monoclonal antibody [PG-M1] (Dako) was diluted 100 times with PBS containing 1% BSA, and the blocking treatment was performed. It was added to the performed tissue array slide and allowed to react for 1 hour at room temperature.

(2-2) Secondary antibody reaction The tissue array slide after the primary antibody reaction was washed with PBS, and then Histofine Simple Stain MAX-PO (MULTI) (Nichirei Biosciences; 049-22831) was added at room temperature. For 30 minutes.

(2-3) HistoGreen staining (2-2) HistoGreen (AbCys; E109) is added to the tissue array slide after the secondary antibody reaction and allowed to react at room temperature for 3 minutes. After the reaction, it was washed with PBS for 5 minutes × 3 times, and further washed with pure water.
(3) MCSFR (CSF-1R) staining step (3-1) Blocking The tissue array slide after washing in (2-2) was blocked in the same manner as in (1-4).

(3-2) Primary antibody reaction Anti-MCSFR rabbit monoclonal antibody [SP211] (abcam) was diluted 50-fold with PBS containing 1% BSA, added to the above-blocked tissue array slide, 4 The reaction was allowed to proceed overnight at ° C.

(3-3) Secondary Antibody Reaction After washing the tissue array slide after the primary antibody reaction with PBS, the biotinylated secondary antibody prepared in Preparation Example 1 was diluted to 2 μg / mL with PBS containing 1% BSA. Was added and allowed to react for 30 minutes.

(3-4) Fluorescent labeling After washing the tissue array slide after the secondary antibody reaction with PBS, the dispersion of the streptavidinated phosphor-aggregated particles prepared in Preparation Example 2 was added and reacted at room temperature for 2 hours. . After 2 hours, the plate was washed 3 times with PBS for 5 minutes, and 4% PFA was added to the section slide to react for 10 minutes.
(4) Fixing treatment The tissue array slide after the PFA reaction was exposed to flowing pure water for 1 minute.
(5) Sealing treatment (5-1) Dehydration / penetration process The section slides were immersed in the order of “99.5% EtOH tank”, “dehydrated EtOH tank” × 3, and “xylene tank” × 3.

(5-2) Encapsulation Step After the dehydration and penetration, the section slide was encapsulated with an automatic encapsulation machine. The section slide was then stored in the dark.

(6) Photographing process First, a dye-stained image (400 times) was photographed using a microscope digital camera “DP73” (Olympus Corporation) attached to a fluorescence microscope “BX-53” (Olympus Corporation). .

Next, a fluorescent image was taken using a fluorescent microscope “BX-53” (Olympus Corporation). The specimen was irradiated with excitation light corresponding to the biotinylated phosphor-aggregated particles used in the fluorescent labeling (3-4) to emit fluorescence, and a stained image in that state was photographed. At this time, the wavelength of the excitation light was set to 575 to 600 nm using the excitation light optical filter provided in the fluorescence microscope, and the wavelength of the fluorescence to be observed was set to 612 to 692 nm using the fluorescence optical filter. The intensity of the excitation light at the time of observation and image photographing with a fluorescence microscope was such that the irradiation energy near the center of the visual field was 900 W / cm 2. The exposure time at the time of image shooting was adjusted within a range in which the luminance of the image was not saturated, and was set to, for example, 4000 μsec.

The dye-stained image and the fluorescence image were superimposed and image processing was performed. Of 150 samples included in the tissue array slide, cells stained with HistGreen, that is, cells stained with CD68 were used as macrophages, and samples containing macrophages were extracted. For samples containing macrophages, bright spots derived from CSF-1R per macrophage cell were further measured. Note that the number of bright spots representing the phosphor-aggregated particles having a luminance of a predetermined value or more was measured.

Table 1 shows the number of bright spots per macrophage and macrophage cells contained in the image of the specimen containing macrophages. It can be seen that the number of bright spots contained (the expression level of CSF-1R) differs for each TAM.

Thereafter, in the pathological diagnosis support system 100, the operation of acquiring and analyzing the above-described fluorescence image and bright field image is the same as the <operation of the pathological diagnosis support system 100> in the first embodiment, and thus detailed description thereof will be made. Description is omitted.

In the image processing apparatus according to the second embodiment, in the same field of view, a dyed staining image performed on the macrophage protein can be obtained by shooting with illumination light applied to the specimen.
In addition, when performing the step (D) of specifying the position and number of macrophages from the dyed stained image, the step (D) is a step of specifying the position and number of M2 macrophages from the dyed stained image. Is preferable, and a step of specifying the position and number of TAMs is more preferable.

In addition, the step (E) of identifying information on the expression state of the target protein based on the signal derived from the target protein and the position and number of the macrophages (M2 macrophages, TAM) identified in the step (D). By performing, information on the position or expression level of the target protein in macrophages can be obtained.

Thus, by accurately determining one of macrophages, preferably M2 macrophages and TAM, more preferably M2 macrophages and TAM, and extracting and measuring a signal of the target protein contained in the macrophages, expression of the target protein The status information can be specified in more detail. Specifically, for example, the average expression level and density of the target protein per macrophage (eg, TAM), the localization of the target protein in the macrophage, and the expression level in the macrophage relative to the total expression level of the target protein per unit area of the specimen. The ratio of the expression level of the target protein.

[Other Embodiments]
As mentioned above, although concretely demonstrated based on embodiment based on this invention, said embodiment is a suitable example of this invention, and is not limited to this.

For example, in the above-described embodiment, the case where the expression level of one kind of biological substance is quantified as an example of the specific protein has been described. However, the present invention is not limited to this. It is also possible to determine the amount of biological material. For example, in breast cancer tissue, breast cancer subtypes can be classified by expression analysis of hormone receptors (estrogen receptor (ER) and progesterone receptor (PgR)), HER2, and Ki67.

In the above embodiment, the expression level is classified as the biological material expression pattern, but the present invention is not limited to this. For example, the intracellular distribution and density of the biological material, Classification can also be performed by a histogram or a curve represented by the expression level and the number of cells corresponding to the expression level.
Specifically, the following classification method can be mentioned.
For example, if HER2 is specifically expressed in the cell membrane, there is a high possibility that it is a cancer cell. Therefore, a threshold value is set for the expression level in the cell membrane, and positive cells or negative cells are classified.
Classification can also be performed according to the type of cell or region to which the biological material belongs. For example, in order to make it easier to observe the level of expression of biological material in T cells attacking cancer cells, there is a method of displaying the biological material expressed in T cells separately from the biological material expressed in B cells. is there.
Alternatively, classification can also be performed according to the distance from a cell or a specific region. For example, by changing the display according to the distance from the edge of the tumor region (portion where cancer cells are gathered), it is possible to easily see how much biological material has infiltrated the tumor region.

In the above embodiment, the cell shape is used as the cell feature amount. However, the present invention is not limited to this, and the shape of the cell nucleus may be extracted as the cell feature amount. Thereby, for example, positive cells or negative cells can be classified by detecting atypia such as enlargement of cell nuclei in cancer cells.

In the above description, an example in which an HDD or a semiconductor non-volatile memory is used as a computer-readable medium of the program according to the present invention is disclosed, but the present invention is not limited to this example. As another computer-readable medium, a portable recording medium such as a CD-ROM can be applied. Further, a carrier wave (carrier wave) is also applied as a medium for providing program data according to the present invention via a communication line.

In addition, the detailed configuration and detailed operation of each device constituting the pathological diagnosis support system 100 can be changed as appropriate without departing from the spirit of the invention.

The present invention can be used for an image processing apparatus and a program.

2A Image processing device 3A Cable 21 control unit (input means, first extraction means, second extraction means, generation means, classification means, display control means, identification means)
22 Operation unit 23 Display unit 231 Information box 232 Drawing box 233 Expression pattern information 234 Cell distribution image 24 Communication I / F
25 storage unit 26 bus 100 pathological diagnosis support system

Claims (7)

1. An input means for inputting a morphological image representing the morphology of a cell and a fluorescent image representing the expression of the biological material in the same range as the morphological image with a fluorescent luminescent spot in a tissue specimen stained with a single or plural types of biological materials; ,
   First extraction means for extracting a cell region from the morphological image;
   Second extraction means for extracting a fluorescent bright spot region from the fluorescent image;
   Generating means for calculating an expression level of the biological material extracted from the number of the fluorescent bright spot regions extracted by the second extraction means, and generating expression pattern information including the expression level;
   An image processing apparatus comprising: a classifying unit that classifies cells according to the expression pattern information generated by the generating unit.
2. Display control means for displaying the expression pattern information generated by the generation means on a display means;
   The image processing apparatus according to claim 1, wherein the display control unit displays the expression pattern information and the morphological image so as to overlap each other.
3. 3. The image processing apparatus according to claim 2, wherein the display control means changes and displays the expression pattern information display method for each cell class classified by the classification means.
4. The image processing apparatus according to claim 2 or 3, wherein the display control means displays the expression pattern information so as not to overlap each other.
5. The image processing apparatus according to any one of claims 2 to 4, wherein the display control means displays the expression pattern information in a color different from a color of the morphological image.
6. 6. The image processing apparatus according to claim 1, further comprising a specifying unit that specifies a cell type based on a feature amount of the cell region extracted by the first extracting unit.
7. The computer of the image processing device
   Input means for inputting a morphological image representing cell morphology and a fluorescent image representing the expression of the biological material in the same range as the morphological image with fluorescent luminescent spots in a tissue specimen stained with a single or plural types of biological materials,
   First extraction means for extracting a cell region from the morphological image;
   Second extraction means for extracting a fluorescent bright spot region from the fluorescent image;
   Generating means for calculating an expression level of the biological material extracted from the number of the fluorescent bright spot regions extracted by the second extraction means, and generating expression pattern information including the expression level;
   Classification means for classifying cells according to the expression pattern information generated by the generation means,
   Program to function as.

Images