JP6597316B2 - Image processing apparatus and program - Google Patents

Description

  The present invention relates to an image processing apparatus and a program.

  In recent years, with the spread of molecular targeted drug treatment centering on antibody drugs, in order to design molecular targeted drugs more effectively, quantification of biological substances on cells to be observed has been demanded. As a method for confirming the presence of a biological material, a tissue analysis method based on the binding of a fluorescent material to which a biological material recognition site is bound and a biological material corresponding to the biological material recognition site is known.

  In such a tissue analysis method, for example, the quantification and distribution analysis of a fluorescent substance representing the expression of a biological substance to be quantified on a cell is performed. An error occurs in the result. For this reason, it is required to correctly extract fluorescent luminescent spots representing the expression of biological substances to be quantified as noise.

Furthermore, when diagnosing the degree and type of a lesion using a pathological tissue specimen, it is required to analyze the expression status of a plurality of types of biological substances. In order to analyze the expression of multiple types of biological materials from a single tissue specimen, multiple types of biological materials are stained (multi-stained) using fluorescent materials with different emission characteristics, and the number and position of fluorescent luminescent spots. Is preferably analyzed for each emission characteristic.
When an analysis based on multiple staining is performed, it is necessary to accurately distinguish a plurality of types of biological substances. However, due to factors such as crosstalk between fluorescent materials with different emission characteristics, it is difficult to accurately determine the fluorescent bright spots that represent the expression of multiple types of biological materials, and errors in analysis results are likely to occur. There is.

  For example, in the tissue analysis method described in Patent Document 1, a cell nucleus region is extracted from a microscopic image of a tissue section fluorescently stained with a specific biological material, and a circular region having a predetermined radius centered on the center of gravity of the cell nucleus region is defined as a cell region. And the expression of the biological material in the cell region is analyzed based on the position and number of fluorescent bright spots.

  However, since the cell region used in Patent Document 1 is a circular region having a certain size, an error from the actual cell region is large. For this reason, for example, a fluorescent luminescent spot derived from noise at a position off the cell cannot be distinguished from a fluorescent luminescent spot representing the expression of a biological substance, and an error occurs in the analysis result.

  On the other hand, in the biological material evaluation method described in Patent Document 2, the tissue specimen is subjected to fluorescent staining of the biological material to be quantified and fluorescent staining of the cell membrane, and the position of the cell membrane in the image is identified. The expression level of the biological substance is evaluated based on the number of fluorescent bright spots on the cell membrane. Therefore, it is possible to distinguish a fluorescent bright spot derived from noise at a position off the cell from a fluorescent bright spot representing the expression of a biological substance.

However, the evaluation method described in Patent Literature 2 cannot solve the problem that an error is likely to occur in the determination and analysis results of fluorescent luminescent spots representing the expression of a plurality of types of biological substances when performing analysis based on multiple staining. .
In addition, when staining is performed using a phosphor having a high emission luminance as specifically used in Patent Document 2, extraction of fluorescent bright spots from an image and discrimination of cell autofluorescence and fluorescent bright spots are performed. Becomes easy. However, at the same time, it is considered that the influence of crosstalk between a plurality of fluorescent materials having different emission characteristics increases, making it difficult to distinguish only by filter settings or the like. Specifically, when extracting a fluorescent luminescent spot based on the emission of one type of fluorescent substance, a fluorescent luminescent spot indicating the emission of another type of biological substance is easily extracted, and an error is likely to occur in the quantitative result. .

  On the other hand, according to the method described in Patent Document 3, a specimen that has been multiple-stained using a plurality of labeling dyes is imaged with an imaging device having a plurality of color filters to obtain luminance data, and Luminance data can be separated for each labeling dye using spectral data created based on the wavelength characteristics of the color filter.

International Publication No. 2013/146843 JP 2013-57631 A JP 2007-10340 A

  However, in the method described in Patent Document 3, it is necessary to prepare spectral data based on the wavelength characteristics of the marker dye and the color filter in advance, and it is necessary to analyze the luminance data for each pixel and each color filter. Therefore, it takes time and effort.

A main object of the present invention is to provide an image processing apparatus and a program capable of easily discriminating fluorescent bright spots representing the expression of a plurality of biological substances.
The above-mentioned problem according to the present invention is solved by the following means.

1. A fluorescence image showing the expression of the plurality of types of biological materials as fluorescent luminescent spots, which is obtained by imaging tissue samples obtained by staining a plurality of types of biological materials using a plurality of types of fluorescent materials having different luminescence characteristics, and the tissue sample An input means for inputting a captured morphological image representing the morphology of a cell or an intracellular organ;
A region extracting means for extracting a region of the cell or organ from the morphological image;
A bright spot extracting means for extracting the fluorescent bright spot from the fluorescent image;
Based on the region and the fluorescent luminescent spot, determination means for determining the type of biological material represented by the fluorescent luminescent spot;
With
The determination means determines the type of biological material represented by the fluorescent luminescent spot based on at least one of the following (a), (b), and (c):
An image processing apparatus.
(A) The distance of the cell or the intracellular organ closest to the fluorescent luminescent spot.
(B) Whether the fluorescent luminescent spot is encapsulated in the cell or an intracellular organ.
(C) The characteristic amount of the cell or organ that is closest to the fluorescent luminescent spot.
2 . A fluorescence image showing the presence of the plurality of types of specific substances as fluorescent luminescent spots, and images of tissue samples obtained by staining a plurality of types of specific substances using a plurality of types of fluorescent substances having different emission characteristics, and the tissue samples An input means for inputting a captured morphological image representing the morphology of a cell or an intracellular organ;
A region extracting means for extracting a region of the cell or organ from the morphological image;
A bright spot extracting means for extracting the fluorescent bright spot from the fluorescent image;
Determination means for determining the type of the specific substance represented by the fluorescent luminescent spot based on the region and the fluorescent luminescent spot;
An image processing apparatus comprising:

3 . The image processing apparatus according to claim 2 , wherein the region is any one of a region of a cell membrane, a cytoplasm, a cell nucleus, or a nucleolus.

4 . The determination means determines the type of the specific substance represented by the fluorescent bright spot based on at least one of the following (a), (b), and (c) : Or the image processing apparatus of Claim 3 .
(A) The distance of the cell or the intracellular organ closest to the fluorescent luminescent spot.
(B) Whether the fluorescent luminescent spot is encapsulated in the cell or an intracellular organ.
(C) The characteristic amount of the cell or organ that is closest to the fluorescent luminescent spot.

5 . The image processing apparatus according to claim 1 , wherein the feature amount is an area.

6 . Said determining means, on the basis of the color of the fluorescent bright spots, characterized in that said determining the type of the specific substance fluorescent bright spots representing the image according to any one of the second term to fifth term Processing equipment.

7 . The said determination means determines that the kind of specific substance which the said fluorescence luminescent point represents is any one of the said multiple types of specific substance . Any one of the 2nd term | claim-6th characterized by the above-mentioned . An image processing apparatus according to 1.

8 . The determination means includes
Calculating a probability that the fluorescent luminescent spot represents each of the plurality of types of specific substances ;
The image processing apparatus according to any one of claims 2 to 6, wherein the type of the specific substance represented by the fluorescent bright spot is determined.

9 . Based on the result determined by the determination unit, paragraph 7, characterized in that said from fluorescent bright spot, a selection means for selecting a fluorescent bright spots representing a particular specific substance among the plurality of types of specific substance An image processing apparatus according to 1.

10 . Computer
A fluorescence image showing the presence of the plurality of types of specific substances as fluorescent luminescent spots, and images of tissue samples obtained by staining a plurality of types of specific substances using a plurality of types of fluorescent substances having different emission characteristics, and the tissue samples An input means for inputting a captured morphological image representing the morphology of a cell or an intracellular organ;
An area extracting means for extracting an area of the cell or organ from the morphological image;
A bright spot extracting means for extracting the fluorescent bright spot from the fluorescent image;
Determination means for determining the type of the specific substance represented by the fluorescent luminescent spot based on the region and the fluorescent luminescent spot,
Program to function as.
11. A tissue image in which a specific substance is stained with a fluorescent substance is imaged, a fluorescence image indicating the presence of the specific substance with a fluorescent bright spot, and a morphological image representing the form of a cell or an intracellular organ is imaged in the tissue specimen. Input means for inputting;
A region extracting means for extracting a region of the cell or organ from the morphological image;
A bright spot extracting means for extracting the fluorescent bright spot from the fluorescent image;
Determination means for determining the type of the specific substance represented by the fluorescent luminescent spot based on the region and the fluorescent luminescent spot;
With
The determination means determines the type of a specific substance represented by the fluorescent bright spot based on at least one of the following (a), (b), and (c): .
(A) The distance of the cell or the intracellular organ closest to the fluorescent luminescent spot.
(B) Whether the fluorescent luminescent spot is encapsulated in the cell or an intracellular organ.
(C) The characteristic amount of the cell or organ that is closest to the fluorescent luminescent spot.

  According to the image processing apparatus and program of the present invention, it is possible to easily discriminate fluorescent luminescent spots representing the expression of a plurality of biological substances.

It is a figure which shows the system configuration | structure of a diagnostic assistance information generation system. It is a block diagram which shows the functional structure of the image processing apparatus of FIG. It is a flowchart which shows the image-analysis process of this invention. It is a flowchart which shows the flow of an area | region extraction process. It is a flowchart which shows the flow of a bright spot extraction process. It is a figure for demonstrating the determination process of 1st Embodiment. It is a figure for demonstrating the determination process of 2nd Embodiment. It is a figure for demonstrating the determination process of 3rd Embodiment. It is a figure for demonstrating the determination process of 4th Embodiment. It is a figure for demonstrating the determination process of 5th Embodiment.

  Hereinafter, although the form for implementing this invention with reference to figures is demonstrated, this invention is not limited to these.

<Configuration of Diagnosis Support Information Generation System 100>
<Configuration of Pathological Diagnosis Support System 100>
FIG. 1 shows an example of the overall configuration of the pathological diagnosis support system 100.
The pathological diagnosis support system 100 acquires a microscopic image of a tissue section of a human body stained with a predetermined staining reagent, and analyzes the acquired microscopic image, thereby expressing the expression of a specific biological material in the tissue section to be observed. This is a system that outputs feature quantities quantitatively.

As shown in FIG. 1, the pathological diagnosis support system 100 is configured by connecting a microscope image acquisition apparatus 1A and an image processing apparatus 2A so that data can be transmitted and received via an interface such as a cable 3A.
The connection method between the microscope image acquisition apparatus 1A and the image processing apparatus 2A is not particularly limited. For example, the microscope image acquisition device 1A and the image processing device 2A may be connected via a LAN (Local Area Network) or may be connected wirelessly.

The microscope image acquisition apparatus 1A is a known camera-equipped microscope, and acquires a microscope image of a tissue section on a slide placed on a slide fixing stage and transmits the microscope image to the image processing apparatus 2A.
The microscope image acquisition apparatus 1A includes an irradiation unit, an imaging unit, an imaging unit, a communication I / F, and the like. The irradiating means includes a light source, a filter, and the like, and irradiates the tissue section on the slide placed on the slide fixing stage with light. The imaging means is composed of an eyepiece lens, an objective lens, and the like, and forms an image of transmitted light, reflected light, or fluorescence emitted from the tissue section on the slide by the irradiated light. The imaging unit is a microscope-installed camera that includes a CCD (Charge Coupled Device) sensor and the like, captures an image formed on the imaging surface by the imaging unit, and generates digital image data of the microscope image. The communication I / F transmits image data of the generated microscope image to the image processing apparatus 2A.
The microscope image acquisition apparatus 1A includes a bright field unit that combines an irradiation unit and an imaging unit suitable for bright field observation, and a fluorescence unit that combines an irradiation unit and an imaging unit suitable for fluorescence observation. It is possible to switch between bright field / fluorescence by switching.
In addition, what installed the camera in arbitrary well-known microscopes (For example, a phase-contrast microscope, a differential interference microscope, an electron microscope, etc.) can be used as 1 A of microscope image acquisition apparatuses.

  Note that the microscope image acquisition apparatus 1A is not limited to a microscope with a camera. For example, a virtual microscope slide creation apparatus (for example, a special microscope scan apparatus that acquires a microscope image of an entire tissue section by scanning a slide on a microscope slide fixing stage). Table 2002-514319 gazette) etc. may be used. According to the virtual microscope slide creation device, it is possible to acquire image data that allows a display unit to view a whole tissue section on a slide at a time.

The image processing apparatus 2A calculates the expression distribution of a specific biological material in the tissue section to be observed by analyzing the microscope image transmitted from the microscope image acquisition apparatus 1A.
FIG. 2 shows a functional configuration example of the image processing apparatus 2A.
As shown in FIG. 2, the image processing apparatus 2 </ b> A includes a control unit 21, an operation unit 22, a display unit 23, a communication I / F 24, a storage unit 25, and the like, and each unit is connected via a bus 26. Yes.

The control unit 21 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), and the like, executes various processes in cooperation with various programs stored in the storage unit 25, and performs image processing 2A. Overall control of the operation.
For example, the control unit 21 executes image analysis processing in cooperation with the image processing program stored in the storage unit 25, and realizes functions as a region extraction unit, a bright spot extraction unit, a determination unit, and a selection unit. To do.

  The operation unit 22 includes a keyboard having character input keys, numeric input keys, various function keys, and the like, and a pointing device such as a mouse, and includes a key press signal pressed by the keyboard and an operation signal by the mouse. Is output to the control unit 21 as an input signal.

  The display unit 23 includes a monitor such as a CRT (Cathode Ray Tube) or an LCD (Liquid Crystal Display), for example, and displays various screens in accordance with instructions of display signals input from the control unit 21.

  The communication I / F 24 is an interface for transmitting and receiving data to and from external devices such as the microscope image acquisition apparatus 1A. The communication I / F 24 realizes a function as an input unit for fluorescent images and morphological images.

The storage unit 25 is configured by, for example, an HDD (Hard Disk Drive), a semiconductor nonvolatile memory, or the like. As described above, the storage unit 25 stores various programs and various data.
In addition, the image processing apparatus 2A may include a LAN adapter, a router, and the like, and be connected to an external device via a communication network such as a LAN.

<About images>
In the present embodiment, the image processing apparatus 2A, for example, transmits an image representing the expression of a specific biological material in a cell transmitted from the microscope image acquisition apparatus 1A (for example, the expression of a specific biological material in a cell with a fluorescent bright spot). It is preferable to perform the analysis using a fluorescent image) and a morphological image (for example, a bright-field image) representing a form of a predetermined structure of a cell such as a cell nucleus or a cell membrane.

The “bright field image” refers to, for example, a tissue section stained with a hematoxylin staining reagent (H staining reagent) or a hematoxylin-eosin staining reagent (HE staining reagent) and enlarged in a bright field in the microscope image acquisition apparatus 1A. It is a microscope image obtained by image and imaging | photography, Comprising: It is a cell form image showing the form of the cell in the said tissue section. Hematoxylin (H) is a blue-violet pigment that stains cell nuclei, bone tissue, part of cartilage tissue, serous components, etc. (basophilic tissue, etc.). Eodine (E) is a red to pink pigment that stains cytoplasm, soft tissue connective tissue, erythrocytes, fibrin, endocrine granules, etc. (eg, acidophilic tissue).
As cell morphology images, in addition to bright-field images, tissue sections are stained with a fluorescent staining reagent that can specifically stain the structure to be diagnosed of cells, and the fluorescence emitted by the fluorescent staining reagent used is photographed. Fluorescent images may be used. Examples of the fluorescent staining reagent that can be used for obtaining a morphological image include DAPI staining capable of staining cell nuclei, Papalonic Nicorou staining capable of staining cytoplasm, and the like. Moreover, you may use a phase difference image, a differential interference image, an electron microscope image, etc. as a form image.

  Fluorescent images that express the expression of a specific biological substance in cells with fluorescent luminescent spots are stained using a fluorescent substance that specifically binds to and / or reacts with a specific biological substance or fluorescent substance-containing nanoparticles. This is a microscopic image obtained by irradiating the tissue section with excitation light having a predetermined wavelength in the microscope image acquisition apparatus 1A to cause the fluorescent material to emit fluorescence, and enlarging and photographing the fluorescence. The “fluorescent substance-encapsulating nanoparticles” are nanoparticles encapsulating a fluorescent substance and will be described in detail later.

<Fluorescent staining reagents and staining methods>
Hereinafter, a fluorescent staining reagent for obtaining a fluorescent image in which the expression of a specific biological substance specifically expressed in cells is expressed by a fluorescent luminescent spot and a staining method of a tissue section using the fluorescent staining reagent will be described.

(1) Fluorescent substance Examples of fluorescent substances used in fluorescent staining reagents include fluorescent organic dyes and quantum dots (semiconductor particles). When excited by ultraviolet to near infrared light having a wavelength in the range of 200 to 700 nm, it preferably emits visible to near infrared light having a wavelength in the range of 400 to 1100 nm.

Examples of fluorescent organic dyes include fluorescein dye molecules, rhodamine dye molecules, Alexa Fluor (Invitrogen) dye molecules, BODIPY (Invitrogen) dye molecules, cascade dye molecules, coumarin dye molecules, and eosin dyes. Examples include molecules, NBD dye molecules, pyrene dye molecules, Texas Red dye molecules, and cyanine dye molecules.
Specifically, 5-carboxy-fluorescein, 6-carboxy-fluorescein, 5,6-dicarboxy-fluorescein, 6-carboxy-2 ′, 4,4 ′, 5 ′, 7,7′-hexachlorofluorescein, 6 -Carboxy-2 ', 4,7,7'-tetrachlorofluorescein, 6-carboxy-4', 5'-dichloro-2 ', 7'-dimethoxyfluorescein, naphthofluorescein, 5-carboxy-rhodamine, 6-carboxy Rhodamine, 5,6-dicarboxy-rhodamine, rhodamine 6G, tetramethylrhodamine, X-rhodamine, and Alexa Fluor 350, Alexa Fluor 405, Alexa Fluor 430, Alexa Fluor 488, Alexa Fluor 500, Alexa Fluor 514, lexa Fluor 532, Alexa Fluor 546, Alexa Fluor 555, Alexa Fluor 568, Alexa Fluor 594, Alexa Fluor 610, Alexa Fluor 633, Alexa Fluor 635, Alexa Fluor 635, Alexa Fluor 635, Alexa Fluor 633 750, BODIPY FL, BODIPY TMR, BODIPY 493/503, BODIPY 530/550, BODIPY 558/568, BODIPY 564/570, BODIPY 576/589, BODIPY 581/591, BODIPY 630/650, BOD 630/650, BOD Jae ), Methoxycoumarin, eosin, NBD, pyrene, Cy5, Cy5.5, Cy7, and the like. These fluorescent organic dyes may be used alone or as a mixture of plural kinds.

Quantum dots include II-VI group compounds, III-V group compounds, or quantum dots containing group IV elements as components ("II-VI group quantum dots", "III-V group quantum dots", " Or “Group IV quantum dots”). These quantum dots may be used alone or in combination of a plurality of types.
Specific examples include, but are not limited to, CdSe, CdS, CdTe, ZnSe, ZnS, ZnTe, InP, InN, InAs, InGaP, GaP, GaAs, Si, and Ge.

It is also possible to use a quantum dot having the above quantum dot as a core and a shell provided thereon. In the following, as a notation of quantum dots having a shell, when the core is CdSe and the shell is ZnS, it is expressed as CdSe / ZnS.
For example, CdSe / ZnS, CdS / ZnS, InP / ZnS, InGaP / ZnS, Si / SiO ₂ , Si / ZnS, Ge / GeO ₂ , Ge / ZnS, and the like can be used, but are not limited thereto.

  As the quantum dots, those subjected to surface treatment with an organic polymer or the like may be used as necessary. Examples thereof include CdSe / ZnS having a surface carboxy group (manufactured by Invitrogen), CdSe / ZnS having a surface amino group (manufactured by Invitrogen), and the like.

(2) Fluorescent substance-encapsulated nanoparticles “Fluorescent substance-encapsulated nanoparticles” are nanoparticles encapsulating a fluorescent substance as described above, and more specifically, those in which a fluorescent substance is dispersed inside the nanoparticle. The substance and the nanoparticle itself may be chemically bonded or may not be bonded.
The material constituting the nanoparticles is not particularly limited, and examples thereof include silica, polystyrene, polylactic acid, and melamine.

The fluorescent substance-containing nanoparticles can be produced by a known method.
For example, silica nanoparticles encapsulating a fluorescent organic dye can be synthesized with reference to the synthesis of FITC-encapsulated silica particles described in Langmuir 8, Vol. 2921 (1992). Various fluorescent organic dye-containing silica nanoparticles can be synthesized by using a desired fluorescent organic dye in place of FITC.
Silica nanoparticles encapsulating quantum dots can be synthesized with reference to the synthesis of CdTe-encapsulated silica nanoparticles described in New Journal of Chemistry, Vol. 33, p. 561 (2009).
Polystyrene nanoparticles encapsulating a fluorescent organic dye may be copolymerized using an organic dye having a polymerizable functional group described in US Pat. No. 4,326,008 (1982) or polystyrene described in US Pat. No. 5,326,692 (1992). It can be produced using a method of impregnating nanoparticles with a fluorescent organic dye.
The polymer nanoparticles encapsulating the quantum dots can be prepared by using the method of impregnating the quantum nanoparticles into polystyrene nanoparticles described in Nature Biotechnology, Vol. 19, page 631 (2001).

Although the average particle diameter of the fluorescent substance-containing nanoparticles is not particularly limited, those having a size of about 30 to 800 nm can be used. Further, the coefficient of variation (= (standard deviation / average value) × 100%) indicating the variation in particle diameter is not particularly limited, but it is preferable to use a coefficient of 20% or less.
The average particle diameter is obtained by taking an electron micrograph using a scanning electron microscope (SEM), measuring the cross-sectional area of a sufficient number of particles, and taking each measured value as the area of the circle. It is the value calculated as. In this embodiment, the arithmetic average of the particle diameters of 1000 particles is defined as the average particle diameter. The coefficient of variation is also a value calculated from the particle size distribution of 1000 particles.

(3) Binding of biological substance recognition site and fluorescent substance-encapsulating nanoparticles In this embodiment, fluorescent substance-encapsulating nanoparticles and biological substance recognition site are used as fluorescent staining reagents that specifically bind and / or react with a specific biological substance. An example in which a direct combination of these is used will be described. A “biological substance recognition site” is a site that specifically binds and / or reacts with a specific biological material.
The specific biological substance is not particularly limited as long as a substance that specifically binds to the specific biological substance exists, but typically includes proteins (peptides) and nucleic acids (oligonucleotides, polynucleotides). It is done.
Therefore, examples of the biological substance recognition site include an antibody that recognizes the protein as an antigen, another protein that specifically binds to the protein, and a nucleic acid having a base sequence that hybridizes to the nucleic acid.
Specific biological substance recognition sites include anti-HER2 antibody that specifically binds to HER2, which is a protein present on the cell surface, anti-ER antibody that specifically binds to estrogen receptor (ER) present in the cell nucleus, cells An anti-actin antibody that specifically binds to actin forming the skeleton is exemplified.
Among these, anti-HER2 antibody and anti-ER antibody combined with fluorescent substance-encapsulating nanoparticles (fluorescent staining reagent) can be used for breast cancer medication selection and are preferable.

  The mode of binding between the biological substance recognition site and the fluorescent substance-encapsulating nanoparticles is not particularly limited, and examples thereof include covalent bonding, ionic bonding, hydrogen bonding, coordination bonding, physical adsorption, and chemical adsorption. A bond having a strong bonding force such as a covalent bond is preferable from the viewpoint of bond stability.

  There may be an organic molecule connecting these between the biological substance recognition site and the fluorescent substance-containing nanoparticle. For example, in order to suppress nonspecific adsorption with a biological substance, a polyethylene glycol chain can be used, and SM (PEG) 12 manufactured by Thermo Scientific can be used.

When the biological substance recognition site is bound to the fluorescent substance-encapsulating silica nanoparticles, the same procedure can be applied regardless of whether the fluorescent substance is a fluorescent organic dye or a quantum dot.
For example, a silane coupling agent that is a compound widely used for bonding an inorganic substance and an organic substance can be used. This silane coupling agent is a compound having an alkoxysilyl group that gives a silanol group by hydrolysis at one end of the molecule and a functional group such as a carboxyl group, an amino group, an epoxy group, an aldehyde group at the other end, Bonding with an inorganic substance through an oxygen atom of the silanol group.
Specifically, mercaptopropyltriethoxysilane, glycidoxypropyltriethoxysilane, aminopropyltriethoxysilane, a silane coupling agent having a polyethylene glycol chain (for example, PEG-silane no. SIM6492.7 manufactured by Gelest), etc. Is mentioned.
When using a silane coupling agent, you may use 2 or more types together.

A known procedure can be used for the reaction procedure of the fluorescent organic dye-encapsulated silica nanoparticles and the silane coupling agent.
For example, the obtained fluorescent organic dye-encapsulated silica nanoparticles are dispersed in pure water, aminopropyltriethoxysilane is added, and the mixture is reacted at room temperature for 12 hours. After completion of the reaction, fluorescent organic dye-encapsulated silica nanoparticles whose surface is modified with an aminopropyl group can be obtained by centrifugation or filtration. Subsequently, by reacting an amino group with a carboxyl group in the antibody, the antibody can be bound to the fluorescent organic dye-encapsulated silica nanoparticles via an amide bond. If necessary, a condensing agent such as EDC (1-Ethyl-3- [3-Dimethylaminopropyl] carbohydrate Hydrochloride: manufactured by Pierce (registered trademark)) can also be used.

  If necessary, a linker compound having a site capable of directly binding to a fluorescent organic dye-encapsulated silica nanoparticle modified with an organic molecule and a site capable of binding to a molecular target substance can be used. As a specific example, sulfo-SMCC having both a site that selectively reacts with an amino group and a site that reacts selectively with a mercapto group (Sulfosuccinimidyl 4 [N-maleimidemethyl] -cyclohexane-1-carboxylate: manufactured by Pierce) When used, by binding the amino group of the fluorescent organic dye-encapsulated silica nanoparticles modified with aminopropyltriethoxysilane and the mercapto group in the antibody, antibody-bound fluorescent organic dye-encapsulated silica nanoparticles can be produced.

  When the biological substance recognition site is bound to the fluorescent substance-encapsulated polystyrene nanoparticles, the same procedure can be applied regardless of whether the fluorescent substance is a fluorescent organic dye or a quantum dot. That is, by impregnating polystyrene nanoparticles having functional groups such as amino groups with fluorescent organic dyes and quantum dots, it is possible to obtain fluorescent substance-containing polystyrene nanoparticles having functional groups, and thereafter using EDC or sulfo-SMCC. Thus, antibody-bound fluorescent substance-encapsulated polystyrene nanoparticles can be produced.

  Examples of biological substance recognition sites include M. actin, MS actin, SM actin, ACTH, Alk-1, α1-antichymotrypsin, α1-antitrypsin, AFP, bcl-2, bcl-6, β-catenin, BCA 225 , CA19-9, CA125, calcitonin, calretinin, CD1a, CD3, CD4, CD5, CD8, CD10, CD15, CD20, CD21, CD23, CD30, CD31, CD34, CD43, CD45, CD45R, CD56, CD57, CD61, CD68 , CD79a, "CD99, MIC2", CD138, chromogranin, c-KIT, c-MET, collagen type IV, Cox-2, cyclin D1, keratin, cytokeratin (high molecular weight), pankeratin, pankeratin, cytokeratin 5 / 6, cytokeratin 7, cytokeratin 8, cytokeratin 8/18, cytokeratin 14, cytokeratin 19, cytokeratin 20, CMV, E-cadherin, EGFR, ER, EMA, EBV, factor VIII related antigen, fascin , FSH, galectin-3, gastrin, GFA P, glucagon, glycophorin A, granzyme B, hCG, hGH, Helicobacter pylori, HBc antigen, HBs antigen, hepatocyte specific antigen, HER2, HSV-I, HSV-II, HHV-8, IgA, IgG, IgM, IGF- 1R, Inhibin, Insulin, Kappa L chain, Ki67, Lambda L chain, LH, Lysozyme, Macrophage, Melan A, MLH-1, MSH-2, Myeloperoxidase, Myogenin, Myoglobin, Myosin, Neurofilament, NSE, p27 ( Kip1), p53, P63, PAX 5, PLAP, Pneumocystis carini, podoplanin (D2-40), PGR, prolactin, PSA, prostate acid phosphatase, Renal Cell Carcinoma, S100, somatostatin, spectrin, synaptophysin, TAG-72 And antibodies that recognize specific antigens such as TdT, thyroglobulin, TSH, TTF-1, TRAcP, tryptase, villin, vimentin, WT1, and Zap-70.

  In addition, the fluorescent substance or fluorescent substance-encapsulating nanoparticles are used by directly binding to a biological substance recognition site in advance as described above, or indirectly in the staining process as in a known indirect method in immunostaining. May be combined. Specifically, for example, a staining reagent in which a biotinylated primary antibody having a specific biological substance as an antigen is reacted with a tissue specimen, and then a fluorescent substance modified with streptavidin or a fluorescent substance-encapsulating nanoparticle is bound. May be further reacted and stained using the fact that streptavidin and biotin specifically bind to form a complex. In addition, for example, after reacting a tissue sample with a primary antibody having a specific protein as an antigen and further reacting with a biotinylated secondary antibody having the primary antibody as an antigen, a fluorescent substance modified with streptavidin or You may make it dye | stain by making the fluorescent substance inclusion nanoparticle react.

(4) Staining method The method for preparing tissue sections is not particularly limited, and those prepared by known methods can be used. The following staining method is not limited to a pathological tissue section, but can also be applied to cultured cells.

(4.1) Deparaffinization process A tissue section is immersed in a container containing xylene to remove paraffin. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, xylene may be exchanged during the immersion.
Next, the tissue section is immersed in a container containing ethanol to remove xylene. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, ethanol may be exchanged during the immersion.
Next, the tissue section is immersed in a container containing water to remove ethanol. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, water may be exchanged during the immersion.

(4.2) Activation process The activation process of the biological material of a tissue section is performed according to a known method.
The activation conditions are not particularly defined, but as the activation liquid, 0.01 M citrate buffer (pH 6.0), 1 mM EDTA solution (pH 8.0), 5% urea, 0.1 M Tris-HCl buffer, etc. are used. be able to. An autoclave, a microwave, a pressure cooker, a water bath, etc. can be used for a heating apparatus. The temperature is not particularly limited, but can be performed at room temperature. The temperature can be 50 to 130 ° C. and the time can be 5 to 30 minutes.
Next, the tissue section after the activation treatment is immersed in a container containing PBS (Phosphate Buffered Saline) and washed. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, PBS may be exchanged during immersion.

(4.3) Staining using a fluorescent staining reagent A PBS dispersion liquid of a fluorescent staining reagent is placed on a tissue section and reacted with a biological material in the tissue section.
By changing the biological material recognition site of the fluorescent staining reagent, staining corresponding to various biological materials becomes possible. When using fluorescent substance-encapsulated nanoparticles to which several types of biological substance recognition sites are bound as a fluorescent staining reagent, each fluorescent substance-encapsulated nanoparticle PBS dispersion may be mixed in advance, or sequentially separately. It may be placed on a tissue section. The temperature is not particularly limited, but can be performed at room temperature. The reaction time is preferably 30 minutes or more and 24 hours or less.
It is preferable to drop a known blocking agent such as BSA-containing PBS before staining with a fluorescent staining reagent.
Next, the stained tissue section is immersed in a container containing PBS, and unreacted fluorescent substance-containing nanoparticles are removed. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, PBS may be exchanged during immersion. A cover glass is placed on the tissue section and sealed. A commercially available encapsulant may be used as necessary.

  In addition, when performing HE dyeing | staining etc. which use a HE dyeing reagent in order to obtain a form image, it carries out before enclosure with a cover glass.

(5) Acquisition of fluorescence image A microscope image (fluorescence image) is acquired from the stained tissue section using the microscope image acquisition device 1A. In the microscope image acquisition apparatus 1A, an excitation light source and an optical filter for fluorescence detection corresponding to the absorption maximum wavelength and fluorescence wavelength of the fluorescent material used for the fluorescent staining reagent are selected.
In addition, when acquiring fluorescent images and analyzing fluorescence emission from tissue specimens that have been stained with a fluorescent dye with weak emission intensity, use a monochrome camera to reliably detect weak emission. Is known to be preferred. However, when tissue staining is performed using the above-described fluorescent substance-encapsulated nanoparticles, the emission luminance per particle is high, so even if a color camera is used, the fluorescence emitted by each particle is captured as a fluorescent bright spot. it can.
Hereinafter, in the embodiment of the present invention, a case where a fluorescent image is acquired using a color camera having a plurality of color filters having different wavelength characteristics will be described as an example. However, in the following embodiment, a monochrome image is obtained unless otherwise specified. A fluorescent image may be acquired using a camera.

<Operation of Diagnosis Support Information Generation System 100 (Including Image Processing Method)>
Hereinafter, in the diagnosis support information generation system 100, an operation of acquiring and analyzing a morphological image representing the morphology of a cell and a fluorescence image representing the expression of a specific biological substance in the cell will be described with a specific embodiment. However, the present invention is not limited to the following embodiments.
The image analysis processing of the present invention is executed in cooperation with the control unit 21 and an image processing program stored in the storage unit 25, and the control unit 21 performs processing described in the following embodiments according to the image processing program. Execute.

(First embodiment)
In the first embodiment of the present invention, the type of the biological material represented by the fluorescent luminescent spot is determined based on the distance between the region of the cell or the intracellular organ extracted from the morphological image and the fluorescent luminescent spot extracted from the fluorescent image. To do.
In the present invention, the term “intracellular organ” refers to an arbitrary structure such as an organelle contained in a cell. Specifically, the region of the cell or intracellular organ is selected from, for example, any one of a cell nucleus, a cell membrane, a cytoplasm, or a nucleolus.

Hereinafter, tissues in which cell nuclei are stained with H staining reagent, and Ki67 expressed in cell nuclei and cytokeratin 7 (hereinafter referred to as CK7) expressed in cytoplasm are respectively stained with two types of fluorescent staining reagents having different luminescent properties. An example in which a section is used as a tissue specimen will be described.
The fluorescent staining reagent in the first embodiment is, for example, a fluorescent substance-encapsulating nanoparticle (fluorescent staining reagent A) bound with an anti-Ki67 antibody and a fluorescent substance-encapsulating nanoparticle (fluorescent staining reagent B) bound with an anti-CK7 antibody.
A bright-field image is acquired as a morphological image representing the morphology of the cell nucleus.

First, the operator performs H staining of the tissue section, and then performs fluorescent staining of Ki67 and CK7.
Then, a bright field image and a fluorescence image are acquired by the following procedures (a1) to (a6) using the microscope image acquisition device 1A.
(A1) The operator places the tissue section on a slide and places the slide on the slide fixing stage of the microscope image acquisition apparatus 1A.
(A2) The unit is set as a bright field unit, the imaging magnification and the focus are adjusted, and the region to be observed on the tissue section is placed in the field of view.
(A3) Shooting is performed by an imaging unit to generate image data of a bright field image (morphological image), and the image data is transmitted to the image processing apparatus 2A.
(A4) Change the unit to a fluorescent unit.
(A5) Photographing is performed by the imaging means without changing the field of view and photographing magnification, image data of the first fluorescent image representing the light emission of the fluorescent staining reagent A is generated, and the image data is transmitted to the image processing apparatus 2A. As the excitation light and the filter, those suitable for the wavelength characteristics of the fluorescent staining reagent A are used.
(A6) First, the excitation light and the filter are changed to those suitable for the wavelength characteristics of the fluorescent staining reagent B without changing the field of view and the imaging magnification, and imaging is performed by the imaging means to represent the emission of the fluorescent staining reagent B. 2 is generated, and the image data is transmitted to the image processing apparatus 2A.

  Thereafter, in the image processing apparatus 2A, image analysis processing for analyzing the expression of the biological substance in the first fluorescent image and the second fluorescent image is executed. FIG. 3 shows a flowchart of the image analysis process.

First, when a morphological image is input from the microscope image acquisition device 1A through the communication I / F 24 (step S11: input step), the control unit 21 extracts a region of a cell nucleus from the morphological image (step S12: region extraction step). ).
In step S12, for example, as shown in FIG. 4, the morphological image is converted into a monochrome image (step S121), and the monochrome image is subjected to threshold processing with a predetermined threshold to binarize each pixel value (step S122). ) Noise processing is executed on the binary image (step S123).

Specifically, the noise process can be performed by performing a closing process on the binary image. The closing process is a process in which the contraction process is performed the same number of times after the expansion process is performed. The expansion process is a process of replacing a target pixel with white when at least one pixel in the range of n × n pixels (n is an integer of 2 or more) from the target pixel is white. The contraction process is a process of replacing a target pixel with black when at least one pixel in the range of n × n pixels from the target pixel contains black. By the closing process, a small area such as noise can be removed.
Through steps S121 to S123, a cell nucleus image in which the cell nucleus region is extracted from the morphological image can be generated.

  Next, when the first fluorescence image acquired by the microscope image acquisition apparatus 1A is input by the communication I / F 24 (step S13: input process), the control unit 21 extracts the fluorescent luminescent spot from the first fluorescence image. (Step S14: Bright spot extraction step).

Specifically, in step S14, as shown in FIG. 5, a color component corresponding to the peak of the emission spectrum of the fluorescent staining reagent A is extracted from the first fluorescence image (step S141), and the color component after the color component extraction is extracted. A threshold value process is performed on the single fluorescent image to generate a binary image from which the fluorescent bright spot region has been extracted (step S142). Further, in the binary image, the center of gravity of each fluorescent luminescent spot region is calculated, and a fluorescent luminescent spot image extracted by using this as the bright spot coordinates is generated (step S143).
In step S141, only the fluorescent bright spots whose peak luminance of the emission wavelength component of the staining reagent A is equal to or higher than a predetermined threshold are extracted. Prior to the threshold processing in step S142, noise removal processing such as cell autofluorescence and other unnecessary signal components may be performed.

  After the step S14, an addition process of the cell nucleus image and the fluorescent luminescent spot image is executed to superimpose the cell nucleus region and the fluorescent luminescent spot (step S15).

  Next, based on the positional relationship between the cell nucleus region and the fluorescent luminescent spot, the type of the biological material represented by the fluorescent luminescent spot shown in the first fluorescent image is determined (step S16: determination step).

  Specifically, in one example of the determination step of step S16 in the first embodiment, first, the distance from the fluorescent bright spot to the contour of the nearest cell nucleus region is calculated. Then, threshold processing is applied to the calculated distance, and it is determined whether the fluorescent bright spot indicates the expression of Ki67 expressed in the cell nucleus or the expression of CK7 expressed in the cytoplasm.

FIG. 6A is a schematic diagram of an image obtained by superimposing the fluorescence bright spots extracted from the cell nucleus region 1 and the first fluorescence image in step S15. FIG. 6B is an example of a histogram in which the distance to the closest cell nucleus on the horizontal axis and the number of fluorescent bright spots on the vertical axis. A fluorescent luminescent spot with a short distance to the nearest cell nucleus (for example, a predetermined threshold A or less) is determined to be a fluorescent luminescent spot indicating the expression of Ki67 expressed in the cell nucleus (see FIG. 6C). A fluorescent luminescent spot having a long distance to the nearest cell nucleus (for example, larger than a predetermined threshold A) is determined to be a fluorescent luminescent spot indicating the expression of CK7 expressed in the cytoplasm (FIG. 6C). reference).
As the predetermined threshold A, a value set in advance based on the type of tissue specimen or the like is used.

  Next, the fluorescent bright spot determined to show Ki67 expression in step S16 is selected from the fluorescent bright spot image (step S17: selection step), and an image showing Ki67 expression is created.

In the first embodiment, when determining the type of biological material represented by the fluorescent bright spot indicated in the second fluorescent image, first, in steps S11 to S16 described above, the second fluorescent image is used instead of the first fluorescent image. The same process is performed except that a fluorescent image is input.
Next, from the fluorescent luminescent spot image created from the second fluorescent image, the fluorescent luminescent spot determined to show the expression of CK7 in step S16 is selected (step S17: selection step), and the image showing the expression of CK7 Is created.

In step S16 (determination step) of the first embodiment, for example, it is determined that the fluorescent luminescent spot inside or on the outline of the cell nucleus region is a fluorescent luminescent spot indicating the expression of Ki67 expressed in the cell nucleus. Also good. In this case, a fluorescent bright spot that is not included in or inside any cell nucleus region is determined to be a fluorescent bright spot indicating the expression of CK7 expressed in the cytoplasm.
In addition, a fluorescent bright spot far from a predetermined value from the cell nucleus may be determined as noise outside the cell and deleted.
In step S16 (determination step), the distance from the fluorescent luminescent point to the contour of the closest cell nucleus region is calculated. For example, the distance from the fluorescent luminescent point to the center of gravity of the cell nucleus region may be calculated. Good.
Further, in step S17 (selection step), the fluorescent luminescent spot determined to show the expression of CK7 in step S16 may be deleted from the fluorescent luminescent spot image, and an image showing the expression of Ki67 may be created.

(Second Embodiment)
In the second embodiment of the present invention, the biological material represented by the fluorescent luminescent spot is based on the distance between the region of the cell or intracellular organ extracted from the morphological image (fluorescent image) and the fluorescent luminescent spot extracted from the fluorescent image. Determine the type.

  Hereinafter, tissue sections in which the cell membrane is stained with a fluorescent staining reagent and Ki67 expressed in the cell nucleus and HER2 expressed in the cell membrane are stained with two types of fluorescent staining reagents having different luminescent properties are used as tissue samples. A case will be described as an example.

The fluorescent staining reagent in the second embodiment is, for example, fluorescent substance-encapsulated nanoparticles bound with an anti-Ki67 antibody (fluorescent staining reagent A) and fluorescent substance-encapsulated nanoparticles bound with an anti-HER2 antibody (fluorescent staining reagent B).
The fluorescent staining of the cell membrane may be performed by any known method. For example, after reacting a tissue specimen with anti-ATPase (primary antibody), a secondary antibody bound to a fluorescent dye (hereinafter referred to as fluorescent staining reagent X). And the cell membrane is fluorescently stained.
As a morphological image representing the morphology of the cell membrane, a fluorescence image is acquired.
Hereinafter, the second embodiment will be described focusing on the configuration different from the above-described first embodiment, and description of the common configuration will be omitted.

First, the operator stains the cell membrane of the tissue section, and then performs fluorescent staining of Ki67 and HER2.
Then, a fluorescence image is acquired by the following procedures (a1) to (a6) using the microscope image acquisition device 1A.
(A1) The operator places the tissue section on a slide and places the slide on the slide fixing stage of the microscope image acquisition apparatus 1A.
(A2) The unit is set as a bright field unit, the imaging magnification and the focus are adjusted, and the region to be observed on the tissue section is placed in the field of view.
(A3) Change the unit to a fluorescent unit.
(A4) Image data of a fluorescent image (morphological image) representing light emission of the fluorescent staining reagent X is generated by imaging with an imaging unit, and the image data is transmitted to the image processing apparatus 2A. As the excitation light and the filter, those suitable for the wavelength characteristics of the fluorescent staining reagent X are used.
(A5) First, the excitation light and the filter are changed to those suitable for the wavelength characteristics of the fluorescent staining reagent A without changing the field of view and the imaging magnification, and imaging is performed with the imaging means to represent the emission of the fluorescent staining reagent A. Image data of one fluorescent image is generated, and the image data is transmitted to the image processing apparatus 2A.
(A6) First, the excitation light and the filter are changed to those suitable for the wavelength characteristics of the fluorescent staining reagent B without changing the field of view and the imaging magnification, and imaging is performed by the imaging means to represent the emission of the fluorescent staining reagent B. 2 is generated, and the image data is transmitted to the image processing apparatus 2A.

  Thereafter, in the image processing apparatus 2A, image analysis processing for analyzing the expression of the biological substance in the first fluorescent image and the second fluorescent image is executed.

In the image analysis processing for analyzing the expression of the biological material in the first fluorescence image in the second embodiment, first, when the morphological image acquired by the microscope image acquisition device 1A is input by the communication I / F 24 (step S11: The input unit), the control unit 21 extracts a cell membrane image in which the cell membrane is extracted from the morphological image (step S12: region extraction step).
Next, fluorescent luminescent spots are extracted from the first fluorescent image in the same manner as steps S13 to S14 of the first embodiment.
Thereafter, similarly to step S15 of the first embodiment, the addition process of the cell membrane image and the fluorescent luminescent spot image is executed, and the cell membrane region and the fluorescent luminescent spot are superimposed (step S15).

Next, based on the positional relationship between the cell nucleus region and the fluorescent luminescent spot, the type of the biological material represented by the fluorescent luminescent spot shown in the first fluorescent image is determined (step S16: determination step).
In an example of the determination process of step S16 in the second embodiment, first, the distance from the fluorescent bright spot to the contour of the closest cell nucleus region is calculated. Then, threshold processing is performed on the calculated distance, and it is determined whether the fluorescent bright spot indicates the expression of Ki67 expressed in the cell nucleus or the expression of HER2 expressed in the cell membrane.

  FIG. 7A is a schematic diagram of an image obtained by superimposing the fluorescent luminescent spots extracted from the cell membrane region 2 and the first fluorescent image of the second embodiment in step S15. FIG. 7B is an example of a histogram in which the horizontal axis represents the distance from the fluorescent bright spot to the nearest cell membrane, and the vertical axis represents the number of fluorescent bright spots. A fluorescent luminescent spot having a short distance to the closest cell membrane (for example, a predetermined threshold A or less) is determined to be a fluorescent luminescent spot indicating the expression of HER2 expressed in the cell membrane (FIG. 7 (c)). reference). A fluorescent bright spot whose distance to the nearest cell nucleus is greater than a predetermined threshold A and equal to or less than a predetermined threshold B is determined to be a fluorescent bright spot indicating the expression of Ki67 expressed in the cell nucleus (FIG. 7 (c)). A fluorescent luminescent spot having a long distance to the nearest cell nucleus (for example, larger than a predetermined threshold B) is determined to be noise existing outside the cell (FIG. 7C, fluorescent luminescent spot 20).

.
Next, only the fluorescent luminescent spot determined to show the expression of HER2 in step S16 is selected from the fluorescent luminescent spot image (step S17: selection step), and an image showing the expression of HER2 is created.

  Note that the distance from the fluorescent luminescent spot to the closest cell membrane in step S15 may be expressed as a negative value when the fluorescent luminescent spot is included in a region surrounded by the cell membrane, for example. Thereby, a fluorescent bright spot derived from extracellular noise slightly separated from the cell membrane can be distinguished from a fluorescent bright spot representing the expression of Ki67 in the cell nucleus. Therefore, it is possible to increase the accuracy of determining the type of bright spot when there is a lot of extracellular noise.

(Third embodiment)
In the third embodiment of the present invention, the type of biological material represented by the fluorescent bright spot is determined based on the feature quantity of the region of the cell or the organ that is closest to the fluorescent bright spot extracted from the morphological image.
The feature amount is, for example, the size (area, circumference, minor axis, or major axis length, etc.), shape (circularity, ratio of minor axis to major axis, etc.), and / or dyeing state (dyeing). Presence or absence of unevenness, etc.).

Hereinafter, tissue sections in which cell nuclei were stained by H staining, and further, NSE (marker for small cell carcinoma, etc.) and CEA (marker for adenocarcinoma, etc.) were each stained with two types of fluorescent staining reagents having different luminescence properties. Is used as an example of a tissue sample.
The fluorescent staining reagent in the third embodiment is, for example, a fluorescent substance-encapsulating nanoparticle (fluorescent staining reagent A) bound with an anti-NSE antibody and a fluorescent substance-encapsulating nanoparticle (fluorescent staining reagent B) bound with an anti-CEA antibody.

A bright-field image is acquired as a morphological image representing the morphology of the cell nucleus. As the feature quantity of the cell or the organ in the cell, the shape, area, and number of nucleoli in the cell nucleus are used.
In general, it is known that the nucleolus of adenocarcinoma cells is one or two round and large in size per cell, whereas the nucleolus of small cell carcinoma cells is small and inconspicuous. . Based on this finding, it is possible to determine which type of cancer cells the cell nucleus extracted from the morphological image.
Hereinafter, the third embodiment will be described with a focus on the configuration different from the first and second embodiments described above, and the description of the common configuration will be omitted.

First, the operator performs H staining of the tissue section, and then performs fluorescent staining of NSE and CEA.
Thereafter, a bright-field image and a fluorescence image are acquired by using the microscope image acquisition device 1 </ b> A by the same procedure and the same as (a1) to (a6) of the first embodiment.
Thereafter, in the image processing apparatus 2A, image analysis processing for analyzing the expression of the biological substance in the first fluorescent image and the second fluorescent image is executed.

In the image analysis process for analyzing the expression of the biological material in the first fluorescence image in the third embodiment, first, the same process as in steps S11 to S12 in the first embodiment is performed, and then the same process as in step S12. Is repeated once to generate a nucleolus image in which the nucleolus region is extracted from the bright field image. The nucleolus region is stained particularly dark blue by H staining inside the cell nucleus region, and can be extracted by changing the threshold used in step S122 of the first embodiment.
Next, the cell nucleus region, the nucleolus region, and the fluorescent bright spot are overlapped by the same process as in steps S13 to S15 of the first embodiment.

  Next, in step S16 (: determination step) of the third embodiment, the biological material represented by the fluorescent luminescent spot indicated in the first fluorescent image based on the size of the nucleolus in the cell nucleus closest to the fluorescent luminescent spot. Determine the type.

FIG. 8A is a schematic diagram of an image obtained by superimposing the fluorescent luminescent spots extracted from the cell nucleus region 3, the nucleolus region 31, and the first fluorescence image in step S15.
In step S16 of the third embodiment, for example, a cell nucleus region closest to the fluorescent luminescent spot is extracted, and the area of the nucleolus region included in the cell nucleus region is measured. When the total area of the nucleolus region is less than or equal to a predetermined threshold, it is determined as a fluorescent bright spot representing the expression of NSE, which is a biological substance expressed in cells of small cell carcinoma, and from the predetermined threshold Is larger, it is determined as a fluorescent bright spot representing the expression of CEA, which is a biological material expressed in adenocarcinoma cells (see FIG. 8B). Although not shown, if the distance from the fluorescent luminescent spot to the nearest cell nucleus region is too long, it is determined that the fluorescent luminescent spot is derived from extracellular noise regardless of the area of the nucleolus region. It is good.

.
Next, the fluorescent luminescent spot determined to show the expression of NSE in step S16 is selected from the fluorescent luminescent spot image (step S17: selection step), and an image showing the expression of NSE is created.

(Fourth embodiment)
In the fourth embodiment of the present invention, based on the color information of the fluorescent bright spot, in addition to the distance between the region of the cell or intracellular organ extracted from the morphological image and the fluorescent bright spot extracted from the fluorescent image, The type of biological material represented by the point is determined.
Hereinafter, the cell nucleus is stained by H staining, and HER2 and EGFR (epidermal growth factor receptor) expressed in the cell membrane and Ki67 expressed in the cell nucleus are respectively stained by three kinds of fluorescent staining reagents having different luminescence characteristics. An example of using a tissue section as a tissue specimen will be described.
The fluorescent staining reagent in the fourth embodiment includes, for example, fluorescent substance-encapsulating nanoparticles bound with anti-HER2 antibody (fluorescent staining reagent A), fluorescent substance-encapsulating nanoparticles bound with anti-EGFR antibody (fluorescent staining reagent B), and anti-HER2 antibody. It is fluorescent substance inclusion | inner_cover nanoparticle (fluorescence dyeing reagent C) which Ki67 antibody couple | bonded.
A bright-field image is acquired as a morphological image representing the morphology of the cell nucleus.
Hereinafter, the fourth embodiment will be described with a focus on the configuration different from the first to third embodiments described above, and the description of the common configuration will be omitted.

First, the operator performs H staining of the tissue section, and then performs fluorescent staining of HER2, EGFR, and Ki67.
Thereafter, using the microscope image acquisition device 1A, a bright-field image and a fluorescence image are acquired by the same procedure as (a1) to (a4) of the first embodiment and the following procedure (a5). In the fourth embodiment, to acquire a fluorescent image, it is necessary to acquire a color image using a camera having a plurality of color filters.
(A5) First, the excitation light and the filter are changed to those suitable for the wavelength characteristics of the fluorescent staining reagent C without changing the field of view and the imaging magnification, and the imaging means is used for imaging to express the expression of the fluorescent staining reagent C. 3 is generated, and the image data is transmitted to the image processing apparatus 2A.

Thereafter, in the image processing apparatus 2A, an image analysis process for analyzing the expression of the biological material in the first fluorescence image, the second fluorescence image, and the third fluorescence image is executed.
Hereinafter, the image analysis processing in the image processing apparatus 2A of the second embodiment will be described focusing on the configuration different from the image processing of the first to third embodiments described above, and description of the common configuration will be omitted.

In the image analysis process for analyzing the expression of the biological material in the first fluorescence image in the fourth embodiment, first, the control unit 21 performs the same process as steps S11 to S141 in the first embodiment.
In step S142 of the fourth embodiment, as described in the first embodiment, after generating the binary image from which the fluorescent luminescent spots are extracted, the binary image is used as a mask to set the fluorescent luminescent spots of the first fluorescent image. An image from which a region has been extracted is generated and used as a fluorescent bright spot region image. Then, the process similar to step S143 of 1st Embodiment is performed, and a fluorescence luminescent point image is produced | generated.

  After the step S14, addition processing of the cell nucleus image, the fluorescent bright spot region image, and the fluorescent bright spot image is executed to superimpose the cell nucleus region, the fluorescent bright spot region, and the fluorescent bright spot (step S15).

  Subsequently, based on the positional relationship between the cell nucleus region and the fluorescent luminescent spot and the color information of the fluorescent luminescent spot region, the type of the biological material represented by the fluorescent luminescent spot shown in the first fluorescent image is determined (step S16: determination step). .

  Specifically, in one example of the determination process of step S16 in the fourth embodiment, first, the distance from the fluorescent bright spot to the contour of the nearest cell nucleus region and the hue of the fluorescent bright spot are calculated. Then, for example, the type of biological material indicated by the fluorescent bright spot is determined as follows.

FIG. 9A is a schematic diagram of an image obtained by superimposing the fluorescence bright spots extracted from the cell nucleus region 4 and the first fluorescence image in step S15. In FIG. 9B, the horizontal axis indicates the hue of the fluorescent bright spot region (the average value of the hue within one region and is expressed in the range of 0 to 360 degrees), and the vertical axis indicates the closest point from the fluorescent bright spot. It is an example of the scatter diagram which took the distance to a cell nucleus. Among the fluorescent luminescent spots shown in the scatter diagram, the fluorescent luminescent spot having a short distance to the nearest cell nucleus (the predetermined threshold A or less) is determined to be a fluorescent luminescent spot indicating the expression of Ki67 expressed in the cell nucleus. Is done.
Furthermore, among the closest to the cell nucleus is long (greater than a predetermined threshold value A) fluorescent bright spot, for example, fluorescent bright spot color is in the range of 0～H _B is expressed on the cell membrane HER2 A fluorescent bright spot having a hue in the range of H _{B to} 360 is determined to be a fluorescent bright spot indicating the expression of EGFR expressed in the cell membrane. H _B is a value determined in advance based on the emission wavelengths of the fluorescent staining reagents A to C.

.
Next, from the fluorescent bright spot images, fluorescent bright spots determined to show expression of Ki67, EGFR, and HER2 in step S16 are respectively selected (step S17: selection process), and an image showing the expression of each biological substance is obtained. Created.

(Fifth embodiment)
In the fifth embodiment of the present invention, as in the first embodiment, fluorescence is determined based on the form of the region of the cell or intracellular organ extracted from the morphological image and the distance between the fluorescent bright spots extracted from the fluorescent image. The type of biological material represented by the bright spot is determined.
Hereinafter, a case where a tissue section in which cell nuclei are stained by H staining and Ki67 expressed in the cell nuclei and HER2 expressed in the cell membrane are stained by two types of fluorescent staining reagents having different emission wavelengths is used as a tissue specimen. Let's take an example.
The fluorescent staining reagent in the fifth embodiment is, for example, a fluorescent substance-encapsulating nanoparticle (fluorescent staining reagent A) bound with an anti-Ki67 antibody and a fluorescent substance-encapsulating nanoparticle (fluorescent staining reagent B) bound with an anti-HER2 antibody.
A bright-field image is acquired as a morphological image representing the morphology of the cell nucleus.
Hereinafter, the fifth embodiment will be described with a focus on the configuration different from the first to fourth embodiments described above, and the description of the common configuration will be omitted.

First, the operator performs H staining of the tissue section, and then performs fluorescent staining of Ki67 and HER2. Thereafter, the bright field image, the first fluorescent image, and the second fluorescent image are acquired by the same procedure as (a1) to (a6) of the first embodiment using the microscope image acquisition device 1A.
Thereafter, in the image processing apparatus 2A, image analysis processing for analyzing the expression of the biological substance in the first fluorescent image and the second fluorescent image is executed.
In the image analysis process for analyzing the expression of the biological material in the first fluorescence image in the fifth embodiment, first, the control unit 21 performs the same process as steps S11 to S15 in the first embodiment.

  Next, in step S16 (determination step) of the fifth embodiment, on the basis of the distance from the fluorescent luminescent spot to the nearest cell nucleus, the probability that the fluorescent luminescent spot is each fluorescently stained biological material is calculated, The type of biological material indicated by the fluorescent bright spot is determined.

  FIG. 10A is an example of a graph showing the correlation between the distance from the fluorescent luminescent spot to the nearest cell nucleus and the probability that the fluorescent luminescent spot is Ki67 expressed in the cell nucleus (nucleus-attached luminescent spot count). According to this graph, for example, when the distance from the fluorescent luminescent spot to the nearest cell nucleus is Ax, the nucleus-attached luminescent spot count is 0.7. It is determined that Ki67 and 0.3 HER2.

  Next, from the fluorescent light spot image, the fluorescent light spot determined to have a probability of Ki67 expression of zero in step S16 is deleted, and only the fluorescent light spot determined to show Ki67 expression is selected (Step S16). S17: Selection step), an image showing the expression of Ki67 is created.

In step S16 of the first embodiment and the fifth embodiment, the type of the biological material indicated by the fluorescent luminescent spot using the positional relationship between the form of the cell or intracellular organ represented in the morphological image and the fluorescent luminescent spot. However, in step S16 of the first embodiment described above, the step of the fifth embodiment is that each fluorescent luminescent spot is specified as any one of a plurality of kinds of stained biological materials. Different from S16.
FIG. 10B is an example of a graph showing the correlation between the distance from the fluorescent luminescent spot to the nearest cell nucleus and the nucleus-attached luminescent spot count in the first embodiment. In the first embodiment, if the distance from the fluorescent luminescent spot to the nearest cell nucleus is equal to or less than the threshold A, for example, even if it is either Ax or 0, the nucleus-attached luminescent spot count is always 1, The fluorescent bright spot is measured as one Ki67.

(Modification)
In the first to fifth embodiments, the image capturing range of the image is changed due to various factors such as switching of a unit for capturing a bright field image and a dark field image and switching of excitation light and a filter when capturing a fluorescent image. The position may be shifted. Therefore, in the first to fifth embodiments, when image processing is performed using a plurality of images, when images are superimposed in step S15, based on a structure that can be commonly recognized by the plurality of images to be superimposed. Thus, it is preferable to perform image alignment.

For image alignment, any known method can be used.
Specifically, for example, eosin staining is performed on a tissue specimen. By eosin staining, the cytoplasmic shape can be observed in both bright field images and fluorescent images. Therefore, in the first to fifth embodiments, the eosin staining area in the morphological image and the eosin staining area in the fluorescence image are extracted in each of steps S12 and S14, and these engine staining areas completely overlap in step S15. As described above, after performing the alignment for adjusting the positions of the morphological image and the fluorescent image, it is possible to correct the image misalignment by superimposing the images. The type of biological material can be accurately determined.

According to the first to fifth embodiments and the modifications described above, the type of the biological material represented by the fluorescent bright spot can be determined based on the morphological image.
As a result, even if a plurality of biological materials are stained using a plurality of types of fluorescent materials having different emission wavelengths, the types of biological materials represented by the fluorescent luminescent spots can be easily determined. The error of quantitative expression of the specific protein in can be easily reduced, and the diagnostic accuracy is improved.

  In addition, the description content in the said embodiment is a suitable example of this invention, and is not limited to this.

  For example, in the first embodiment and the fifth embodiment, instead of the distance between the region of the cell or the organ in the cell extracted from the morphological image and the fluorescent luminescent spot extracted from the fluorescent image, for example, the hue of the fluorescent luminescent spot Alternatively, based on the area of the nucleolus, the type of biological material represented by the fluorescent bright spot may be determined.

  Moreover, you may perform the determination process in the image analysis process of the 1st-5th embodiment demonstrated above and a modification in combination. For example, in the fourth embodiment described above, first, based on the distance from the fluorescent luminescent spot to the closest cell nucleus, the fluorescent luminescent spot determined to show the expression of Ki67 expressed in the cell nucleus is extracted, and the remaining fluorescent luminescent spots are extracted. For the points, the probability that the fluorescent luminescent spots are EGFR and HER2, respectively, is calculated based on the hue of the fluorescent luminescent spot region, and the type of biological material indicated by each fluorescent luminescent spot is determined. To do.

  In addition, the type of biological material to be quantified in the present invention is not limited to the example used in the above embodiment, and the biological material when acquiring a fluorescent image according to the lesion (cancer) type to be diagnosed If the recognition sites are different, it is possible to provide a doctor with a feature quantity that quantitatively indicates the expression level of the specific protein corresponding to the lesion type. For example, in the diagnosis of vascular tumor or angiosarcoma, fluorescent staining of CD31 or CD34 is performed. Moreover, adenocarcinoma of unknown primary origin can be diagnosed by performing multiple staining of CK7 and CK20 expressed in the cytoplasm and quantifying.

In the above description, an example in which an HDD, a semiconductor nonvolatile memory, or the like is used as a computer-readable medium for the program according to the present invention is disclosed, but the present invention is not limited to this example. As other computer-readable media, a portable recording medium such as a CD-ROM can be applied. Further, a carrier wave (carrier wave) is also applied as a medium for providing program data according to the present invention via a communication line.
In addition, the detailed configuration and detailed operation of each device constituting the diagnosis support information generating system 100 can be changed as appropriate without departing from the spirit of the invention.

DESCRIPTION OF SYMBOLS 1A Microscope image acquisition apparatus 2A Image processing apparatus 21 Control part 22 Operation part 23 Display part 24 Communication I / F
25 storage unit 26 bus 3A cable 100 diagnostic support information generation system

Claims (11)

A fluorescence image showing the expression of the plurality of types of biological materials as fluorescent luminescent spots, which is obtained by imaging tissue samples obtained by staining a plurality of types of biological materials using a plurality of types of fluorescent materials having different luminescence characteristics, and the tissue sample An input means for inputting a captured morphological image representing the morphology of a cell or an intracellular organ;
A region extracting means for extracting a region of the cell or organ from the morphological image;
A bright spot extracting means for extracting the fluorescent bright spot from the fluorescent image;
Based on the region and the fluorescent luminescent spot, determination means for determining the type of biological material represented by the fluorescent luminescent spot;
With
The determination means determines the type of biological material represented by the fluorescent luminescent spot based on at least one of the following (a), (b), and (c):
An image processing apparatus.
(A) The distance of the cell or organ that is closest to the fluorescent luminescent spot
(B) Whether or not the fluorescent bright spot is encapsulated in the cell or an intracellular organ
(C) Characteristic amount of the cell or organ that is closest to the fluorescent luminescent spot A fluorescence image showing the presence of the plurality of types of specific substances as fluorescent luminescent spots, and images of tissue samples obtained by staining a plurality of types of specific substances using a plurality of types of fluorescent substances having different emission characteristics, and the tissue samples An input means for inputting a captured morphological image representing the morphology of a cell or an intracellular organ;
A region extracting means for extracting a region of the cell or organ from the morphological image;
A bright spot extracting means for extracting the fluorescent bright spot from the fluorescent image;
Determination means for determining the type of the specific substance represented by the fluorescent luminescent spot based on the region and the fluorescent luminescent spot;
An image processing apparatus comprising: The image processing apparatus according to claim 2 , wherein the region is any one of a region of a cell membrane, a cytoplasm, a cell nucleus, or a nucleolus. Said determination means, the following (a), (b), and based on at least one of (c), claim, wherein the determining the type of the specific substance the representative fluorescent bright spots 2 Or the image processing apparatus of 3 .
(A) Distance between the fluorescent luminescent spot and the cell or organ inside the cell (b) Whether the fluorescent luminescent spot is included in the cell or intracellular organ (c) The cell closest to the fluorescent luminescent spot Or feature quantity of intracellular organ The image processing apparatus according to claim 1 or 4, wherein the characteristic amount is the area. The image processing apparatus according to claim 2 , wherein the determination unit determines a type of a specific substance represented by the fluorescent luminescent spot based on a color of the fluorescent luminescent spot. . The determination means, the type of a specific substance that fluorescent bright spots represent, according to any one of claims 2-6, characterized in that determining said plurality of types of is any one of a specific substance Image processing apparatus. The determination means includes
Calculating a probability that the fluorescent luminescent spot represents each of the plurality of types of specific substances ;
The image processing apparatus according to claim 2 , wherein the type of the specific substance represented by the fluorescent bright spot is determined. Based on the result determined by the determination means, wherein the fluorescent bright spots, claim, characterized in that it comprises a selection means for selecting a fluorescent bright spots representing a particular specific substance among the plurality of types of specific substance 7 An image processing apparatus according to 1. Computer
A fluorescence image showing the presence of the plurality of types of specific substances as fluorescent luminescent spots, and images of tissue samples obtained by staining a plurality of types of specific substances using a plurality of types of fluorescent substances having different emission characteristics, and the tissue samples An input means for inputting a captured morphological image representing the morphology of a cell or an intracellular organ;
An area extracting means for extracting an area of the cell or organ from the morphological image;
A bright spot extracting means for extracting the fluorescent bright spot from the fluorescent image;
Determination means for determining the type of the specific substance represented by the fluorescent luminescent spot based on the region and the fluorescent luminescent spot,
Program to function as. A tissue image in which a specific substance is stained with a fluorescent substance is imaged, a fluorescence image indicating the presence of the specific substance with a fluorescent bright spot, and a morphological image representing the form of a cell or an intracellular organ is imaged in the tissue specimen. Input means for inputting;
A region extracting means for extracting a region of the cell or organ from the morphological image;
A bright spot extracting means for extracting the fluorescent bright spot from the fluorescent image;
Determination means for determining the type of the specific substance represented by the fluorescent luminescent spot based on the region and the fluorescent luminescent spot;
With
The determination means determines the type of a specific substance represented by the fluorescent bright spot based on at least one of the following (a), (b), and (c): .
(A) The distance of the cell or organ that is closest to the fluorescent luminescent spot
(B) Whether or not the fluorescent bright spot is encapsulated in the cell or an intracellular organ
(C) Characteristic amount of the cell or organ that is closest to the fluorescent luminescent spot

Images