JPWO2015190225A1 - Diagnosis support information generation method, image processing apparatus, diagnosis support information generation system, and image processing program - Google Patents

Abstract

Provided are a diagnosis support information generation method, an image processing apparatus, a diagnosis support information generation system, and an image processing program capable of obtaining more accurate diagnosis information. In a diagnosis support information generation method for generating diagnosis support information using a specimen in which a specific biological material is stained with a staining reagent using a fluorescent substance, a first method that expresses the expression of the biological substance in the specimen as a fluorescent bright spot. A first input step for inputting an image; and analysis region information for extracting a predetermined region of the sample as an analysis region and calculating analysis region information including at least one of the number, area, and circumference of the analysis region A calculation step, a bright spot region information extraction step of extracting a bright spot region indicating the fluorescent bright spot from the first image, and extracting bright spot region information including information on the distribution of the bright spot region, and the analysis And an image score calculating step of calculating an image score as the diagnosis support information based on the region information and the bright spot region information.

Description

  The present invention relates to a diagnosis support information generation method, an image processing apparatus, a diagnosis support information generation system, and an image processing program.

In recent years, with the spread of molecular targeted drug treatment centering on antibody drugs, in order to design molecular targeted drugs more effectively, quantification of biological substances on cells to be observed has been demanded. As a method for confirming the presence of a biological material, a tissue analysis method based on the binding of a fluorescent material to which a biological material recognition site is bound and a biological material corresponding to the biological material recognition site is known.
It is known that various noises that are unrelated to the degree of disease can occur in tissue specimens and cells to be observed, for example, fluorescent substances aggregate at the time of specimen preparation. If there are variations in the diagnostic results due to noise, there is a problem that it may lead to a misdiagnosis or it takes time to diagnose because the diagnostic results are reexamined one by one. Therefore, in the tissue analysis, it is difficult to be affected by noise, and it is a problem to obtain a stable and accurate quantitative result.

For example, Patent Document 1 describes scoring as a method of staining a biomarker and an intracellular compartment to extract each region and quantifying the amount of the biomarker present in the intracellular compartment with good reproducibility. Yes.
According to the method described in Patent Document 1, by scoring the amount of biomarker for each subcellular compartment, it is difficult to be affected by noise and quantification with high reproducibility is possible.

Special table 2012-503180 gazette

  However, it is known that the grade of malignancy of diseases such as cancer differs depending not only on the amount of biomarker expressed but also on the distribution bias. According to the invention of Patent Document 1, the expression level of a biomarker for each intracellular compartment can be evaluated, but the distribution of the biomarker in the intracellular compartment is not evaluated, and the accuracy of quantification is sufficient. I could not say.

  A main object of the present invention is to provide a diagnosis support information generation method, an image processing apparatus, a diagnosis support information generation system, and an image processing program capable of obtaining more accurate diagnosis information.

In order to solve the above problem, according to the invention described in claim 1,
In a diagnostic support information generation method for generating diagnostic support information using a specimen in which a specific biological material is stained with a staining reagent using a fluorescent substance,
A first input step of inputting a first image representing the expression of the biological material in the specimen as a fluorescent luminescent spot;
An analysis region information calculation step of extracting a predetermined region of the sample as an analysis region and calculating analysis region information including at least one of the number, area, and circumference of the analysis regions;
A bright spot region information extracting step of extracting a bright spot region indicating the fluorescent bright spot from the first image and extracting bright spot region information including information on the distribution of the bright spot region;
Based on the analysis area information and the bright spot area information, an image score calculation step of calculating an image score that is the diagnosis support information;
A diagnostic support information generation method characterized by comprising:

According to the invention described in claim 2, in the diagnosis support information generation method according to claim 1,
The staining reagent is a fluorescent particle in which a plurality of the fluorescent substances are integrated,
The bright spot area information includes information on the number of the fluorescent particles contained in each of the bright spot areas, and provides a diagnosis support information generation method characterized in that

According to the invention described in claim 3, in the diagnosis support information generation method according to claim 1 or 2,
A second input step of inputting a second image capable of extracting a predetermined region of cells in the specimen;
A cell information calculation step of extracting a predetermined region of the cell from the second image as a cell region and calculating cell information including at least one of the number, area, and circumference of the cell region. And
The image score calculation step includes:
Based on the bright spot region information and the cell information, to calculate a cell score that has analyzed the expression of the biological material per cell region,
A diagnostic support information generation method is provided, wherein the image score is calculated based on the analysis region information and the cell score.

According to the invention described in claim 4, in the diagnosis support information generating method according to claim 1 or 2,
The image score calculation step includes:
Based on the bright spot area information, calculate a bright spot score by analyzing the expression of the biological material per bright spot area,
A diagnostic support information generation method is provided, wherein the image score is calculated based on the analysis region information and the bright spot score.

According to the invention described in claim 5, in the diagnosis support information generation method according to claim 1 or 2,
A second input step of inputting a second image capable of extracting a predetermined region of cells in the specimen;
A cell information calculation step of extracting a predetermined region of cells as a cell region from the second image and calculating cell information including at least one of the number, area, and circumference of the cell region;
Have
The image score calculation step includes:
Based on the bright spot area information, calculating a bright spot score by analyzing the expression of the biological material per bright spot area,
Based on the cell information and the bright spot score, calculate a cell score by analyzing the distribution of the bright spot score per cell area,
A diagnostic support information generation method is provided, wherein the image score is calculated based on the analysis region information and the cell score.

According to the invention of claim 6,
In an image processing apparatus that generates diagnosis support information using a specimen in which a specific biological substance is stained with a staining reagent using a fluorescent substance,
A first input means for inputting a first image representing the expression of the biological material in the specimen as a fluorescent luminescent spot;
An analysis region information calculating means for extracting a predetermined region of the sample as an analysis region and calculating analysis region information including at least one of the number, area, and circumference of the analysis region;
A bright spot area information extracting means for extracting a bright spot area indicating the fluorescent bright spot from the first image and extracting bright spot area information including information on the distribution of the bright spot area;
Based on the analysis area information and the bright spot area information, an image score calculation means for calculating an image score that is the diagnosis support information;
An image processing apparatus is provided.

According to the invention of claim 7,
An image processing apparatus according to claim 6;
An image acquisition device for acquiring the first image used in the image processing device;
A diagnostic support information generation system is provided.

According to the invention described in claim 8,
A computer that generates diagnosis support information using a specimen in which a specific biological material is stained with a staining reagent using a fluorescent material,
A first input means for inputting a first image representing the expression of the biological material in the specimen as a fluorescent bright spot;
An analysis region information calculating means for extracting a predetermined region of the sample as an analysis region and calculating analysis region information including at least one of the number, area, and circumference of the analysis regions;
A bright spot area information extracting unit that extracts a bright spot area indicating the fluorescent bright spot from the first image and extracts bright spot area information including information on a distribution of the bright spot area;
An image score calculating means for calculating an image score which is the diagnosis support information based on the analysis area information and the bright spot area information;
An image processing program for functioning as

  According to the present invention, since the number and distribution of specific biological substances expressed in a specimen can be scored and evaluated, more accurate diagnostic information can be obtained and appropriate treatment can be performed.

It is a figure which shows the system configuration | structure of a diagnostic assistance information generation system. It is a block diagram which shows the functional structure of the image processing apparatus of FIG. It is a figure which shows an example of a bright field image. It is a figure which shows an example of a fluorescence image. It is a flowchart which shows the image analysis process performed by the control part of FIG. It is a figure which shows a fluorescence image. It is a figure which shows the image from which the luminescent spot area | region was extracted. It is a schematic diagram which shows the example which calculates a bright spot score using the number of fluorescent particles and the area of a bright spot area | region. It is a flowchart which shows the detail of the process of step S10 of FIG. It is a figure which shows a bright field image. It is a figure which shows the image from which the cell area | region was extracted. It is a schematic diagram which shows the example which calculates a cell score using the average value of a bright spot score, and distribution of a bright spot area | region. It is a schematic diagram which shows the example which calculates a cell score using the number of fluorescent particles and the area of a bright spot area | region. It is a schematic diagram which shows the example which calculates a cell score using the average value and cell distribution of a cell score. It is a schematic diagram which shows the example which calculates an image score using the average value of a bright spot score, and distribution of a bright spot area | region. It is a schematic diagram which shows the example which calculates an image score using the number of fluorescent particles, and the area of a bright spot area | region. It is a schematic diagram which shows the specific example of the method of evaluating the distribution of the bright spot area | region per cell in step S12 of FIG. It is a schematic diagram which shows the specific example of the method of evaluating the distribution of the bright spot area | region per cell in step S12 of FIG. It is a schematic diagram which shows the specific example of the method of evaluating the distribution of the bright spot area | region per cell in step S12 of FIG. It is a schematic diagram which shows the specific example of the method of evaluating the distribution of the bright spot area | region per cell in step S12 of FIG. It is a schematic diagram which shows the specific example of the method of evaluating the distribution of the bright spot area | region per cell in step S12 of FIG. It is a schematic diagram which shows the specific example of the method of evaluating the distribution of the bright spot area | region per cell in step S12 of FIG. It is a schematic diagram which shows the specific example of the method of evaluating the distribution of the bright spot area | region per cell in step S12 of FIG. It is a schematic diagram which shows the specific example of the method of evaluating the distribution of the bright spot area | region per cell in step S12 of FIG. It is a schematic diagram which shows the specific example of the method of evaluating the distribution of the bright spot area | region per cell in step S12 of FIG. It is a schematic diagram which shows the specific example of the method of evaluating the distribution of the bright spot area | region per cell in step S12 of FIG. It is a schematic diagram which shows the example of calculation of the cell score and bright spot score in an image.

  Hereinafter, although the form for implementing this invention with reference to figures is demonstrated, this invention is not limited to these.

<Configuration of Diagnosis Support Information Generation System 100>
FIG. 1 shows an example of the overall configuration of a diagnostic support information generation system 100 using the fluorescent substance quantification method of the present invention. The diagnosis support information generation system 100 acquires a microscopic image of a human tissue sample stained with a predetermined staining reagent, and analyzes the acquired microscopic image, thereby expressing a specific biological substance in the tissue sample to be observed. This is a system that outputs a feature quantity that quantitatively represents.

  As shown in FIG. 1, the diagnostic support information generation system 100 is configured by connecting a microscope image acquisition device 1A and an image processing device 2A so that data can be transmitted and received via an interface such as a cable 3A. The connection method between the microscope image acquisition device 1A and the image processing device 2A is not particularly limited. For example, the microscope image acquisition apparatus 1A and the image processing apparatus 2A may be connected via a LAN (Local Area Network) or may be connected wirelessly.

The microscope image acquisition apparatus 1A is a known optical microscope with a camera, and acquires a microscope image of a tissue specimen on a slide placed on a slide fixing stage, and transmits the microscope image to the image processing apparatus 2A.
The microscope image acquisition apparatus 1A includes an irradiation unit, an imaging unit, an imaging unit, a communication I / F, and the like. The irradiation means includes a light source, a filter, and the like, and irradiates the tissue specimen on the slide placed on the slide fixing stage with light. The imaging means is composed of an eyepiece lens, an objective lens, and the like, and forms an image of transmitted light, reflected light, or fluorescence emitted from the tissue specimen on the slide by the irradiated light. The imaging means is a microscope-installed camera that includes a CCD (Charge Coupled Device) sensor and the like, captures an image formed on the imaging surface by the imaging means, and generates digital image data of the microscope image. The communication I / F transmits image data of the generated microscope image to the image processing apparatus 2A. In the present embodiment, the microscope image acquisition apparatus 1A includes a bright field unit that combines an irradiation unit and an imaging unit suitable for bright field observation, and a fluorescence unit that combines an irradiation unit and an imaging unit suitable for fluorescence observation. It is possible to switch between bright field / fluorescence by switching units. Any light source such as a mercury lamp, a xenon lamp, an LED, or a laser beam can be used as a light source for fluorescence observation.

  Note that the microscope image acquisition apparatus 1A is not limited to a microscope with a camera. For example, a virtual microscope slide creation apparatus (for example, a special microscope microscope) that acquires a microscope image of an entire tissue specimen by scanning a slide on a microscope slide fixing stage. Table 2002-514319 gazette) etc. may be used. According to the virtual microscope slide creation device, it is possible to acquire image data that allows the entire tissue specimen image on the slide to be viewed on the display unit at a time.

The image processing apparatus 2A calculates the expression distribution of a specific biological substance in the tissue specimen to be observed by analyzing the microscope image transmitted from the microscope image acquisition apparatus 1A.
FIG. 2 shows a functional configuration example of the image processing apparatus 2A. As shown in FIG. 2, the image processing apparatus 2 </ b> A includes a control unit 21, an operation unit 22, a display unit 23, a communication I / F 24, a storage unit 25, and the like, and each unit is connected via a bus 26. Yes.

  The control unit 21 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), and the like. The control unit 21 executes various processes in cooperation with various programs stored in the storage unit 25, and performs image processing 2A. Overall control of the operation. For example, the control unit 21 executes image analysis processing (see FIG. 5) in cooperation with a program stored in the storage unit 25, and the bright spot region information extraction unit, the analysis region information, the image score, and the cell information A function as a calculation means is realized.

  The operation unit 22 includes a keyboard having character input keys, numeric input keys, various function keys, and the like, and a pointing device such as a mouse, and a key pressing signal pressed by the keyboard and an operation signal by the mouse. Are output to the control unit 21 as an input signal.

  The display unit 23 includes a monitor such as a CRT (Cathode Ray Tube) or an LCD (Liquid Crystal Display), for example, and displays various screens according to instructions of a display signal input from the control unit 21. In the present embodiment, the display unit 23 functions as an output unit for outputting an image analysis result.

  The communication I / F 24 is an interface for transmitting and receiving data to and from external devices such as the microscope image acquisition apparatus 1A. The communication I / F 24 functions as a bright field image and fluorescent image input unit (first input unit and second input unit).

The storage unit 25 includes, for example, an HDD (Hard Disk Drive), a semiconductor nonvolatile memory, or the like. As described above, the storage unit 25 stores various programs, various data, and the like.
In addition, the image processing apparatus 2A may include a LAN adapter, a router, and the like and be connected to an external device via a communication network such as a LAN.

The image processing apparatus 2A in the present embodiment preferably performs analysis using the bright field image and the fluorescence image transmitted from the microscope image acquisition apparatus 1A.
A bright-field image is obtained by enlarging and photographing a tissue specimen stained with an H (hematoxylin) staining reagent, an HE (hematoxylin-eosin) staining reagent, or the like in a bright field in the microscope image acquisition apparatus 1A. It is a microscope image, Comprising: It is a cell form image showing the form of the cell in the said tissue specimen. Hematoxylin is a blue-violet pigment that stains cell nuclei, bone tissue, part of cartilage tissue, serous components, etc. (basophilic tissue, etc.). Eosin is a red to pink pigment that stains cytoplasm, soft tissue connective tissue, red blood cells, fibrin, endocrine granules, and the like (eosinophilic tissue, etc.). FIG. 3 shows an example of a bright field image obtained by photographing a tissue specimen subjected to HE staining.
Fluorescence images are obtained on tissue specimens that have been stained with a staining reagent that contains fluorescent substances that specifically bind to and / or react with specific biological substances or nanoparticles that contain fluorescent substances (referred to as fluorescent substance-containing nanoparticles). On the other hand, it is a microscope image obtained by irradiating the fluorescent substance-containing nanoparticles by irradiating excitation light of a predetermined wavelength in the microscope image acquisition device 1A, and enlarging and photographing this fluorescence.
That is, the fluorescence that appears in the fluorescence image indicates the expression of a specific biological material in the tissue specimen. FIG. 4 shows an example of the fluorescence image.

<Acquisition of fluorescence image>
Here, a method for acquiring a fluorescent image will be described in detail including a staining reagent used for acquiring the fluorescent image, a method for staining a tissue specimen with the staining reagent, and the like.

[Fluorescent substance]
Examples of the fluorescent substance used in the staining reagent for obtaining a fluorescent image include fluorescent organic dyes and quantum dots (semiconductor particles). When excited by ultraviolet to near infrared light having a wavelength in the range of 200 to 700 nm, it preferably emits visible to near infrared light having a wavelength in the range of 400 to 1100 nm.

  Fluorescent organic dyes include fluorescein dye molecules, rhodamine dye molecules, Alexa Fluor (Invitrogen) dye molecules, BODIPY (Invitrogen) dye molecules, cascade dye molecules, coumarin dye molecules, and eosin dyes. And a molecule, an NBD dye molecule, a pyrene dye molecule, a cyanine dye molecule, and an aromatic hydrocarbon molecule.

  Specifically, 5-carboxy-fluorescein, 6-carboxy-fluorescein, 5,6-dicarboxy-fluorescein, 6-carboxy-2 ′, 4,4 ′, 5 ′, 7,7′-hexachlorofluorescein, 6 -Carboxy-2 ', 4,7,7'-tetrachlorofluorescein, 6-carboxy-4', 5'-dichloro-2 ', 7'-dimethoxyfluorescein, naphthofluorescein, 5-carboxy-rhodamine, 6-carboxy Rhodamine, 5,6-dicarboxy-rhodamine, rhodamine 6G, tetramethylrhodamine, X-rhodamine, sulforhodamine B, sulforhodamine 101, and Alexa Fluor 350, Alexa Fluor 405, Alexa Fluor 430, Alexa Fluor 488, Alexa Fluor 500, Alexa Fluor 514, Alexa Fluor 532, Alexa Fluor 546, Alexa Fluor 555, A lexa Fluor 568, Alexa Fluor 594, Alexa Fluor 610, Alexa Fluor 633, Alexa Fluor 635, Alexa Fluor 647, Alexa Fluor 660, Alexa Fluor 680, Alexa Fluor 700, Alexa Fluor 750, BODIPY FL, BODIPY TMR, BODIPY 493 , BODIPY 530/550, BODIPY 558/568, BODIPY 564/570, BODIPY 576/589, BODIPY 581/591, BODIPY 630/650, BODIPY 650/665 (manufactured by Invitrogen), methoxycoumarin, eosin, NBD, pyrene , Cy5, Cy5.5, Cy7, HiLyte Fluor 594 (registered trademark, manufactured by Anaspec), DyLight 594 (registered trademark, manufactured by Thermo Scientific), dye molecule, ATTO 594 (registered trademark, manufactured by ATTO-TEC) ), MFP 594 (registered trademark, manufactured by Mobitec), 5,10,15,20-tetraphenylporphine tetrasulfonic acid, zinc 5,10,15,20-tetraphenylporphine tetrasulfonic acid, phthalocyanine tetrasulfonic acid, zinc Phthalocyanine Torasulfonic acid, N, N-Bis- (2,6-diisopropylphenyl) -1,6,7,12- (4-tert-butylphenoxy) -perylen-3,4,9,10-tetracarbonacid diimide, N, N '-Bis (2,6-diisopropylphenyl) -1,6,7,12-tetraphenoxyperylene-3,4: 9,10-tetracarboxdiimide, Benzenesulfonic acid, 4,4', 4 '', 4 '' '-[( 1,3,8,10-tetrahydro-1,3,8,10-tetraoxoperylo [3,4-cd: 9,10-c'd '] dipyran-5,6,12,13-tetrayl) tetralis (oxy )] Tetrakis- and the like. Fluorescent organic dyes may be used singly or as a mixture of plural kinds.

  Quantum dots include II-VI group compounds, III-V group compounds, or quantum dots containing group IV elements as components ("II-VI group quantum dots", "III-V group quantum dots", " Or “Group IV quantum dots”). You may use individually or what mixed multiple types.

  Specific examples include, but are not limited to, CdSe, CdS, CdTe, ZnSe, ZnS, ZnTe, InP, InN, InAs, InGaP, GaP, GaAs, Si, and Ge.

It is also possible to use a quantum dot having the above quantum dot as a core and a shell provided thereon. Hereinafter, as a notation of a quantum dot having a shell in this specification, when the core is CdSe and the shell is ZnS, it is expressed as CdSe / ZnS. For example, CdSe / ZnS, CdS / ZnS, InP / ZnS, InGaP / ZnS, Si / SiO2, Si / ZnS, Ge / GeO2, Ge / ZnS, and the like can be used, but not limited thereto.
As the quantum dots, those subjected to surface treatment with an organic polymer or the like may be used as necessary. Examples thereof include CdSe / ZnS having a surface carboxy group (manufactured by Invitrogen), CdSe / ZnS having a surface amino group (manufactured by Invitrogen), and the like.

[Fluorescent substance-containing nanoparticles]
In the present embodiment, the fluorescent substance-encapsulating nanoparticles are those in which the fluorescent substance is dispersed inside the nanoparticles, whether the fluorescent substance and the nanoparticles themselves are chemically bonded or not. Good.
The material constituting the nanoparticles is not particularly limited, and examples thereof include polystyrene, polylactic acid, silica, and melamine.

  The fluorescent substance-containing nanoparticles used in the present embodiment can be produced by a known method. For example, silica nanoparticles encapsulating a fluorescent organic dye can be synthesized with reference to the synthesis of FITC-encapsulated silica particles described in Langmuir 8, Vol. 2921 (1992). Various fluorescent organic dye-containing silica nanoparticles can be synthesized by using a desired fluorescent organic dye instead of FITC.

  Silica nanoparticles encapsulating quantum dots can be synthesized with reference to the synthesis of CdTe-encapsulated silica nanoparticles described in New Journal of Chemistry, Vol. 33, p. 561 (2009).

  Polystyrene nanoparticles encapsulating a fluorescent organic dye may be copolymerized using an organic dye having a polymerizable functional group described in US Pat. No. 4,326,008 (1982) or polystyrene described in US Pat. No. 5,326,692 (1992). It can be produced using a method of impregnating nanoparticles with a fluorescent organic dye.

  The polymer nanoparticles encapsulating the quantum dots can be prepared by using the method of impregnating the quantum nanoparticles into polystyrene nanoparticles described in Nature Biotechnology, Vol. 19, page 631 (2001).

[Bonding of biological substance recognition sites and fluorescent substance-containing nanoparticles]
In this embodiment, a case where fluorescent substance-containing nanoparticles are used as a fluorescent substance and a fluorescent substance-encapsulating nanoparticle and a biological substance recognition site are directly bonded in advance will be described as an example. The biological material recognition site according to the present embodiment is a site that specifically binds and / or reacts with the target biological material. The target biological substance is not particularly limited as long as a substance that specifically binds to the target biological substance exists, but typically, protein (peptide), nucleic acid (oligonucleotide, polynucleotide), antibody, etc. Is mentioned. Accordingly, substances that bind to the target biological substance include antibodies that recognize the protein as an antigen, other proteins that specifically bind to the protein, and nucleic acids having a base sequence that hybridizes to the nucleic acid. Is mentioned. Specifically, an anti-HER2 antibody that specifically binds to HER2, which is a protein present on the cell surface, an anti-ER antibody that specifically binds to an estrogen receptor (ER) present in the cell nucleus, and actin that forms a cytoskeleton And an anti-actin antibody that specifically binds to. Among them, those in which anti-HER2 antibody and anti-ER antibody are bound to fluorescent substance-encapsulating nanoparticles can be used for selection of breast cancer medication, and are preferable.

  The mode of binding between the biological substance recognition site and the fluorescent substance-encapsulating nanoparticles is not particularly limited, and examples thereof include covalent bonding, ionic bonding, hydrogen bonding, coordination bonding, physical adsorption, and chemical adsorption. A bond having a strong bonding force such as a covalent bond is preferred from the viewpoint of bond stability.

  Moreover, there may be an organic molecule that connects between the biological substance recognition site and the fluorescent substance-encapsulating nanoparticles. For example, in order to suppress non-specific adsorption with a biological substance, a polyethylene glycol chain can be used, and SM (PEG) 12 manufactured by Thermo Scientific can be used.

  When the biological substance recognition site is bound to the fluorescent substance-encapsulating silica nanoparticles, the same procedure can be applied regardless of whether the fluorescent substance is a fluorescent organic dye or a quantum dot. For example, a silane coupling agent that is a compound widely used for bonding an inorganic substance and an organic substance can be used. This silane coupling agent is a compound having an alkoxysilyl group that gives a silanol group by hydrolysis at one end of the molecule and a functional group such as a carboxyl group, an amino group, an epoxy group, an aldehyde group at the other end, Bonding with an inorganic substance through an oxygen atom of the silanol group. Specifically, mercaptopropyltriethoxysilane, glycidoxypropyltriethoxysilane, aminopropyltriethoxysilane, a silane coupling agent having a polyethylene glycol chain (for example, PEG-silane no.SIM6492.7 manufactured by Gelest), etc. Is mentioned. When using a silane coupling agent, you may use 2 or more types together.

  A known procedure can be used for the reaction procedure of the fluorescent organic dye-encapsulated silica nanoparticles and the silane coupling agent. For example, the obtained fluorescent organic dye-encapsulated silica nanoparticles are dispersed in pure water, aminopropyltriethoxysilane is added, and the mixture is reacted at room temperature for 12 hours. After completion of the reaction, fluorescent organic dye-encapsulated silica nanoparticles whose surface is modified with an aminopropyl group can be obtained by centrifugation or filtration. Subsequently, by reacting an amino group with a carboxyl group in the antibody, the antibody can be bound to the fluorescent organic dye-encapsulated silica nanoparticles via an amide bond. If necessary, a condensing agent such as EDC (1-Ethyl-3- [3-Dimethylaminopropyl] carbodiimide Hydrochloride: manufactured by Pierce (registered trademark)) can also be used.

  If necessary, a linker compound having a site capable of directly binding to a fluorescent organic dye-encapsulated silica nanoparticle modified with an organic molecule and a site capable of binding to a molecular target substance can be used. As a specific example, sulfo-SMCC (Sulfosuccinimidyl 4 [N-maleimidomethyl] -cyclohexane-1-carboxylate: manufactured by Pierce) having both a site that selectively reacts with an amino group and a site that reacts selectively with a mercapto group When used, by binding the amino group of the fluorescent organic dye-encapsulated silica nanoparticles modified with aminopropyltriethoxysilane and the mercapto group in the antibody, antibody-bound fluorescent organic dye-encapsulated silica nanoparticles can be produced.

When the biological substance recognition site is bound to the fluorescent substance-encapsulated polystyrene nanoparticles, the same procedure can be applied regardless of whether the fluorescent substance is a fluorescent organic dye or a quantum dot. That is, by impregnating polystyrene nanoparticles having functional groups such as amino groups with fluorescent organic dyes and quantum dots, it is possible to obtain polystyrene nanoparticles encapsulating functional groups with fluorescent substances, and thereafter using EDC or sulfo-SMCC. Thus, antibody-bound fluorescent substance-encapsulated polystyrene nanoparticles can be produced.
When the biological substance recognition site is bound to the fluorescent substance-encapsulated melamine nanoparticles, the same procedure as that for the fluorescent substance-encapsulated silica nanoparticles can be applied. In order to further improve the reactivity, the number of surface amino groups may be increased by reacting melamine nanoparticles with a polyfunctional amine compound in advance.

  Antibodies that recognize specific antigens include M. actin, MS actin, SM actin, ACTH, Alk-1, α1-antichymotrypsin, α1-antitrypsin, AFP, bcl-2, bcl-6, β-catenin, BCA 225, CA19-9, CA125, calcitonin, calretinin, CD1a, CD3, CD4, CD5, CD8, CD10, CD15, CD20, CD21, CD23, CD30, CD31, CD34, CD43, CD45, CD45R, CD56, CD57, CD61, CD68, CD79a, "CD99, MIC2", CD138, chromogranin, c-KIT, c-MET, collagen type IV, Cox-2, cyclin D1, keratin, cytokeratin (high molecular weight), pankeratin, pankeratin, cytokeratin 5/6, cytokeratin 7, cytokeratin 8, cytokeratin 8/18, cytokeratin 14, cytokeratin 19, cytokeratin 20, CMV, E-cadherin, EGFR, ER, EMA, EBV, factor VIII related antigen, Fassin, FSH, Galectin-3 Gastrin, GFAP, glucagon, glycophorin A, granzyme B, hCG, hGH, Helicobacter pylori, HBc antigen, HBs antigen, hepatocyte specific antigen, HER2, HSV-I, HSV-II, HHV-8, IgA, IgG, IgM, IGF-1R, Inhibin, Insulin, Kappa L chain, Ki67, Lambda L chain, LH, Lysozyme, Macrophage, Melan A, MLH-1, MSH-2, Myeloperoxidase, Myogenin, Myoglobin, Myosin, Neurofilament, NSE, p27 (Kip1), p53, p53, P63, PAX 5, PLAP, Pneumocystis carini, podoplanin (D2-40), PGR, prolactin, PSA, prostate acid phosphatase, Renal Cell Carcinoma, S100, somatostatin, spectrin, synaptophysin TAG-72, TdT, thyroglobulin, TSH, TTF-1, TRAcP, tryptase, villin, vimentin, WT1, Zap-70 It is.

  The fluorescent substance or the fluorescent substance-encapsulating nanoparticles and the biological substance recognition site may be combined in advance as described above as a staining reagent, but in the staining step as in the known indirect method in immunostaining. It may be indirectly coupled. Specifically, for example, a tissue specimen is reacted with a primary antibody having a specific protein as an antigen, and then a fluorescent substance or fluorescent substance-containing nanoparticles are bound to a secondary antibody having the primary antibody as an antigen. The reagent may be further reacted to perform staining. In addition, for example, after reacting a tissue specimen with a primary antibody having a specific protein as an antigen and a biotinylated secondary antibody having the primary antibody as an antigen, a fluorescent substance or fluorescent substance-encapsulated substance modified with streptavidin The nanoparticle may be further reacted as a staining reagent, and staining may be performed by utilizing a specific binding between streptavidin and biotin to form a complex.

[Dyeing method]
Hereinafter, a method for staining a tissue specimen will be described, but the present invention is not limited to a tissue specimen, and can also be applied to a specimen such as a cell fixed on a substrate.
Moreover, the preparation method of the tissue specimen which can apply the dyeing | staining method demonstrated below is not specifically limited, What was produced by the well-known method can be used.

1) Deparaffinization process The tissue specimen is immersed in a container containing xylene to remove paraffin. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, xylene may be exchanged during the immersion.
Next, the tissue specimen is immersed in a container containing ethanol to remove xylene. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. Further, if necessary, ethanol may be exchanged during the immersion.
Next, the tissue specimen is immersed in a container containing water to remove ethanol. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. Moreover, you may exchange water in the middle of immersion as needed.

2) Activation process The activation process of the target biological substance is performed according to a known method. The activation conditions are not particularly defined, but as the activation liquid, 0.01 M citrate buffer (pH 6.0), 1 mM EDTA solution (pH 8.0), 5% urea, 0.1 M Tris-HCl buffer, etc. Can be used. As the heating device, an autoclave, a microwave, a pressure cooker, a water bath, or the like can be used. The temperature is not particularly limited, but can be performed at room temperature. The temperature can be 50-130 ° C. and the time can be 5-30 minutes.
Next, the tissue specimen after the activation treatment is immersed in a container containing PBS (Phosphate Buffered Saline) and washed. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, the PBS may be replaced during the immersion.

3) Staining using fluorescent substance-encapsulating nanoparticles combined with biological substance recognition sites Place PBS dispersion of fluorescent substance-encapsulating nanoparticles combined with biological substance recognition sites on a tissue specimen and react with the target biological substance . By changing the biological material recognition site to be combined with the fluorescent substance-containing nanoparticles, staining corresponding to various biological materials becomes possible. When using fluorescent substance-encapsulated nanoparticles to which several kinds of biological substance recognition sites are bound, each fluorescent substance-encapsulated nanoparticle PBS dispersion may be mixed in advance or separately placed on a tissue specimen separately. Also good.
The temperature is not particularly limited, but can be performed at room temperature. The reaction time is preferably 30 minutes or more and 24 hours or less.
It is preferable to drop a known blocking agent such as BSA-containing PBS before staining with fluorescent substance-encapsulating nanoparticles.
Next, the stained tissue specimen is immersed in a container containing PBS to remove unreacted fluorescent substance-containing nanoparticles. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, the PBS may be replaced during the immersion. Place the cover glass on the tissue specimen and enclose it. A commercially available encapsulant may be used as necessary.
In addition, when dyeing | staining using a HE dyeing reagent, HE dyeing is performed before enclosure with a cover glass.

(Fluorescence image acquisition)
A fluorescence image (first image) and a bright-field image (second image) are acquired from the stained tissue specimen using the microscope image acquisition device 1A. In the microscope image acquisition apparatus 1A, an excitation light source and a fluorescence detection optical filter corresponding to the absorption maximum wavelength and fluorescence wavelength of the fluorescent material used for the staining reagent are selected.

<Operation of Diagnosis Support Information Generation System 100 (Including Image Processing Method)>
Hereinafter, an operation of acquiring and analyzing the above-described fluorescence image and bright field image in the diagnosis support information generation system 100 will be described with specific embodiments. Here, staining is performed using a staining reagent containing fluorescent substance-encapsulating nanoparticles bound to a biological substance recognition site that recognizes a specific protein (here, HER2 protein in breast cancer tissue, hereinafter referred to as a specific protein). However, the present invention is not limited to this example.

First, the operator uses two types of staining reagents, a HE staining reagent and a staining reagent using fluorescent substance-encapsulated nanoparticles bound with a biological substance recognition site that recognizes a specific protein as a fluorescent labeling material. Stain.
Thereafter, in the microscope image acquisition apparatus 1A, a bright field image and a fluorescence image are acquired by the procedures (a1) to (a5). In the present embodiment, the microscope image acquisition apparatus 1A is an upright microscope that observes a specimen from above.
(A1) The operator places the tissue specimen stained with the HE staining reagent and the staining reagent containing the fluorescent substance-containing nanoparticles on the slide, and places the slide on the slide fixing stage of the microscope image acquisition apparatus 1A.
(A2) Set to the bright field unit, adjust the imaging magnification and focus, and place the observation target area on the tissue in the field of view.
(A3) Shooting is performed by the imaging unit to generate bright field image data, and the image data is transmitted to the image processing apparatus 2A.
(A4) Change the unit to a fluorescent unit.
(A5) Shooting is performed by the imaging means without changing the field of view and the shooting magnification to generate image data of a fluorescent image, and the image data is transmitted to the image processing apparatus 2A.

In the image processing apparatus 2A, image analysis processing is executed based on the bright field image and the fluorescence image.
FIG. 5 shows a flowchart of image analysis processing in the image processing apparatus 2A. The image analysis processing shown in FIG. 5 is executed in cooperation with the control unit 21 and the program stored in the storage unit 25.
When the user operates the operation unit 22 to start image analysis processing, the setting program stored in the storage unit 25 is executed by the control unit 21.

First, when a bright field image is input from the microscope image acquisition apparatus 1A through the communication I / F 24 (step S1: second input step), an analysis region is extracted in step S2 and analysis region information is calculated ( Analysis area information calculation step).
The analysis region is a predetermined region in the tissue specimen for performing diagnosis by the diagnosis support information generation method of the present invention, and the extraction method is arbitrary. For example, when extracting an analysis region based on the bright field image input in step S1, the user selects and extracts an arbitrary region in the bright field image using the operation unit 22 (for example, a mouse operation). Alternatively, a program for automatically extracting a specific region in the tissue from the brightness and color of the bright field image may be executed. The analysis region may be extracted based on the fluorescence image input in step S3 described later. Also in this case, the user may select and extract an arbitrary area in the fluorescent image through the operation of the operation unit 22 (for example, mouse operation), or may be based on the luminance or color of the fluorescent image. A program for automatically extracting a specific area in the organization may be executed.
Specifically, for example, the control unit 21 extracts a region where cells are densely packed in the tissue as an analysis region. The number of analysis regions extracted in one image analysis process is arbitrary.
The analysis region information is arbitrary as long as it is a value uniquely determined from the analysis region, and examples thereof include the number, area, circumference, and the like of the analysis region. The calculated analysis area information may be one type or a plurality of types, and is calculated by any known method.

On the other hand, in steps S3 to S6, information on the bright spot region derived from the fluorescence of the fluorescent substance-containing nanoparticles (hereinafter simply referred to as “fluorescent particles”) is extracted from the fluorescent image.
First, when a fluorescence image as shown in FIG. 6A is input from the microscope image acquisition apparatus 1A through the communication I / F 24 (step S3: first input step), color components are extracted in accordance with the wavelength of the fluorescent bright spot. Performed (step S4). For example, when the fluorescent wavelength emitted by the fluorescent particles is 550 nm, only the fluorescent bright spot having the wavelength component is extracted as a fluorescent image.

  Next, threshold processing is performed on the extracted fluorescent image to generate a binarized image, and a bright spot region is extracted (step S5). Note that noise removal processing such as cell autofluorescence and other unnecessary signal components may be performed before the threshold processing, and a low-pass filter such as a Gaussian filter or a high-pass filter such as a second derivative is preferably used. FIG. 6B shows an example of an image from which the bright spot area is extracted.

  Next, bright spot area information is extracted from each bright spot area (step S6: bright spot area information extracting step). The bright spot area information is arbitrary as long as it is a value uniquely determined from the bright spot area, and examples thereof include the area of the bright spot area, the perimeter, distribution variation, and the number of fluorescent particles in the bright spot area. The extracted bright spot area information may be one type or a plurality of types.

  After the processing of step S2 and step S6, the presence / absence of a bright spot score calculation step in the image analysis process is determined (step S7). When the bright spot score calculating step is executed (step S7: Yes), the process proceeds to step S8, and the bright spot score is calculated from the bright spot area information calculated in step S6. The bright spot score is a result of analyzing the expression of a specific protein for each bright spot region, and is evaluated in, for example, 10 levels of 1+ to 10+. The calculated bright spot score is stored in the storage unit 25. Note that the evaluation of the bright spot score is not limited to 10 levels, and can be evaluated at any number of levels depending on the type of specimen and the type of specific protein.

FIG. 7 is a diagram schematically illustrating an example in which a bright spot score is calculated based on the number and area of fluorescent particles per one bright spot region. The size of the ellipse in the figure indicates the area of the bright spot region, and the peak shown on each ellipse indicates the number of fluorescent particles. 1+, 2+, 3+ indicate the calculated bright spot scores, and the larger the numerical value, the higher the malignancy of the cancer.
In general, the higher the number of fluorescent particles, the higher the malignancy of the cancer and the higher the bright spot score is calculated. However, if the number of fluorescent particles is the same, the fluorescent particles are more widely distributed. Since the degree of malignancy is high, a higher bright spot score is calculated as the area of the bright spot region is larger. Similarly, in general, the larger the area of the bright spot region, the higher the malignancy of the cancer and the higher the bright spot score is calculated, but when the area of the bright spot region is the same, the number of fluorescent particles is The higher the malignancy, the higher the bright spot score.

  For example, in FIG. 7, the bright spot score of the bright spot region having a small area and a small number of fluorescent particles as shown in the lower left is calculated as 1+, and a bright spot having a small area but a large number of fluorescent particles is shown in the lower right. As shown in the upper left area, the bright spot score of the bright spot area having a large area but a small number of fluorescent particles is calculated as 2+. Then, as shown in the upper right, the bright spot score of the bright spot region having a large area and a large number of fluorescent particles is calculated as 3+.

  Note that the combination of the bright spot area information used for calculation of the bright spot score and the value of the bright spot score corresponding to the combination are appropriately changed according to the type of biological material or specimen.

  In step S7, when the bright spot score calculation step is not executed (step S7: No), the process proceeds to step S9, and the bright spot area information is stored in the storage unit 25.

  Next, the presence / absence of a cell score calculation step is determined (step S9). When the cell score calculation step is executed (step S9: Yes), the process proceeds to step S10, and first, a specific area (cell area) of the cell is extracted. Although any part such as cell nucleus or cytoplasm can be extracted as a cell region, it is preferable to extract a region where a biological substance to be quantified is specifically expressed. In this embodiment, a specific protein is expressed particularly in a cell membrane. Therefore, it is preferable to extract the entire cell as a cell region.

FIG. 8 shows an example of a detailed flow of processing for extracting a cell region in step S10. First, a bright field image is converted into a monochrome image (step S101). FIG. 9A shows an example of a bright field image subjected to monochrome conversion.
Next, threshold processing is performed on the monochrome image using a predetermined threshold, and the value of each pixel is binarized (step S102).

  Next, noise processing is performed (step S103). Specifically, the noise process can be performed by performing a closing process on the binary image. The closing process is a process in which the contraction process is performed the same number of times after the expansion process is performed. The expansion process is a process of replacing a target pixel with white when at least one pixel in the range of n × n pixels (n is an integer of 2 or more) from the target pixel is white. The contraction process is a process of replacing a target pixel with black when at least one pixel in the range of n × n pixels from the target pixel contains black. By the closing process, a small area such as noise can be removed. FIG. 9B shows an example of an image after noise processing. As shown in FIG. 9B, after noise processing, an image from which cell regions have been extracted is generated.

  Next, a labeling process is performed on the image after the noise process, and a label is given to each of the extracted cell regions (step S104). The labeling process is a process for identifying an object in an image by assigning the same label (number) to connected pixels. By the labeling process, each cell can be identified from the image after the noise process and a label can be applied.

  Next, in step S11, it is determined whether a bright spot score has already been calculated. When the bright spot score has been calculated (step S11: Yes), the process proceeds to step S12, and the bright spot score information calculated in step S8 and the cell region extracted in step S10 are calculated. A cell score is calculated based on the information (cell information). The cell score is a result of analyzing the expression of a specific protein for each cell region, and is evaluated, for example, in 10 levels of 1+ to 10+. The calculated cell score is stored in the storage unit 25. Note that the evaluation of the cell score is not limited to 10 levels, and can be evaluated at any number of levels depending on the type of specimen and the type of specific protein.

In the step of calculating cell information from the cell region, for example, the number, area, circumference, etc. of the cell region are calculated.
The information on the bright spot score is arbitrary as long as it is a value uniquely determined from the value of the bright spot score in the cell area. For example, the average value of the bright spot score existing in one cell area, a predetermined bright spot The number, distribution, ratio, density, and the like of the bright spot area having a score are given. The bright spot score information may be one or plural, and the extraction method is arbitrary.

FIG. 10 is a diagram schematically illustrating an example in which a cell score is calculated based on an average value of bright spot scores existing in one cell region and a distribution of bright spot regions. In the figure, the bright spot score of a bright spot indicated by a white circle is 1+, the bright spot score of a bright spot indicated by a gray circle is 2+, and the bright spot score of a bright spot indicated by a black circle is 3+. is there. 1+, 2+, and 3+ indicate calculated cell scores, and the larger the value, the higher the malignancy of the cancer.
In general, the higher the average value of the bright spot score, the higher the malignancy of the cancer and the higher the cell score is calculated, but if the average value of the bright spot score is the same, the bright spot area is the whole cell The higher the malignancy is, the higher the cell score is calculated. Similarly, in general, the higher the distribution of the bright spot region, the higher the malignancy of the cancer and the higher the cell score is calculated, but when the distribution of the bright spot region is the same, the bright spot score The higher the average value, the higher the malignancy and the higher the cell score is calculated.

  For example, in FIG. 10, the cell score of a cell having a narrow luminescent spot distribution range and an average luminescent spot score of 2+ is calculated as 1+ as shown in the lower left, and the bright spot distribution is shown in the lower right. The cell score of a cell with a narrow range but an average bright spot score of 3+ or a cell with a wide distribution of bright spots but an average bright spot score of 1+ as shown in the upper left is calculated as 2+ Is done. Then, as shown in the upper right, the cell score of a cell having a wide distribution of bright spots and an average bright spot score of 2+ is calculated as 3+.

Note that the combination of the bright spot score and the cell region information used for calculating the cell score and the value of the cell score corresponding to the combination are appropriately changed according to the type of biological material or specimen.
In FIG. 10, the cell score may be calculated from the average value of the bright spot score per area or perimeter of the cell area and the spread of the bright spot area distribution.

If it is determined in step S11 that the bright spot score has not been calculated (step S11: No), the process proceeds to step S13, and the bright spot area information extracted in step S6 and the bright spot area information extracted in step S10. The cell score is calculated from the information of the cell area.
The cell area information is, for example, the number, area, circumference, etc. of the cell area.

FIG. 11 is a diagram schematically illustrating an example in which a cell score is calculated based on the number of fluorescent particles present in one cell region and the area of the bright spot region. The size of the ellipse in the figure indicates the area of the bright spot region, and the peak shown on each ellipse indicates the number of fluorescent particles. 1+, 2+, and 3+ indicate calculated cell scores, and the larger the value, the higher the malignancy of the cancer.
In general, the higher the number of fluorescent particles, the higher the malignancy of the cancer and the higher the cell score is calculated. However, when the number of fluorescent particles is the same, it is more malignant that the fluorescent particles are widely distributed. Since the degree is high, a larger cell score is calculated as the area of the bright spot region is larger. Similarly, in general, the larger the area of the bright spot region, the higher the malignancy of the cancer and the higher the cell score is calculated, but when the area of the bright spot region is the same, the number of fluorescent particles is large. Since the malignancy is higher, a higher cell score is calculated.

For example, in FIG. 11, the bright spot score of the bright spot region with a small area and a small number of fluorescent particles as shown in the lower left is calculated as 1+, and a bright spot with a small area but a large number of fluorescent particles as shown in the lower right. As shown in the upper left area, the bright spot score of the bright spot area having a large area but a small number of fluorescent particles is calculated as 2+. As shown in the upper right, the bright spot score of the bright spot region having a large area and a large number of fluorescent particles is calculated as 3+.

Note that the combination of bright spot region information used for calculation of the cell score and the value of the cell score corresponding to the combination are appropriately changed according to the type of biological material or specimen.
In FIG. 11, the cell score may be calculated from the area of the cell region or the number of fluorescent particles per circumference and the area of the bright spot region.

  When the cell score is calculated in step S12 or step S13, the process proceeds to step S15, and the image score is calculated from the cell score information calculated in step S12 or step S13 and the analysis region information extracted in step S2. Is calculated (image score calculation step). The image score is a result of analyzing the expression of a specific protein in the analysis region, and is evaluated in, for example, 10 levels of 1+ to 10+. The calculated image score is stored in the storage unit 25. Note that the evaluation of the image score is not limited to 10 levels, and can be evaluated at an arbitrary number of levels depending on the type of specimen and the type of specific protein.

The information on the analysis region is, for example, the number, area, circumference, etc. of the analysis region.
The cell score information is arbitrary as long as it is a value uniquely determined from the value of the cell score in the analysis region. For example, the average value of the cell scores existing in one analysis region or the cell score having a predetermined cell score Number, distribution, ratio, density, etc. The cell score information may be one or more, and the extraction method is arbitrary.

FIG. 12 is a diagram schematically illustrating an example in which an image score is calculated based on an average value of cell scores and a cell distribution existing in one analysis region. In the figure, the cell score of cells indicated by white circles is 1+, the cell score of cells indicated by gray circles is 2+, and the cell score of cells indicated by black circles is 3+. 1+, 2+, 3+ indicate the calculated image scores, and the higher the numerical value, the higher the malignancy of the cancer.
In general, the higher the average value of the cell score, the higher the malignancy of the cancer and the higher the image score is calculated, but when the average value of the cell score is the same, the cells are distributed over the entire image and distributed The higher the malignancy is, the higher the image score is calculated. Similarly, in general, the higher the cell distribution, the higher the malignancy of the cancer and the higher the image score is calculated, but when the cell distribution is the same, the average value of the cell score is large The malignancy is higher and a higher image score is calculated.

  For example, in FIG. 12, the image score of an image in which the distribution range of the cells in the analysis region is narrow as shown in the lower left and the average value of the cell score is 2+ is calculated as 1+, and as shown in the lower right, An image with a narrow distribution range but an average cell score of 3+, or an image with a wide cell distribution but an average cell score of 1+ as shown in the upper left is calculated as 2+ . Then, as shown in the upper right, the image score of an image with a wide cell distribution and an average cell score of 2+ is calculated as 3+.

Note that the combination of the cell score and the analysis region information used for the calculation of the image score and the value of the image score corresponding to the combination are appropriately changed according to the type of biological material or specimen.
In FIG. 12, the image score may be calculated from the average value of the cell score per area or circumference of the analysis region and the spread of the cell distribution.

In step S9, when the cell score calculation step is not executed (step S9: No), the process proceeds to step S16, and it is determined whether or not the bright spot score has already been calculated. When the bright spot score is calculated (step S16: Yes), the process proceeds to step S17, and from the bright spot score information calculated in step S8 and the analysis area information extracted in step S2, An image score is calculated. The calculated image score is stored in the storage unit 25.

The information on the analysis region is, for example, the number, area, circumference, etc. of the analysis region.
The information on the bright spot score is arbitrary as long as it is a value uniquely determined from the value of the bright spot score in the cell area. For example, the average value of the bright spot score existing in one cell area, or a predetermined bright spot score The number, distribution, ratio, density, etc. of bright spot regions having The bright spot score information may be one or plural, and the extraction method is arbitrary.

FIG. 13 is a diagram schematically illustrating an example in which an image score is calculated based on the average value of bright spot scores existing in one analysis region and the spread of the distribution of bright spot regions. In the figure, the bright spot score of a bright spot indicated by a white circle is 1+, the bright spot score of a bright spot indicated by a gray circle is 2+, and the bright spot score of a bright spot indicated by a black circle is 3+. is there. 1+, 2+, 3+ indicate the calculated image scores, and the higher the numerical value, the higher the malignancy of the cancer.
In general, the higher the average value of the bright spot score, the higher the malignancy of the cancer and the higher the image score is calculated, but when the average value of the bright spot score is the same, the bright spot region is the entire image. If the image is spread and distributed, the malignancy is higher, and a higher image score is calculated. Similarly, in general, the greater the spread of the bright spot region distribution, the higher the malignancy of the cancer and the higher the image score is calculated. The higher the score score, the higher the malignancy, and a higher image score is calculated.

  For example, in FIG. 13, as shown in the lower left, the image score of an image in which the distribution range of the bright spots in the analysis region is narrow and the average value of the bright spot scores is 2+ is calculated as 1+, and as shown in the lower right. An image score of an image having a narrow distribution range of bright spots but an average value of bright spot scores of 3+, or an image having a wide distribution of bright spots but an average value of bright spot scores of 1+ as shown in the upper left 2+. Then, as shown in the upper right, the image score of an image in which the distribution of bright spots is wide and the average value of bright spot scores is 2+ is calculated as 3+.

Note that the combination of the bright spot score and the analysis region information used for calculating the image score and the value of the image score corresponding to the combination are appropriately changed according to the type of biological material or specimen.
In FIG. 13, the image score may be calculated from the average value of the bright spot score per area of the analysis region or perimeter and the spread of the bright spot region distribution.

When it is determined in step S16 that the bright spot score has not been calculated (step S16: No), the process proceeds to step S18, and the bright spot area information extracted in step S6 and the bright spot area information extracted in step S2. An image score is calculated from the information of the analyzed area.
The information on the analysis region is, for example, the number, area, circumference, etc. of the analysis region.

FIG. 14 is a diagram schematically illustrating an example in which an image score is calculated based on the number of fluorescent particles present in one cell region and the area of the bright spot region. The size of the ellipse in the figure indicates the area of the bright spot region, and the peak shown on each ellipse indicates the number of fluorescent particles. 1+, 2+, 3+ indicate the calculated image scores, and the higher the numerical value, the higher the malignancy of the cancer.
In general, the higher the number of fluorescent particles, the higher the malignancy of the cancer and the higher the image score is calculated. However, if the number of fluorescent particles is the same, it is more malignant that the fluorescent particles are widely distributed. Since the degree is high, a higher image score is calculated as the area of the bright spot region is larger. Similarly, generally, the larger the area of the bright spot region, the higher the malignancy of the cancer and the higher the image score is calculated. However, when the area of the bright spot region is the same, the number of fluorescent particles is large. Since the malignancy is higher, a higher image score is calculated.

For example, in FIG. 14, the image score of an image in which the area of the bright spot area in the analysis area is small and the number of fluorescent particles is small as shown in the lower left is calculated as 1+. The image score of an image with a small number of fluorescent particles but a large number of fluorescent particles or an image with a large area of the bright spot region but a small number of fluorescent particles as shown in the upper left is calculated as 2+. Then, as shown in the upper right, the image score of an image having a large bright spot area and a large number of fluorescent particles is calculated as 3+.

Note that the combination of the bright spot area information and the analysis area information used for the calculation of the image score and the value of the image score corresponding to the combination are appropriately changed according to the type of biological material or specimen.
In FIG. 11, the image score may be calculated from the area of the analysis region or the number of fluorescent particles per circumference and the area of the bright spot region.

FIGS. 15A to 15J are schematic diagrams illustrating a specific example of a method for evaluating the distribution of bright spot areas when calculating the cell score from the bright spot area information in step S13.
15A and 15B are diagrams showing a method using the comprehensive area of the bright spot region. The comprehensive area is defined as, for example, the smallest rectangular area that covers all the bright spot regions included in one cell, as indicated by the one-dot chain line in FIGS. 15A and 15B. When the value obtained by dividing the global area of the bright spot area by the general area of the cell (not shown) is close to 1, the bright spot area is widely distributed in the cell as shown in FIG. It exists densely as shown in FIG. 15B.

  15C and 15D are diagrams showing a method of using the area of the area inside the line connecting the bright spot areas (the bright spot inner area). The method for calculating the inner area of the bright spot is arbitrary. For example, as shown by the hatched portion in FIG. 15C and FIG. 15D, two adjacent bright spot areas are connected by a straight line, and the area surrounded by the straight line is the bright spot. The inner area. When the value obtained by dividing the area inside the bright spot by the area of the cell is close to 1, the bright spot area is widely distributed in the cell as shown in FIG. 15C, and when close to 0, the area is dense as shown in FIG. 15D. Exist.

  FIG. 15E and FIG. 15F are diagrams showing a method using the total area of the bright spot regions. When the sum of the areas of the bright spot areas (shaded areas) contained in one cell is divided by the area of the cells and the normalized value is close to 1, the bright spot area is wide in the cell as shown in FIG. 15E. When distributed and close to 0, they exist densely as shown in FIG. 15F.

  FIG. 15G and FIG. 15H are diagrams showing a method using the sum of distances connecting all the bright spot regions with lines. The method of connecting the bright spot areas with lines is arbitrary, but, for example, one continuous line that passes through all the bright spot areas once is drawn so that the sum of the lengths of the lines becomes the shortest. For example, when the value obtained by dividing the length by the perimeter of the cell is large, the bright spot region is distributed over a wide range in the cell as shown in FIG. 15G, and when it is small, it is dense as shown in FIG. 15H. Existing.

  FIGS. 15I and 15J are diagrams illustrating a method using the variation in coordinates of the bright spot region. For example, when the standard deviations of the x coordinate position and the y coordinate position of the bright spot area are std_x and std_y, respectively, the value calculated by “sqrt (std_x2 + std_y2)” When the value normalized by dividing the area by the cell area is large, the bright spot region is distributed over a wide range in the cell as shown in FIG. 15I, and when it is small, it is densely present as shown in FIG. 15J.

In addition, in the method for evaluating the distribution of the bright spot regions described in FIGS. 15A to 15J, normalization is not necessarily required when the shapes and sizes of the cells are all similar, but in order to increase accuracy. Is preferably normalized.
15A to 15J not only evaluates the distribution of bright spot regions in step S12, but also evaluates the distribution of bright spot regions in step S8, step S13, and step S18, step S15. In the case of evaluating the cell distribution in step S17, the present invention can also be applied to the case of evaluating the bright spot region distribution in step S17.

  In an actual tissue specimen, cell scores of various values are mixed in one image, and various values of bright spot scores exist in each cell. An example of calculation results of various scores in such a case will be specifically described with reference to the schematic diagram of FIG.

In FIG. 16, when the cell score is calculated from the bright spot score, 1 and no. The bright spot score of the bright spot area included in the two cells is all 2+, and since the bright spot area is spread over the entire cell, the cell score is calculated as 2+. No. The bright spot score of the bright spot area included in the 6 cells is all 3+, and the bright spot area spreads over the entire cell, so the cell score is calculated as 3+. No. The bright spot score of the bright spot area included in cell No. 7 is all 1+, and the bright spot area spreads over the entire cell, so the cell score is calculated as 1+.
Here, for example, no. No. 1 cell, even when the bright spot score of one bright spot region is measured as 10+ due to noise, no. According to the distribution of 1 in the whole cell, 2+ is dominant, so the cell score is calculated as 2+, and the diagnosis result is not affected by noise.

No. In the cell of 3, the luminescent spot area having the 2+ and 3+ luminescent spot scores is mixed, but for example, the 3+ luminescent spot area has a larger number than the 2+ luminescent spot area and is distributed over a wide range. The cell score is calculated as 3+.
No. In the cell No. 5, the bright spot scores of 1+ and 2+ are mixed, but for example, the 1+ bright spot area has a larger number than the 2+ bright spot area and is distributed over a wide range. The cell score is calculated as 1+.
No. In the cell No. 4, the bright spot regions having the bright spot scores of 1+, 2+, and 3+ are mixed. For example, the cell score is calculated as 2+ from the average value of the bright spot scores.

  In FIG. 16, when the image score is calculated from the cell score using the entire image as the analysis region, for example, the image score is calculated as 2+ from the average value of the cell scores and the distribution of the cell score values.

  In FIG. 16, when the image score is calculated from the bright spot score using the entire image as the analysis region, for example, the image score is calculated as 2+ from the average value of the bright spot scores and the distribution of the bright spot score values. The

According to the embodiment of the present invention described above, the image score obtained by scoring the expression level of the biological material is evaluated as diagnostic information based on the bright spot region information and the analysis region information, and thus is not easily affected by noise. Stable and accurate quantification can be performed. In addition, if the image score is calculated based on the bright spot score or cell score calculated based on the bright spot area information and the analysis area information, it is less susceptible to noise, and stable and accurate quantification. It can be performed.
In addition, the description content in the said embodiment is a suitable example of this invention, and is not limited to this.

  In the above-described embodiment, the analysis region and the cell region are extracted by performing the processing of Step S2 and Step S10 of FIG. 5 on the bright field image. For example, the cytoplasm or the cell nucleus is stained with a fluorescent substance and input in Step S3. The analysis region and the cell region may be extracted by performing the processing in steps S2 and S10 in FIG. 5 on the same fluorescent image as the fluorescent image or a fluorescent image different from the fluorescent image input in step S3. .

Moreover, in the said embodiment, although only 1 type of specific protein was made into object, you may use 2 or more types of fluorescent particles from which a light emission wavelength mutually differs with respect to several specific protein. In such a case, each color component may be extracted using a filter work or the like in step S4, and the processing in steps S5 to S6 may be executed for each extracted color component (wavelength component). Depending on the type of lesion (cancer) to be diagnosed, if the biological material recognition site when acquiring a fluorescence image is different, the feature quantity that quantitatively indicates the expression level of the specific protein corresponding to the lesion type It can be provided to a doctor.
In addition, when dyeing | staining using a some fluorescent substance, the combination from which the excitation light and fluorescence wavelength of a fluorescent substance do not interfere mutually is selected.

In the above description, an example in which an HDD or a semiconductor nonvolatile memory is used as a computer-readable medium for the program according to the present invention is disclosed, but the present invention is not limited to this example. As another computer-readable medium, a portable recording medium such as a CD-ROM can be applied. Further, a carrier wave (carrier wave) is also applied as a medium for providing program data according to the present invention via a communication line.
In addition, the detailed configuration and detailed operation of each device constituting the diagnosis support information generating system 100 can be changed as appropriate without departing from the spirit of the invention.

  The present invention is characterized by scoring and evaluating the number and distribution of specific biological substances expressed in a specimen, and can be particularly suitably used for acquiring more accurate diagnostic information.

DESCRIPTION OF SYMBOLS 1A Microscope image acquisition apparatus 2A Image processing apparatus 3A Cable 21 Control part 22 Operation part 23 Display part 24 Communication I / F
25 storage unit 26 bus 100 diagnostic support information generation system

Claims (8)

In a diagnostic support information generation method for generating diagnostic support information using a specimen in which a specific biological material is stained with a staining reagent using a fluorescent substance,
A first input step of inputting a first image representing the expression of the biological material in the specimen as a fluorescent luminescent spot;
An analysis region information calculation step of extracting a predetermined region of the sample as an analysis region and calculating analysis region information including at least one of the number, area, and circumference of the analysis regions;
A bright spot region information extracting step of extracting a bright spot region indicating the fluorescent bright spot from the first image and extracting bright spot region information including information on the distribution of the bright spot region;
Based on the analysis area information and the bright spot area information, an image score calculation step of calculating an image score that is the diagnosis support information;
A diagnostic support information generating method characterized by comprising: The diagnostic support information generation method according to claim 1,
The staining reagent is a fluorescent particle in which a plurality of the fluorescent substances are integrated,
The bright spot area information includes information on the number of the fluorescent particles included in each of the bright spot areas. The diagnostic support information generation method according to claim 1 or 2,
A second input step of inputting a second image capable of extracting a predetermined region of cells in the specimen;
A cell information calculation step of extracting a predetermined region of cells as a cell region from the second image and calculating cell information including at least one of the number, area, and circumference of the cell region;
Have
The image score calculation step includes:
Based on the bright spot region information and the cell information, to calculate a cell score that has analyzed the expression of the biological material per cell region,
The diagnostic support information generation method, wherein the image score is calculated based on the analysis region information and the cell score. The diagnostic support information generation method according to claim 1 or 2,
The image score calculation step includes:
Based on the bright spot area information, calculate a bright spot score by analyzing the expression of the biological material per bright spot area,
A diagnostic support information generation method, wherein the image score is calculated based on the analysis area information and the bright spot score. The diagnostic support information generation method according to claim 1 or 2,
A second input step of inputting a second image capable of extracting a predetermined region of cells in the specimen;
A cell information calculation step of extracting a predetermined region of cells as a cell region from the second image and calculating cell information including at least one of the number, area, and circumference of the cell region;
Have
The image score calculation step includes:
Based on the bright spot area information, calculating a bright spot score by analyzing the expression of the biological material per bright spot area,
Based on the cell information and the bright spot score, calculate a cell score by analyzing the distribution of the bright spot score per cell area,
The diagnostic support information generation method, wherein the image score is calculated based on the analysis region information and the cell score. In an image processing apparatus that generates diagnosis support information using a specimen in which a specific biological substance is stained with a staining reagent using a fluorescent substance,
A first input means for inputting a first image representing the expression of the biological material in the specimen as a fluorescent luminescent spot;
An analysis region information calculating means for extracting a predetermined region of the sample as an analysis region and calculating analysis region information including at least one of the number, area, and circumference of the analysis region;
A bright spot area information extracting means for extracting a bright spot area indicating the fluorescent bright spot from the first image and extracting bright spot area information including information on the distribution of the bright spot area;
Based on the analysis area information and the bright spot area information, an image score calculation means for calculating an image score that is the diagnosis support information;
An image processing apparatus comprising: An image processing apparatus according to claim 6;
An image acquisition device for acquiring the first image used in the image processing device;
A diagnostic support information generation system comprising: A computer that generates diagnosis support information using a specimen in which a specific biological material is stained with a staining reagent using a fluorescent material,
A first input means for inputting a first image representing the expression of the biological material in the specimen as a fluorescent bright spot;
An analysis region information calculating means for extracting a predetermined region of the sample as an analysis region and calculating analysis region information including at least one of the number, area, and circumference of the analysis regions;
A bright spot area information extracting unit that extracts a bright spot area indicating the fluorescent bright spot from the first image and extracts bright spot area information including information on a distribution of the bright spot area;
An image score calculating means for calculating an image score which is the diagnosis support information based on the analysis area information and the bright spot area information;
Image processing program to function as

Images