JP2019153341A - Image processing method, image generation method, image processing device, and program - Google Patents

Abstract

To provide an image processing method, an image generation method, an image processing device, and a program that can create an image that facilitates visual confirmation of the expression of biological substances based on the emission of fluorescent substances in bright-field images.SOLUTION: An image processing method includes: an input step of inputting a bright field image representing the form of cells in a specimen and a fluorescent image labeled with particles having an average particle diameter of 30 to 800 nm and representing a specific substance present in the specimen as a fluorescent bright spot; a bright spot extraction step of extracting an area of the fluorescent bright spot in the fluorescent image; a composite image generation step of generating a composite image in which the bright field image and the fluorescent image before extraction of the fluorescent bright spot are superposed; and a composite image analysis step of identifying the position or amount of the specific substance in the specimen from the composite image.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an image processing method, an image generation method, an image processing apparatus, and a program for generating a pathological image from a bright-field image representing cell morphology and a fluorescence image representing the expression of a specific biological substance as a fluorescent bright spot.

  In pathological diagnosis, identifying a specific biological substance such as a protein that is overexpressed in a tissue section and its expression level can be very important information for determining a prognosis and determining a subsequent treatment plan. Therefore, in recent years, in order to improve the accuracy of quantifying the expression level of biological materials, many quantitative methods have been developed for staining specific biological materials using fluorescent materials and analyzing the fluorescence of fluorescent materials in fluorescent images.

On the other hand, there is a demand from pathologists and researchers who want to increase the quantitative accuracy of the expression level and at the same time visually check the expression position of a specific biological substance on an image representing the cell morphology.
In pathological diagnosis, a bright-field image obtained by imaging a specimen that has been subjected to tissue staining such as hematoxylin staining or hematoxylin-eosin staining is generally used as an image representing cell morphology. When it is desired to visually confirm the expression position and expression level of a specific protein on a bright field image stained in this way, an image obtained by simply superimposing a bright field image and a fluorescent image (dark field image) Due to low brightness and contrast, there is a problem that it is difficult to determine the expression position of the biological material. Further, when a region close to the hue of the fluorescent bright spot of the fluorescent image exists in the bright field image, it is further difficult to visually determine the expression state of a specific biological substance.

  For example, Patent Document 1 generates a thumbnail image in which a fluorescent image (dark field image) obtained by photographing an entire biological sample labeled with a phosphor and a bright field image are superimposed, and the biological sample is observed from the thumbnail image. It describes that an area is set.

JP 2011-70140 A

  However, in Patent Document 1, the bright field image is used not for obtaining cell morphology but for obtaining information such as a label of a biological sample. Therefore, in the invention of Patent Document 1, a bright field image representing the morphology of cells subjected to tissue staining and a dark field image representing the expression of a specific protein by fluorescence emission are combined to determine the expression status of a specific biological substance. There is a problem that an image that is easy to visually recognize cannot be obtained.

  A main object of the present invention is to provide an image processing method, an image generation method, an image processing apparatus, and a program capable of generating an image that can easily visually confirm the expression of a biological material based on light emission of a fluorescent material in a bright-field image. There is to do.

In order to solve the above-described problem, according to the invention described in item 1 of the present invention,
An input step for inputting a bright-field image representing the morphology of cells in the specimen, and a fluorescence image labeled with particles having an average particle diameter of 30 to 800 nm and representing a specific substance present in the specimen as a fluorescent luminescent spot;
A bright spot extracting step of extracting the fluorescent spot area in the fluorescent image;
A composite image generating step for generating a composite image in which the bright field image and the fluorescent image before extraction of the fluorescent bright spot are superimposed;
From the composite image, a composite image analysis step for specifying the position or amount of the specific substance in the specimen;
An image processing method is provided.

According to the invention described in item 2 of the present invention,
It has an image adjustment process for performing at least one image processing among hue conversion and contrast adjustment for the bright field image, and hue conversion, contrast adjustment and transmittance adjustment for the fluorescent image. An image processing method according to claim 1 is provided.

According to the invention described in item 3 of the present invention,
The image processing method according to claim 2, wherein, in the image adjustment step, a hue of the fluorescent bright spot is converted with respect to the fluorescent image after the fluorescent bright spot is extracted.

According to the invention described in item 4 of the present invention,
4. The image processing method according to claim 2, wherein, in the image adjustment step, contrast of the fluorescent bright spot is adjusted with respect to the fluorescent image after extraction of the fluorescent bright spot. 5. .

According to the invention described in item 5 of the present invention,
5. The image according to claim 2, wherein, in the image adjustment step, a transmittance of the fluorescent luminescent spot is adjusted with respect to the fluorescent image after extraction of the fluorescent luminescent spot. 6. A processing method is provided.

According to the invention described in item 6 of the present invention,
In the image adjustment step, based on color information of the bright field image and the fluorescent image, among the fluorescence image after the extraction of the fluorescent luminescent spot, hue conversion, contrast adjustment, transmittance adjustment The image processing method according to claim 2, wherein at least one image processing is performed.

According to the invention described in item 7 of the present invention,
A calculation step of calculating coordinates of the fluorescent luminescent spot;
A coordinate image generation step of generating a coordinate image indicating the calculated coordinates,
The image processing method according to any one of claims 1 to 6, wherein, in the composite image generation step, a composite image is generated by superimposing the bright field image, the fluorescent image, and the coordinate image. Provided.

According to the invention described in item 8 of the present invention,
An input step for inputting a bright-field image representing the morphology of cells in the specimen, and a fluorescence image labeled with particles having an average particle diameter of 30 to 800 nm and representing a specific substance present in the specimen as a fluorescent luminescent spot;
A bright spot extracting step of extracting the fluorescent spot area in the fluorescent image;
A composite image generating step for generating a composite image in which the bright field image and the fluorescent image before extraction of the fluorescent bright spot are superimposed;
From the composite image, a composite image analysis step for specifying the position or amount of the specific substance in the specimen;
An image generating method is provided.

According to the invention described in item 9 of the present invention,
An input means for inputting a bright-field image representing the morphology of cells in the specimen, and a fluorescence image labeled with particles having an average particle diameter of 30 to 800 nm and representing a specific substance present in the specimen as a fluorescent luminescent spot;
A bright spot extracting means for extracting a region of the fluorescent bright spot in the fluorescent image;
A composite image generating means for generating a composite image obtained by superimposing the bright field image and the fluorescent image before extraction of the fluorescent bright spot;
From the composite image, a composite image analysis means for specifying the position or amount of the specific substance in the specimen;
An image processing apparatus is provided.

According to the invention described in item 10 of the present invention,
Computer
An input means for inputting a bright-field image representing the morphology of cells in the specimen and a fluorescent image labeled with particles having an average particle diameter of 30 to 800 nm and representing a specific substance present in the specimen as a fluorescent luminescent spot;
A bright spot extracting means for extracting a region of the fluorescent bright spot in the fluorescent image;
A composite image generating means for generating a composite image in which the bright field image and the fluorescent image before extraction of the fluorescent bright spot are superimposed;
Synthetic image analysis means for identifying the position or amount of the specific substance in the specimen from the synthetic image;
A program for functioning as a server is provided.

  According to the present invention, the expression of a biological material based on the emission of a fluorescent material can be easily visually confirmed in a bright field image.

It is a figure which shows the system configuration | structure of the pathological-diagnosis assistance system using the biological material quantification method of this invention. It is a block diagram which shows the functional structure of the image processing apparatus of FIG. It is a figure which shows an example of a bright field image. It is a figure which shows an example of a fluorescence image. It is a flowchart which shows the 1st image processing method performed by the control part of FIG. It is a figure which shows a luminescent spot image. It is a figure which shows the luminescent spot image in which contrast was adjusted. It is an example of the image which overlap | superposed the bright spot image and the bright-field image in which the contrast was adjusted. FIG. 7B is an enlarged view of a region surrounded by a rectangle in FIG. 7A. It is an example of the image which superimposed the bright field image and the fluorescence image. It is an enlarged view of the area | region enclosed by the rectangle in FIG. 8A. It is a flowchart which shows the 2nd image processing method performed by the control part of FIG. It is a flowchart which shows the 3rd image processing method performed by the control part of FIG. It is an example of the coordinate image which extracted the bright spot coordinate from the bright spot image. It is an example of the synthesized image which overlap | superposed the bright spot image, the bright field image, and the coordinate image. It is an example of the coordinate image which extracted the outline of the bright spot area | region from the bright spot image. It is an example of the synthesized image which overlap | superposed the bright spot image, the bright field image, and the coordinate image.

  Hereinafter, although the form for implementing this invention with reference to figures is demonstrated, this invention is not limited to these.

<Configuration of Pathological Diagnosis Support System 100>
FIG. 1 shows an example of the overall configuration of a pathological diagnosis support system 100 using the biological material quantification method of the present invention. The pathological diagnosis support system 100 acquires a microscopic image of a specimen stained with a predetermined staining reagent and analyzes the acquired microscopic image to quantitatively express the expression of a specific biological material in a tissue specimen to be observed. It is a system that outputs the feature quantity to represent.

As shown in FIG. 1, the pathological diagnosis support system 100 is configured by connecting a microscope image acquisition device 1A and an image processing device 2A via an interface such as a cable 3A so that data can be transmitted and received. The connection method between the microscope image acquisition device 1A and the image processing device 2A is not particularly limited. For example, the microscope image acquisition apparatus 1A and the image processing apparatus 2A may be connected via a LAN (Local Area Network) or may be connected wirelessly.
Furthermore, the pathological diagnosis support system 100 may include a staining apparatus that automatically stains a specimen.

The microscope image acquisition apparatus 1A is a known optical microscope with a camera, and acquires a microscope image of a tissue specimen on a slide placed on a slide fixing stage, and transmits the microscope image to the image processing apparatus 2A.
The microscope image acquisition apparatus 1A includes an irradiation unit, an imaging unit, an imaging unit, a communication I / F, and the like. The irradiation means includes a light source, a filter, and the like, and irradiates the tissue specimen on the slide placed on the slide fixing stage with light. The imaging means is composed of an eyepiece lens, an objective lens, and the like, and forms an image of transmitted light, reflected light, or fluorescence emitted from the tissue specimen on the slide by the irradiated light. The imaging means is a microscope-installed camera that includes a CCD (Charge Coupled Device) sensor and the like, captures an image formed on the imaging surface by the imaging means, and generates digital image data of the microscope image. The communication I / F transmits image data of the generated microscope image to the image processing apparatus 2A. In the present embodiment, the microscope image acquisition apparatus 1A includes a bright field unit that combines an irradiation unit and an imaging unit suitable for bright field observation, and a fluorescence unit that combines an irradiation unit and an imaging unit suitable for fluorescence observation. It is possible to switch between bright field / fluorescence by switching units.

  Note that the microscope image acquisition apparatus 1A is not limited to a microscope with a camera. For example, a virtual microscope slide creation apparatus (for example, a special microscope microscope) that acquires a microscope image of an entire tissue specimen by scanning a slide on a microscope slide fixing stage. Table 2002-514319 gazette) etc. may be used. According to the virtual microscope slide creation device, it is possible to acquire image data that allows the entire tissue specimen image on the slide to be viewed on the display unit at a time.

The image processing apparatus 2A displays the expression distribution of a specific biological substance in the tissue specimen to be observed by analyzing the microscope image transmitted from the microscope image acquisition apparatus 1A.
FIG. 2 shows a functional configuration example of the image processing apparatus 2A. As shown in FIG. 2, the image processing apparatus 2 </ b> A includes a control unit 21, an operation unit 22, a display unit 23, a communication I / F 24, a storage unit 25, and the like, and each unit is connected via a bus 26. Yes.

  The control unit 21 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), and the like, executes various processes in cooperation with various programs stored in the storage unit 25, and performs image processing 2A. Overall control of the operation. For example, the control unit 21 executes image analysis processing (see FIG. 5) in cooperation with a program stored in the storage unit 25, and performs bright spot extraction means, synthesized image generation means, image adjustment means, and bright spot detection. Functions as coordinate calculation means and coordinate image generation means are realized.

  The operation unit 22 includes a keyboard having character input keys, numeric input keys, various function keys, and the like, and a pointing device such as a mouse, and a key pressing signal pressed by the keyboard and an operation signal by the mouse. Are output to the control unit 21 as an input signal.

  The display unit 23 includes a monitor such as a CRT (Cathode Ray Tube) or an LCD (Liquid Crystal Display), for example, and displays various screens according to instructions of a display signal input from the control unit 21. In the present embodiment, the display unit 23 functions as an output unit for outputting an image analysis result.

  The communication I / F 24 is an interface for transmitting and receiving data to and from external devices such as the microscope image acquisition apparatus 1A. The communication I / F 24 functions as a bright field image and fluorescent image input unit.

The storage unit 25 includes, for example, an HDD (Hard Disk Drive), a semiconductor nonvolatile memory, or the like. As described above, the storage unit 25 stores various programs, various data, and the like.
In addition, the image processing apparatus 2A may include a LAN adapter, a router, and the like and be connected to an external device via a communication network such as a LAN.

The image processing apparatus 2A in the present embodiment preferably generates a pathological image using a bright field image and a fluorescence image obtained by photographing the same field range transmitted from the microscope image acquisition apparatus 1A.
A bright-field image is obtained by enlarging and photographing a tissue specimen stained with an H (hematoxylin) staining reagent and an HE (hematoxylin-eosin) staining reagent in a bright field in the microscope image acquisition apparatus 1A. It is an image and is a cell form image showing the form of the cell in the tissue specimen. Hematoxylin is a blue-violet pigment that stains cell nuclei, bone tissue, part of cartilage tissue, serous components, etc. (basophilic tissue, etc.). Eosin is a red to pink pigment that stains cytoplasm, soft tissue connective tissue, red blood cells, fibrin, endocrine granules, and the like (eosinophilic tissue, etc.). FIG. 3 shows an example of a bright field image obtained by photographing a tissue specimen subjected to HE staining.
The fluorescence image emits a fluorescent material by irradiating a tissue sample stained with a staining reagent containing a fluorescent material that specifically labels a specific biological material with excitation light having a predetermined wavelength in the microscope image acquisition device 1A. It is a microscope image obtained by (fluorescence) and enlarging and photographing this fluorescence. That is, the fluorescence that appears in the fluorescence image indicates the expression of a specific biological material in the tissue specimen. FIG. 4 shows an example of the fluorescence image.

<Acquisition of fluorescence image>
Here, a method for acquiring a fluorescent image will be described in detail including a staining reagent used for acquiring the fluorescent image, a method for staining a tissue specimen with the staining reagent, and the like.

[Fluorescent substance]
Examples of the fluorescent substance used in the staining reagent for obtaining a fluorescent image include fluorescent organic dyes and quantum dots (semiconductor particles). When excited by ultraviolet to near infrared light having a wavelength in the range of 200 to 700 nm, it preferably emits visible to near infrared light having a wavelength in the range of 400 to 1100 nm.

  Fluorescent organic dyes include fluorescein dye molecules, rhodamine dye molecules, Alexa Fluor (Invitrogen) dye molecules, BODIPY (Invitrogen) dye molecules, cascade dye molecules, coumarin dye molecules, and eosin dyes. Examples include molecules, NBD dye molecules, pyrene dye molecules, Texas Red dye molecules, and cyanine dye molecules.

  Specifically, 5-carboxy-fluorescein, 6-carboxy-fluorescein, 5,6-dicarboxy-fluorescein, 6-carboxy-2 ′, 4,4 ′, 5 ′, 7,7′-hexachlorofluorescein, 6 -Carboxy-2 ', 4,7,7'-tetrachlorofluorescein, 6-carboxy-4', 5'-dichloro-2 ', 7'-dimethoxyfluorescein, naphthofluorescein, 5-carboxy-rhodamine, 6-carboxy Rhodamine, 5,6-dicarboxy-rhodamine, rhodamine 6G, tetramethylrhodamine, X-rhodamine, and Alexa Fluor 350, Alexa Fluor 405, Alexa Fluor 430, Alexa Fluor 488, Alexa Fluor 500, Alexa Fluor 514, Alexa Fluor 532, Alexa Fluor 546, Alexa Fluor 555, Alexa Fluor 568, Alexa Fluor 594, Alexa Fluor 610, Alexa Fluor 633, Alexa Fluor 635, Alexa Fluor 647, Alexa Fluor 660, Alexa Fluor 680, Alexa Fluor 700, Alexa Fluor 750, BODIPY FL, BODIPY TMR, BODIPY 493/503, BODIPY 530/550, BODIPY 558/568, BODIPY 564 / 570, BODIPY 576/589, BODIPY 581/591, BODIPY 630/650, BODIPY 650/665 (manufactured by Invitrogen), methoxycoumarin, eosin, NBD, pyrene, Cy5, Cy5.5, Cy7, etc. it can. You may use individually or what mixed multiple types.

  Quantum dots include II-VI group compounds, III-V group compounds, or quantum dots containing group IV elements as components ("II-VI group quantum dots", "III-V group quantum dots", " Or “Group IV quantum dots”). You may use individually or what mixed multiple types.

  Specific examples include, but are not limited to, CdSe, CdS, CdTe, ZnSe, ZnS, ZnTe, InP, InN, InAs, InGaP, GaP, GaAs, Si, and Ge.

[Fluorescent substance-containing nanoparticles]
In the present embodiment, the fluorescent material may form fluorescent material-containing nanoparticles (fluorescent particles). The fluorescent particles are those in which a fluorescent substance is dispersed inside the nanoparticles, and the fluorescent substance and the nanoparticles themselves may or may not be chemically bonded. The material constituting the nanoparticles is not particularly limited, and examples thereof include polystyrene, polylactic acid, silica, and melamine.

Moreover, the quantum dot which used the quantum dot as the core and provided the shell on it can also be used as a fluorescent particle. Hereinafter, as a notation of a quantum dot having a shell in this specification, when the core is CdSe and the shell is ZnS, it is expressed as CdSe / ZnS. For example, CdSe / ZnS, CdS / ZnS, InP / ZnS, InGaP / ZnS, Si / SiO ₂ , Si / ZnS, Ge / GeO ₂ , Ge / ZnS, and the like can be used, but are not limited thereto.
As the quantum dots, those subjected to surface treatment with an organic polymer or the like may be used as necessary. Examples thereof include CdSe / ZnS having a surface carboxy group (manufactured by Invitrogen), CdSe / ZnS having a surface amino group (manufactured by Invitrogen), and the like.

  The fluorescent particles used in the present embodiment can be produced by a known method, and in the encapsulation of the fluorescent dye, a method of synthesizing particles by binding the fluorescent dye molecules to the monomer as a raw material, a resin Any method may be used for introducing the fluorescent dye into the resin, such as a method of adsorbing and introducing the fluorescent dye into the resin.

  Polystyrene nanoparticles encapsulating a fluorescent organic dye may be copolymerized using an organic dye having a polymerizable functional group described in US Pat. No. 4,326,008 (1982) or polystyrene described in US Pat. No. 5,326,692 (1992). It can be produced using a method of impregnating nanoparticles with a fluorescent organic dye.

  The polymer nanoparticles encapsulating the quantum dots can be prepared by using the method of impregnating the quantum nanoparticles into polystyrene nanoparticles described in Nature Biotechnology, Vol. 19, page 631 (2001).

  Although the average particle diameter of the fluorescent substance-containing nanoparticles used in the present embodiment is not particularly limited, those having a size of about 30 to 800 nm can be used. Further, the coefficient of variation (= (standard deviation / average value) × 100%) indicating the variation in particle diameter is not particularly limited, but it is preferable to use a coefficient of 20% or less. The average particle diameter is obtained by taking an electron micrograph using a scanning electron microscope (SEM), measuring the cross-sectional area of a sufficient number of particles, and taking each measured value as the area of the circle. As sought. In the present application, the arithmetic average of the particle sizes of 1000 particles is defined as the average particle size. The coefficient of variation was also a value calculated from the particle size distribution of 1000 particles.

[Surface modification of fluorescent particles]
The fluorescent particles are subjected to surface modification for specifically binding and / or reacting with the biological material to be stained. The biological material to be stained is not particularly limited as long as a substance that specifically binds to it exists, but typically, protein (peptide) and nucleic acid (oligonucleotide, polynucleotide), antibody, etc. Is mentioned. Therefore, as a substance that binds to the biological material to be stained, an antibody that recognizes the protein as an antigen, another protein that specifically binds to the protein, a nucleic acid having a base sequence that hybridizes to the nucleic acid, or the like Is mentioned. Specifically, anti-Ki67 antibody that specifically binds to Ki67, a protein present in the cell nucleus, anti-ER antibody that specifically binds to estrogen receptor (ER) present in the cell nucleus, and actin that forms the cytoskeleton Specific examples include anti-actin antibodies that specifically bind. Among them, those obtained by binding anti-Ki67 antibody and anti-ER antibody to fluorescent particles can be used for selection of breast cancer medication, and are preferable.

The surface-modified fluorescent particles and the biological material to be stained may be directly bonded, or may be indirectly bonded via another substance. For example, a fluorescent material modified with streptavidin that specifically binds to biotin via a primary antibody that specifically binds to the biological material to be stained and a biotinylated secondary antibody that specifically binds to the primary antibody The biological material to be dyed may be bound.
As the primary antibody and the secondary antibody, any combination of antibodies capable of specifically binding fluorescent particles to a biological material to be stained can be used.

  Specific antigens can be exemplified as follows, and antibodies recognizing each antigen are available from various antibody manufacturers and can be prepared based on general knowledge. Examples of specific antigens include M. actin, MS actin, SM actin, ACTH, Alk-1, α1-antichymotrypsin, α1-antitrypsin, AFP, bcl-2, bcl-6, β- Catenin, BCA225, CA19-9, CA125, calcitonin, calretinin, CD1a, CD3, CD4, CD5, CD8, CD10, CD15, CD20, CD21, CD23, CD30, CD31, CD34, CD43, CD45, CD45R, CD56, CD57, CD61, CD68, CD79a, “CD99, MIC2”, CD138, chromogranin, c-KIT, c-MET, collagen type IV, Cox-2, cyclin D1, keratin, cytokeratin (high molecular weight), pankeratin, pankeratin, Cytokeratin 5/6 , Cytokeratin 7, cytokeratin 8, cytokeratin 8/18, cytokeratin 14, cytokeratin 19, cytokeratin 20, CMV, E-cadherin, EGFR, ER, EMA, EBV, factor VIII related antigen, fascin, FSH , Galectin-3, gastrin, GFAP, glucagon, glycophorin A, granzyme B, hCG, hGH, Helicobacter pylori, HBc antigen, HBs antigen, hepatocyte specific antigen, HER2, HSV-I, HSV-II, HHV-8, IgA , IgG, IgM, IGF-1R, inhibin, insulin, kappa L chain, Ki67, lambda L chain, LH, lysozyme, macrophage, melan A, MLH-1, MSH-2, myeloperoxidase, myogenin, myoglobin, myosin, Neuro Filament, NSE, p27 (Kip1), p53, p53, P63, PAX 5, PLAP, Pneumocystis carini, podoplanin (D2-40), PGR, prolactin, PSA, prostate acid phosphatase, Renal Cell Carcinoma, S100, somatostatin, Examples include spectrin, synaptophysin, TAG-72, TdT, thyroglobulin, TSH, TTF-1, TRAcP, tryptase, villin, vimentin, WT1, and Zap-70.

When the target biological material is a nucleic acid, examples of the specific nucleic acid gene that has been pointed out to be associated with a disease can include the following, and probes that recognize each specific nucleic acid gene are available as BAC probes: Can be created based on general knowledge. Specific specific nucleic acid genes include HER2, TOP2A, HER3, EGFR, P53, MET, and the like as genes related to cancer growth and the response rate of molecular target drugs, and are also known as various cancer-related genes. The following genes are listed as genes. As tyrosine kinase-related genes, ALK, FLT3, AXL, FLT4 (VEGFR3, DDR1, FMS (CSF1R), DDR2, EGFR (ERBB1), HER4 (ERBB4), EML4-ALK, IGF1R, EPHA1, INSR, EPHA2, IRR (INSRR) ), EPHA3, KIT, EPHA4, LTK, EPHA5, MER (MERTK), EPHA6, MET, EPHA7, MUSK, EPHA8, NPM1-ALK, EPHB1, PDGFRα (PDGFRA), EPHB2, PDGFRβ (PDGFRB4, RETHB, REPHEP , RON (MST1R), FGFR1, ROS (ROS1), FGFR2, TIE2 (TEK), FGFR3, TRKA (NTRK1), FGFR4, TRKB (N TRK2), FLT1 (VEGFR1), TRKC (NTRK3), and breast cancer-related genes include ATM, BRCA1, BRCA2, BRCA3, CCND1, E-Cadherin, ERBB2, ETV6, FGFR1, HRAS, KRAS, NRAS, TRAS Genes associated with carcinoid tumors include BCL2, BRD4, CCND1, CDKN1A, CDKN2A, CTNNB1, HES1, MAP2, MEN1, NF1, NOTCH1, NUT, RAF, SDHD, and VEGFA. Cancer-related genes include APC, MSH6, AXIN2, MYH, BMPR1A, p53, DCC, PMS2, KRAS2 (or Ki-ras), PTEN, MLH1, SM D4, MSH2, STK11, and MSH6 Lung cancer-related genes include ALK, PTEN, CCND1, RASSF1A, CDKN2A, RB1, EGFR, RET, EML4, ROS1, KRAS2, TP53, and MYC. Related genes include Axin1, MALAT1, b-catenin, p16 INK4A, c-ERBB-2, p53, CTNNB1, RB1, Cyclin D1, SMAD2, EGFR, SMAD4, IGFR2, TCF1, and KRAS. Related genes include Alpha, PRCC, ASPSCR1, PSF, CLTC, TFE3, p54nrb / NONO, and TFEB.
Examples of thyroid cancer-related genes include AKAP10, NTRK1, AKAP9, RET, BRAF, TFG, ELE1, TPM3, H4 / D10S170, and TPR. Examples of ovarian cancer-related genes include AKT2, MDM2, BCL2, MYC, BRCA1, NCOA4, CDKN2A, p53, ERBB2, PIK3CA, GATA4, RB, HRAS, RET, KRAS, and RNASET2. Examples of prostate cancer-related genes include AR, KLK3, BRCA2, MYC, CDKN1B, NKX3.1, EZH2, p53, GSTP1, and PTEN. Examples of bone tumor-related genes include CDH11, COL12A1, CNBP, OMD, COL1A1, THRAP3, COL4A5, and USP6.

  The mode of bonding between the fluorescent particle modifying substance and the fluorescent particle is not particularly limited, and examples thereof include covalent bond, ionic bond, hydrogen bond, coordination bond, physical adsorption, and chemical adsorption. A bond having a strong bonding force such as a covalent bond is preferable in terms of bond stability.

  In addition, there may be an organic molecule that connects between the fluorescent particle and a substance that modifies the fluorescent particle. For example, in order to suppress non-specific adsorption with a biological substance, a polyethylene glycol chain can be used, and SM (PEG) n (n = 2 to 24) manufactured by Thermo Scientific can be used.

  When the surface of the fluorescent substance-encapsulating silica nanoparticles is modified, the same procedure can be applied regardless of whether the fluorescent substance is a fluorescent organic dye or a quantum dot. For example, a silane coupling agent that is a compound widely used for bonding an inorganic substance and an organic substance can be used. This silane coupling agent is a compound having an alkoxysilyl group that gives a silanol group by hydrolysis at one end of the molecule and a functional group such as a carboxyl group, an amino group, an epoxy group, an aldehyde group at the other end, Bonding with an inorganic substance through an oxygen atom of the silanol group. Specifically, mercaptopropyltriethoxysilane, glycidoxypropyltriethoxysilane, aminopropyltriethoxysilane, a silane coupling agent having a polyethylene glycol chain (for example, PEG-silane no. SIM6492.7 manufactured by Gelest), etc. Is mentioned. When using a silane coupling agent, you may use 2 or more types together.

  A known procedure can be used for the reaction procedure of the fluorescent organic dye-containing nanoparticles and the silane coupling agent. For example, the obtained fluorescent organic dye-containing nanoparticles are dispersed in pure water, aminopropyltriethoxysilane is added, and the mixture is reacted at room temperature for 12 hours. After completion of the reaction, fluorescent organic dye-containing nanoparticles whose surface is modified with an aminopropyl group can be obtained by centrifugation or filtration. Subsequently, by reacting an amino group with a carboxyl group in the antibody, the antibody can be bound to the fluorescent organic dye-containing nanoparticles via an amide bond. If necessary, a condensing agent such as EDC (1-Ethyl-3- [3-Dimethylaminopropyl] carbodiimide Hydrochloride: manufactured by Pierce (registered trademark)) can also be used.

  If necessary, a linker compound having a site capable of directly binding to a fluorescent organic dye-encapsulated nanoparticle modified with an organic molecule and a site capable of binding to a molecular target substance can be used. As a specific example, sulfo-SMCC (Sulfosuccinimidyl 4 [N-maleimidomethyl] -cyclohexane-1-carboxylate: manufactured by Pierce) having both a site selectively reacting with an amino group and a site selectively reacting with a mercapto group When it is used, antibody-bound fluorescent organic dye-encapsulated nanoparticles can be produced by binding the amino group of fluorescent organic dye-encapsulated nanoparticles modified with aminopropyltriethoxysilane to the mercapto group in the antibody.

  When the biological substance recognition site is bound to the fluorescent substance-encapsulated polystyrene nanoparticles, the same procedure can be applied regardless of whether the fluorescent substance is a fluorescent organic dye or a quantum dot. That is, by impregnating a polystyrene nanoparticle having a functional group such as an amino group with a fluorescent organic dye or a quantum dot, a fluorescent substance-containing polystyrene nanoparticle having a functional group can be obtained, and thereafter EDC or sulfo-SMCC is used. In this way, antibody-bound fluorescent substance-encapsulated polystyrene nanoparticles can be produced.

[Dyeing method]
Hereinafter, a method for staining a tissue specimen will be described, but the present invention is not limited to a tissue specimen, and can also be applied to a specimen such as a cell fixed on a substrate.
Moreover, the preparation method of the sample which can apply the dyeing | staining method demonstrated below is not specifically limited, What was produced by the well-known method can be used.

1) Deparaffinization process The tissue specimen is immersed in a container containing xylene to remove paraffin. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, xylene may be exchanged during the immersion.
Next, the tissue specimen is immersed in a container containing ethanol to remove xylene. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. Further, if necessary, ethanol may be exchanged during the immersion.
Next, the tissue specimen is immersed in a container containing water to remove ethanol. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. Moreover, you may exchange water in the middle of immersion as needed.

2) Activation process The activation process of the target biological substance is performed according to a known method. The activation conditions are not particularly defined, but as the activation liquid, 0.01 M citrate buffer (pH 6.0), 1 mM EDTA solution (pH 8.0), 5% urea, 0.1 M Tris-HCl buffer, etc. Can be used. As the heating device, an autoclave, a microwave, a pressure cooker, a water bath, or the like can be used. The temperature is not particularly limited, but can be performed at room temperature. The temperature can be 50-130 ° C. and the time can be 5-30 minutes.
Next, the specimen after the activation treatment is immersed in a container containing PBS (Phosphate Buffered Saline) and washed. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter.
If necessary, the PBS may be replaced during the immersion.

3) Staining using a surface-modified fluorescent substance A PBS dispersion liquid of a surface-modified fluorescent substance is placed on a tissue specimen and reacted with a target biological substance. By changing the surface modification of the fluorescent substance, staining corresponding to various biological substances becomes possible. When a plurality of types of fluorescent materials having surface modifications are used, the respective fluorescent material PBS dispersion liquids may be mixed in advance or may be separately placed on the tissue specimen.
The temperature is not particularly limited, but can be performed at room temperature. The reaction time is preferably 30 minutes or more and 24 hours or less.
It is preferable to drop a known blocking agent such as BSA-containing PBS before staining with a fluorescent substance.
Next, the stained tissue specimen is immersed in a container containing PBS to remove unreacted fluorescent material. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, the PBS may be replaced during the immersion. Place the cover glass on the tissue specimen and enclose it. A commercially available encapsulant may be used as necessary.
In addition, when dyeing | staining using H dye reagent or HE dye reagent, H dye or HE dye is performed before enclosure with a cover glass.

(Fluorescence image acquisition)
A microscope image (fluorescence image) with a wide field of view is acquired from the stained tissue specimen using the microscope image acquisition device 1A. In the microscope image acquisition apparatus 1A, an excitation light source and a fluorescence detection optical filter corresponding to the absorption maximum wavelength and fluorescence wavelength of the fluorescent material used for the staining reagent are selected.

<Operation of Pathological Diagnosis Support System 100 (Including Image Processing Method)>
Hereinafter, an image processing method for acquiring the above-described fluorescence image and bright field image and generating a pathological image in the pathological diagnosis support system 100 will be described. In this case, a tissue sample stained with a staining reagent containing fluorescent particles combined with a biological substance recognition site that recognizes an HE staining reagent and a specific protein (hereinafter referred to as a specific protein) is to be observed. An example will be described, but the present invention is not limited to this.

First, an operator stains a tissue specimen using two types of staining reagents, an HE staining reagent and a staining reagent that uses fluorescent particles combined with a biological substance recognition site that recognizes a specific protein as a fluorescent labeling material.
Thereafter, in the microscope image acquisition apparatus 1A, a bright-field image and a fluorescent image obtained by photographing the same range as the bright-field image are acquired by the procedures (a1) to (a5).
(A1) The operator places the tissue specimen stained with the HE staining reagent and the staining reagent containing fluorescent particles on the slide, and places the slide on the slide fixing stage of the microscope image acquisition apparatus 1A.
(A2) Set to the bright field unit, adjust the imaging magnification and focus, and place the observation target area on the tissue in the field of view.
(A3) Shooting is performed by the imaging unit to generate bright field image data, and the image data is transmitted to the image processing apparatus 2A.
(A4) Change the unit to a fluorescent unit.
(A5) Shooting is performed by the imaging means without changing the field of view and the shooting magnification to generate image data of a fluorescent image, and the image data is transmitted to the image processing apparatus 2A.

  In the image processing apparatus 2A, image processing for generating a pathological image in which the bright field image and the fluorescence image are superimposed is executed. Image processing in the image processing apparatus 2A is executed in cooperation with the control unit 21 and a program stored in the storage unit 25.

[First image processing method]
FIG. 5 shows a flowchart of a first image processing method for generating a pathological image in the image processing apparatus 2A.
First, when a bright field image as shown in FIG. 3 is input from the microscope image acquisition apparatus 1A via the communication I / F 24 (step S11), the control unit 21 adjusts the contrast of the bright field image (step S12: image). An adjustment step) to generate a bright-field image in which the tissue-stained cell region is emphasized.

On the other hand, when a fluorescent image as shown in FIG. 4 is input from the microscope image acquisition apparatus 1A by the communication I / F 24 (step S13: input step), the control unit 21 displays a bright point indicating a fluorescent bright spot from the fluorescent image. An image from which the point area is extracted (bright spot image) is generated (step S14: bright spot extraction step).
The method of generating the bright spot image is arbitrary. For example, a color component corresponding to the wavelength of the fluorescence emitted by the fluorescent material is extracted from the fluorescent image, and pixels whose luminance is less than a predetermined threshold are transparent. It is generated by making the background area that does not have a bright spot transparent. For example, when the peak of the emission wavelength of the fluorescent particles is 615 nm, a pixel having a wavelength component of 615 nm or more is left as a bright spot region, and a bright spot image obtained by making other pixels a background region is made transparent. Generate. FIG. 6A is an example of a bright spot image.
Note that noise removal processing such as cell autofluorescence and other unnecessary signal components may be performed before the processing in step S14, and a low-pass filter such as a Gaussian filter or a high-pass filter such as a second derivative can be preferably used. FIG. 6A shows an example of a bright spot image.

After the processing of step S12 and step S14 is completed, the control unit 21 displays a composite image obtained by superimposing the bright field image and the bright spot image whose contrast has been adjusted on the display unit 23 (step S15: composite image generation step).
Next, the control unit 21 converts the hue of the bright spot image (step S16), adjusts the contrast (step S17), and changes the transmittance of the bright spot area (step S18) (image adjustment process). In the image adjustment process of steps S16 to S18, for example, when the operator performs an operation of changing the hue, contrast, and transmittance of the fluorescent image via the operation unit 22, the control unit 21 causes the display unit 23 to display. Update the composite image as needed. Thus, the operator can select the hue, contrast, and transmittance that make it easier to visually recognize the bright spot region on the bright field image while checking the composite image displayed on the display unit 23.
FIG. 6B is an example of the bright spot image after the processes of steps S16 to S18 are performed on the bright spot image of FIG. 6A.

  FIG. 7A is an example of a composite image after the process of step S18, and FIG. 7B shows an enlarged view of a region surrounded by a rectangle in FIG. 7A. As a comparison, an image obtained by simply adding the bright field image (FIG. 3) and the fluorescence image (FIG. 4) input from the communication I / F 24 without performing the processes of steps S12, S14, and S16 to S18 is shown in FIG. Shown in FIG. 8B shows an enlarged view of a region surrounded by a rectangle in FIG. 8A. From FIG. 7A, FIG. 7B, FIG. 8A, and FIG. 8B, by performing the processing of steps S12, S14, and S16 to S18, it is possible to obtain an image that makes it easy to confirm the distribution of the bright spot region on the bright field image. Recognize.

It should be noted that the composite image in the present embodiment is in a state where a plurality of images exist as layers, and image processing can be individually performed on each layer until all the image processing in FIG. 5 is completed. An image is preferred. After all the image processing in FIG. 5 is completed, the processing may be performed by integrating the layers into one image.
Further, the timing for displaying the composite image is arbitrary, and is not limited to after the processing of steps S12 and S14. For example, it may be displayed before the contrast adjustment of the bright field image in step S12 and the bright spot image generation in step S14, or after the processing in steps S16 to S18.

[Second image processing method]
FIG. 9 shows a flowchart of a second image processing method for generating a pathological image in the image processing apparatus 2A. Note that the second image processing method will be described mainly with respect to a configuration different from the first image processing method described above, and description of the common configuration will be omitted.
First, when a bright field image as shown in FIG. 3 is input from the microscope image acquisition apparatus 1A through the communication I / F 24 (step S21), the control unit 21 converts the hue of the bright field image (step S22: image). Adjusting step) to generate a bright field image in which the cell region to be observed in the bright field image is emphasized. For example, when it is desired to observe a specific protein expressed in the region of the cell nucleus of the sample subjected to HE staining, in step S22, hue conversion is performed such that the bright-field image is color-separated and the blue component is emphasized. .
Next, the control unit 21 extracts the hue of the bright field image subjected to the hue conversion (step S23). For example, the average value of the RGB values in all the pixels in the image is calculated and extracted as the hue of the bright field image.

On the other hand, when the fluorescence image from the microscope image acquisition device 1A is input by the communication I / F 24 (step S24), the control unit 21 generates a bright spot image in which the bright spot area is extracted from the fluorescent image (step S25). : Bright spot extraction process).
The details of the bright spot image generation method are the same as the bright spot image generation method in step S14 of the first image processing method.

  After the processing of step S23 and step S25 is completed, the control unit 21 converts the hue of the bright spot image based on the hue of the bright field image extracted in step S23 (step S26), and the brightness converted in step S22. A composite image obtained by superimposing the visual field image and the bright spot image whose hue is converted in step S26 is displayed on the display unit 23 (step S27: composite image generation step).

In step S26, the control unit 21 preferably automatically converts the hue of the bright spot region based on the hue of the bright field image extracted in step S23. Specifically, for example, the RGB value of the complementary color of the hue of the bright field image extracted in step S23 is calculated, and the hue of the bright spot image obtained by monochrome conversion is converted to the complementary color.
In addition, after the control unit 21 automatically performs the hue conversion of the bright spot region in step S26 and displays the composite image in step S27, the operator combines the same as in step S16 of the first image processing method. You may perform operation which changes a hue via the operation part 22, confirming an image.

In the second image processing method, at least one of the contrast adjustment process of the bright field image, the contrast adjustment of the bright spot image, and the transmittance change, similar to steps S16 to S18 of the first image processing method. An image adjustment step may be included.
The timing for displaying the composite image is arbitrary, and is not limited to after the hue conversion in step S26. For example, you may display before the hue conversion of step S26.

[Third image processing method]
FIG. 10 shows a flowchart of a third image processing method for generating a pathological image in the image processing apparatus 2A. Note that the third image processing method will be described mainly with respect to the configuration different from the first image processing method and the second image processing method described above, and description of the common configuration will be omitted.

The processes in steps S31 to S34 in the third image processing method are the same as the processes in steps S11 to S14 in the first image processing method.
After the process of step S34, the control unit 21 calculates the bright spot coordinates representing the position of each bright spot area (step S35: calculation step). The definition of the bright spot coordinates is arbitrary, but for example, the bright spot image generated in step S34 may be subjected to threshold processing, and the centroid of each binarized bright spot area may be calculated and used as the bright spot coordinates. In addition, when a plurality of fluorescent particles labeled with a specific protein are densely contained in one bright spot region, the position of each fluorescent particle is determined by any known method. You may calculate as a bright spot coordinate.

Next, the control unit 21 generates a coordinate image indicating the calculated bright spot coordinates (step S36: coordinate image generation step). In the coordinate image, the area other than the bright spot coordinates is transparent, and the display method of the bright spot coordinates is arbitrary. For example, the hue, transparency, size, etc. of the bright spot coordinates may be changed and displayed based on the feature values (luminance integrated value, area, perimeter length, etc.) of each bright spot area.
Also, for example, an image obtained by extracting the binarized bright spot region outline may be used as the coordinate image.
In such a coordinate image, for example, the area outside the outline of the bright spot area is transparent, and the outline of the bright spot area and the hue inside the outline are arbitrary. The hue, transparency, thickness, etc. of the contour line may be changed based on the feature amount (luminance integrated value, area, perimeter length, etc.) of each bright spot region.

  After the processing of step S32 and step S36 is completed, the control unit 21 displays three images, that is, the bright-field image whose contrast is adjusted, the bright spot image generated in step S34, and the coordinate image generated in step S36. The superimposed composite image is displayed on the display unit 23 (step S37: composite image generation step).

11A is an example of an enlarged view of the same area as FIG. 7B in the coordinate image obtained by extracting the bright spot coordinates from the bright spot image of FIG. 6B, and FIG. 11B is an example of a composite image. As shown in FIG. 11B, by displaying the synthesized image by superimposing the bright spot coordinates in addition to the bright spot area on the bright field image, it becomes easier to grasp the expression position and the expression number of the specific protein.
12A is an example of an enlarged view of the same area as FIG. 7B in the coordinate image obtained by extracting the outline of the bright spot area from the bright spot image of FIG. 6B, and FIG. 12B is an example of a composite image. As shown in FIG. 12B, by displaying a composite image in which the outline of the bright spot area is superimposed in addition to the bright spot area on the bright field image, it becomes easier to grasp the expression area of the specific protein.
In the composite image generation step in step S37, for example, an image obtained by superimposing only two images of the bright field image and the coordinate image with adjusted contrast may be displayed on the display unit 23 as a composite image.

Next, hue conversion (step S38) and transmittance adjustment (step S39) of the coordinate image are performed. In the processing of steps S38 to S39, for example, when the operator performs an operation of changing the hue and transmittance of the coordinate image via the operation unit 22, the control unit 21 displays a composite image displayed on the display unit 23. Update from time to time. Thereby, the operator can select the hue and the transmittance that make it easy to visually recognize the coordinate image on the bright field image while confirming the composite image displayed on the display unit 23.

Also in the third image processing method, the bright field image hue conversion and hue extraction steps similar to those in steps S22 to S23 in the second image processing method, and steps S16 to S18 in the first image processing method. At least one image adjustment process of hue conversion of the bright spot image, contrast adjustment, and transmittance change may be included.
Further, the timing for displaying the composite image is arbitrary, and is not limited to after the end of steps S32 and S36. For example, it may be displayed before the contrast adjustment of the bright field image in step S32 and the bright spot image generation in step S34.

According to the present embodiment described above, by synthesizing the fluorescence image subjected to image adjustment with the bright field image, the specific protein based on the fluorescence image is captured on the bright field image obtained by photographing the tissue specimen stained with the cell morphology. It is possible to generate a composite image whose expression can be easily confirmed by visual observation.
Further, by performing hue conversion and contrast adjustment for a bright field image, it is easier to grasp the expression of the specific protein, and a composite image with high visibility can be obtained.
If the hue of the fluorescent image is converted based on the hue of the bright field image as in the second image processing method, a synthesized image with high visibility of the expression position and expression level of the specific protein is automatically generated. Can be generated.
Also, as in the third image processing method, by displaying the coordinate image in addition to the bright spot area on the bright field image, the expression position, number of expression and expression area of the specific protein can be grasped more easily. can do.

In addition, the description content in the said embodiment is a suitable example of this invention, and is not limited to this.
For example, in the first to third image processing methods, the control unit 21 of the image processing apparatus 2A automatically performs all image processing without any operation performed by the operator via the operation unit 22. A composite image may be generated. Further, it is preferable that the operator can adjust the composite image by automatically adjusting the composite image automatically generated by the control unit 21 of the image processing apparatus 2 </ b> A via the operation unit 22.
Further, the types of images to be superimposed as a composite image may be arbitrarily selected and combined by the operator.

  Further, in the hue conversion of the bright spot image, for example, a heat map display using a plurality of hues may be used to perform hue conversion such that the hues are displayed in different colors according to the luminance of the fluorescence.

  In the first to third image processing methods, the background area of the fluorescence image is made transparent. However, even if the background area where there is no measurement object such as a cell or a cell nucleus is made transparent in the bright field image, good. The expression of a specific protein can also be achieved by performing a hue conversion such that the background area of the bright field image and the background area of the fluorescent image are substantially the same, for example, by changing the hue of the background area of the bright field image to black. It is possible to obtain a composite image with high visibility.

  Further, the biological material displayed on the bright field image by the method of the present invention is not limited to one type. For example, if a plurality of biological materials are stained with fluorescent materials that emit fluorescence of different wavelengths, the image processing of the present invention is performed on the fluorescent images representing the expression of each biological material, and bright field images are obtained. The expression of a plurality of biological substances can be displayed in different hues.

DESCRIPTION OF SYMBOLS 1A Microscope image acquisition apparatus 2A Image processing apparatus 3A Cable 21 Control part 22 Operation part 23 Display part 24 Communication I / F
25 storage unit 26 bus 100 pathological diagnosis support system

Claims (10)

An input step for inputting a bright-field image representing the morphology of cells in the specimen, and a fluorescence image labeled with particles having an average particle diameter of 30 to 800 nm and representing a specific substance present in the specimen as a fluorescent luminescent spot;
A bright spot extracting step of extracting the fluorescent spot area in the fluorescent image;
A composite image generating step for generating a composite image in which the bright field image and the fluorescent image before extraction of the fluorescent bright spot are superimposed;
From the composite image, a composite image analysis step for specifying the position or amount of the specific substance in the specimen;
An image processing method comprising:   It has an image adjustment process for performing at least one image processing among hue conversion and contrast adjustment for the bright field image, and hue conversion, contrast adjustment and transmittance adjustment for the fluorescent image. The image processing method according to claim 1.   The image processing method according to claim 2, wherein, in the image adjustment step, a hue of the fluorescent bright spot is converted with respect to the fluorescent image after the fluorescent bright spot is extracted.   The image processing method according to claim 2 or 3, wherein, in the image adjustment step, a contrast of the fluorescent bright spot is adjusted with respect to the fluorescent image after the fluorescent bright spot is extracted.   5. The image according to claim 2, wherein, in the image adjustment step, a transmittance of the fluorescent luminescent spot is adjusted with respect to the fluorescent image after extraction of the fluorescent luminescent spot. 6. Processing method.   In the image adjustment step, based on color information of the bright field image and the fluorescent image, among the fluorescence image after the extraction of the fluorescent luminescent spot, hue conversion, contrast adjustment, transmittance adjustment The image processing method according to claim 2, wherein at least one image processing is performed. A calculation step of calculating coordinates of the fluorescent luminescent spot;
A coordinate image generation step of generating a coordinate image indicating the calculated coordinates,
The image processing method according to claim 1, wherein in the composite image generation step, a composite image is generated by superimposing the bright field image, the fluorescent image, and the coordinate image. An input step for inputting a bright-field image representing the morphology of cells in the specimen, and a fluorescence image labeled with particles having an average particle diameter of 30 to 800 nm and representing a specific substance present in the specimen as a fluorescent luminescent spot;
A bright spot extracting step of extracting the fluorescent spot area in the fluorescent image;
A composite image generating step for generating a composite image in which the bright field image and the fluorescent image before extraction of the fluorescent bright spot are superimposed;
From the composite image, a composite image analysis step for specifying the position or amount of the specific substance in the specimen;
An image generation method characterized by comprising: An input means for inputting a bright-field image representing the morphology of cells in the specimen, and a fluorescence image labeled with particles having an average particle diameter of 30 to 800 nm and representing a specific substance present in the specimen as a fluorescent luminescent spot;
A bright spot extracting means for extracting a region of the fluorescent bright spot in the fluorescent image;
A composite image generating means for generating a composite image obtained by superimposing the bright field image and the fluorescent image before extraction of the fluorescent bright spot;
From the composite image, a composite image analysis means for specifying the position or amount of the specific substance in the specimen;
An image processing apparatus comprising: Computer
An input means for inputting a bright-field image representing the morphology of cells in the specimen and a fluorescent image labeled with particles having an average particle diameter of 30 to 800 nm and representing a specific substance present in the specimen as a fluorescent luminescent spot;
A bright spot extracting means for extracting a region of the fluorescent bright spot in the fluorescent image;
A composite image generating means for generating a composite image in which the bright field image and the fluorescent image before extraction of the fluorescent bright spot are superimposed;
Synthetic image analysis means for identifying the position or amount of the specific substance in the specimen from the synthetic image;
Program to function as.

Images