JP6375925B2 - Image processing apparatus, image processing system, image processing program, and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing system, an image processing program, and an image processing method.

  In the pathological diagnosis, dehydration is first performed in order to fix the collected tissue, and processing such as blocking with paraffin is performed. Then, the slice is cut into thin pieces having a thickness of 2 to 8 μm, the paraffin is removed, stained, and observed with a microscope. In this microscopic image, a pathologist makes a diagnosis based on morphological information such as a change in the size and shape of a cell nucleus and a change in a pattern as a tissue, and staining information.

In pathological diagnosis, specifying a specific biological substance that is overexpressed in a tissue section and the amount of expression thereof can be very important information for determining a prognosis and determining a subsequent treatment plan.
Examples of specific biological substances that are overexpressed include, for example, HER2 protein that is expressed on the cell membrane and Ki67 that is expressed on the cell nucleus in breast cancer, lung cancer, colon cancer, stomach cancer, bladder cancer, and other cells. Examples include proteins.

  In a conventional pathological diagnosis, for example, the expression level and distribution of a biological material were evaluated from an image of a biological tissue stained with a specific biological material by a dye staining method using an enzyme such as the DAB method. However, a conventional dye staining method such as the DAB method has low accuracy when the amount of expression of a biological substance is quantified in detail.

  On the other hand, in recent years, pathological diagnosis has been performed in which a specific biological material is stained with a fluorescent material to evaluate the expression level and distribution of the biological material. By using a fluorescent substance, the expression of a specific biological substance in a tissue section is expressed as a fluorescent bright spot on a fluorescent image, and the expression level and distribution can be grasped with high accuracy, which is extremely useful.

  When evaluating the expression level and distribution of a biological substance per cell in a pathological diagnosis using a fluorescent substance, a bright-field image showing the form of a cell taken in the same visual field range as the fluorescent image is acquired, and the fluorescent image And the bright field image are superposed to grasp the positional relationship between the fluorescent bright spot and the cell region. However, when the bright-field image and the fluorescence image are overlaid, the imaging areas of the two images may be shifted due to vibrations associated with switching of the imaging means, deviation of the optical axis of the microscope, and the like. There was a problem that errors could occur in the results.

  In addition, since it is difficult to automatically extract a cell membrane from a bright field image after HE staining that is generally performed in conventional pathological diagnosis, for example, in the technique described in Patent Document 1, a predetermined value is determined from the center of gravity of a cell nucleus. Diagnosis is made by regarding a circular region within the range of the distance as a cell region surrounded by a cell membrane. However, there is a large error with respect to the actual cell region, and the accuracy is insufficient to perform pathological diagnosis by analyzing the expression level and distribution of biological material for each cell.

  Therefore, for example, in Patent Document 2, an alignment marker that can be detected in both a two-dimensional visible image obtained by optical microscope photography and a two-dimensional analysis image obtained by microscopic mass spectrometry is attached to a specimen, thereby superimposing two images. A technique for automatically aligning at the time of alignment is described.

  In addition, for example, in Patent Document 3, in addition to fluorescent staining of a biological material to be quantified, by performing fluorescent staining of a cell membrane, the position of the cell membrane is specified on a fluorescent image, and the biological material on the cell membrane is detected. Techniques for assessing expression are described.

International Publication No. 2013/146843 JP 2010-85219 A JP 2013-57631 A

  According to the technique described in Patent Document 2 or 3, in addition to the conventional staining necessary for pathological diagnosis, a marker substance attaching step for alignment or a fluorescent staining step for cell membrane is separately required. In addition, there is a problem that the specimen is easily deteriorated.

  Therefore, the main object of the present invention is to accurately express the expression of a specific biological material in a cell to be observed without adding a marker substance or staining the sample in addition to staining generally performed in conventional pathological diagnosis. An object is to provide an image processing apparatus, an image processing system, an image processing program, and an image processing method that can be diagnosed.

In order to solve the above problem, according to the invention described in claim 1,
An input means for inputting a fluorescence image representing the expression of a specific biological substance in a specimen with a fluorescent bright spot, and a cell morphology image representing a cell morphology based on an information source different from the fluorescent bright spot;
An alignment means for aligning the cell morphology image and the fluorescent image based on a positional relationship between the cell morphology in the cell morphology image and the fluorescent bright spot in the fluorescence image;
An image processing apparatus is provided.

According to invention of Claim 2,
The alignment means performs alignment by performing at least one image processing of translation, rotation, enlargement, or reduction on at least one of the cell morphology image and the fluorescence image. An image processing apparatus according to claim 1 is provided.

According to invention of Claim 3,
3. The image processing apparatus according to claim 1, wherein the morphology of the cell in the cell morphology image is a nucleus region indicating a region of a stained cell nucleus.

According to invention of Claim 4,
The image processing apparatus according to claim 3, wherein the specific biological substance is expressed in a cell nucleus.

According to the invention of claim 5,
The image processing apparatus according to claim 3, wherein the specific biological substance is expressed in a cell membrane.

According to the invention of claim 6,
Having a calculation means for calculating the number of the fluorescent bright spots overlapping the nucleus region;
The image processing apparatus according to claim 4, wherein the alignment unit performs alignment so that the number of fluorescent bright spots is maximized.

According to the invention of claim 7,
Having a calculation means for calculating the number of the fluorescent bright spots overlapping the nucleus region;
The image processing apparatus according to claim 5, wherein the alignment unit performs alignment so that the number of fluorescent bright spots is minimized.

According to the invention described in claim 8,
Based on the positional relationship between the nuclear region and the fluorescent luminescent spot, there is a determining means for determining which of the nuclear regions each of the fluorescent luminescent points corresponds to,
5. The alignment means calculates a distance between the fluorescent luminescent spot and a centroid of a corresponding nuclear region, and performs alignment so that a total sum of the distances with the centroid is minimized. Is provided.

According to the invention of claim 9,
Based on the positional relationship between the nuclear region and the fluorescent luminescent spot, there is a determining means for determining which of the nuclear regions each of the fluorescent luminescent points corresponds to,
The alignment means calculates the shortest distance between the fluorescent luminescent spot and the contour of the corresponding nuclear region, and if the fluorescent luminescent spot overlaps the corresponding nuclear region, the shortest distance from the contour is calculated. 6. The image processing apparatus according to claim 5, wherein after the replacement with a predetermined fixed value, alignment is performed so that the sum of the shortest distances to the contour is minimized.

According to the invention of claim 10,
Based on the positional relationship between the nuclear region and the fluorescent luminescent spot, there is a determining means for determining which of the nuclear regions each of the fluorescent luminescent points corresponds to,
The alignment means calculates the distance between the fluorescent luminescent spot and the center of gravity of the corresponding nuclear region, calculates the standard deviation of the distance from the center of gravity for each of the core regions, and the sum of the standard deviations is the smallest The image processing apparatus according to claim 5, wherein alignment is performed so that

According to the invention of claim 11,
The alignment means performs alignment so that the total sum of the standard deviations calculated in the nucleus region in which the number of corresponding fluorescent bright spots is larger than a predetermined value is minimized. 10 is provided.

According to the invention of claim 12,
12. The alignment according to claim 2, wherein the alignment unit performs alignment by performing parallel movement within a range of a predetermined limit value on at least one of the cell morphology image and the fluorescence image. An image processing apparatus according to any one of the items is provided.

According to the invention of claim 13,
13. The alignment unit performs alignment by performing enlargement or reduction within a predetermined limit value range on at least one of the cell morphology image and the fluorescence image. An image processing apparatus according to any one of the above is provided.

According to the invention of claim 14,
9. The image processing according to claim 4, further comprising a deletion unit that deletes the fluorescent bright spot that does not overlap the nucleus region after the alignment by the alignment unit. An apparatus is provided.

According to the invention of claim 15,
The image processing apparatus according to claim 1, wherein the alignment unit performs alignment every time the cell morphology image and the fluorescence image are acquired.

According to the invention of claim 16,
The image processing apparatus according to any one of claims 1 to 15,
An image acquisition device used in the image processing device for acquiring the cell morphology image and the fluorescence image;
An image processing system is provided.

According to the invention of claim 17,
Computer
An input means for inputting a fluorescence image representing the expression of a specific biological substance in a specimen with a fluorescent bright spot and a cell morphology image representing a cell morphology based on an information source different from the fluorescent bright spot;
An alignment means for aligning the cell morphology image and the fluorescent image based on a positional relationship between the cell morphology in the cell morphology image and the fluorescent bright spot in the fluorescence image;
An image processing program for functioning as

According to the invention of claim 18,
An input step of inputting a fluorescence image representing the expression of a specific biological substance in a specimen with a fluorescent bright spot, and a cell morphology image representing a cell morphology based on an information source different from the fluorescent bright spot;
An alignment step of aligning the cell morphology image and the fluorescent image based on the positional relationship between the cell morphology in the cell morphology image and the fluorescent bright spot in the fluorescence image;
An image processing method is provided.

  According to the present invention, it is possible to accurately diagnose the expression of a specific biological substance in a cell to be observed without attaching a marker substance or staining the specimen in addition to staining generally performed in conventional pathological diagnosis. it can.

It is a figure which shows the system configuration | structure of a pathological-diagnosis assistance system. It is a block diagram which shows the functional structure of the image processing apparatus of FIG. It is a figure which shows an example of a bright field image. It is a figure which shows an example of a fluorescence image. It is a figure which shows the relationship between the number of bright spots at the time of using the label | marker material A, and a FISH score. It is a figure which shows the relationship between the number of bright spots at the time of using the labeling material B, and a FISH score. It is a figure which shows the relationship between the number of bright spots at the time of using the label | marker material C, and a FISH score. It is a figure which shows the relationship between the number of bright spots at the time of using the label | marker material D, and a FISH score. It is a figure which shows the square value of the correlation coefficient of the number of bright spots measured from the microscope image of each visual field, and a FISH score about each labeling material AD. It is a flowchart which shows the image analysis process performed by the control part of FIG. It is a flowchart which shows the detail of the process of step S2 of FIG. It is an example of the image based on a bright field image, Comprising: It is a figure which shows a monochrome image. It is an example of the image based on a bright field image, Comprising: It is a figure which shows the binary image after a threshold value process. It is an example of the image based on a bright field image, Comprising: It is a figure which shows the binary image after a noise process. It is a flowchart which shows the detail of a process of FIG.10 S4. It is an example of the image based on a fluorescence image, Comprising: It is a figure which shows a fluorescent luminescent point candidate image. It is an example of the image based on a fluorescence image, Comprising: It is a figure which shows the fluorescence luminescent spot image obtained after the noise removal in a fluorescence luminescent spot candidate image. It is a schematic diagram explaining an example of the positioning method in case a specific protein is a nucleoprotein. It is a schematic diagram explaining an example of the positioning method in case a specific protein is a membrane protein. It is a figure which shows an example of an analysis result screen.

  Hereinafter, although the form for implementing this invention with reference to figures is demonstrated, this invention is not limited to these.

<Configuration of Pathological Diagnosis Support System 100>
FIG. 1 shows an example of the overall configuration of a pathological diagnosis support system 100 in the present embodiment. The pathological diagnosis support system 100 acquires a microscopic image of a tissue section of a human body stained with a predetermined staining reagent, and analyzes the acquired microscopic image, thereby expressing the expression of a specific biological material in the tissue section to be observed. This is a system that outputs feature quantities quantitatively.

  As shown in FIG. 1, the pathological diagnosis support system 100 is configured by connecting a microscope image acquisition device 1A and an image processing device 2A so that data can be transmitted and received via an interface such as a cable 3A. The connection method between the microscope image acquisition device 1A and the image processing device 2A is not particularly limited. For example, the microscope image acquisition apparatus 1A and the image processing apparatus 2A may be connected via a LAN (Local Area Network) or may be connected wirelessly.

The microscope image acquisition apparatus 1A is a known optical microscope with a camera, and acquires a microscope image of a tissue section on a slide placed on a slide fixing stage and transmits it to the image processing apparatus 2A.
The microscope image acquisition apparatus 1A includes an irradiation unit, an imaging unit, an imaging unit, a communication I / F, and the like. The irradiating means includes a light source, a filter, and the like, and irradiates light to a tissue section on the slide placed on the slide fixing stage. The imaging means is composed of an eyepiece lens, an objective lens, and the like, and forms an image of transmitted light, reflected light, or fluorescence emitted from the tissue section on the slide by the irradiated light. The imaging means is a microscope-installed camera that includes a CCD (Charge Coupled Device) sensor and the like, captures an image formed on the imaging surface by the imaging means, and generates digital image data of the microscope image. The communication I / F transmits image data of the generated microscope image to the image processing apparatus 2A. In the present embodiment, the microscope image acquisition apparatus 1A includes a bright field unit that combines an irradiation unit and an imaging unit suitable for bright field observation, and a fluorescence unit that combines an irradiation unit and an imaging unit suitable for fluorescence observation. It is possible to switch between bright field / fluorescence by switching units.

  Note that the microscope image acquisition apparatus 1A is not limited to a microscope with a camera. For example, a virtual microscope slide creation apparatus (for example, a special microscope scan apparatus that acquires a microscope image of an entire tissue section by scanning a slide on a microscope slide fixing stage). Table 2002-514319 gazette) etc. may be used. According to the virtual microscope slide creation device, it is possible to acquire image data that allows a display unit to view a whole tissue section on a slide at a time.

The image processing device 2A calculates a feature amount quantitatively indicating the expression level of a specific biological material in the tissue section to be observed by analyzing the microscope image transmitted from the microscope image acquisition device 1A. Output feature values.
FIG. 2 shows a functional configuration example of the image processing apparatus 2A. As shown in FIG. 2, the image processing apparatus 2 </ b> A includes a control unit 21, an operation unit 22, a display unit 23, a communication I / F 24, a storage unit 25, and the like, and each unit is connected via a bus 26. Yes.

  The control unit 21 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), and the like, executes various processes in cooperation with various programs stored in the storage unit 25, and performs image processing 2A. Overall control of the operation. For example, the control unit 21 executes image analysis processing (see FIG. 10) in cooperation with a program stored in the storage unit 25, and realizes functions as an alignment unit, a calculation unit, a determination unit, and a deletion unit. To do.

  The operation unit 22 includes a keyboard having character input keys, numeric input keys, various function keys, and the like, and a pointing device such as a mouse, and a key pressing signal pressed by the keyboard and an operation signal by the mouse. Are output to the control unit 21 as an input signal.

  The display unit 23 includes a monitor such as a CRT (Cathode Ray Tube) or an LCD (Liquid Crystal Display), for example, and displays various screens according to instructions of a display signal input from the control unit 21. In the present embodiment, the display unit 23 functions as an output unit for outputting an image analysis result.

  The communication I / F 24 is an interface for transmitting and receiving data to and from external devices such as the microscope image acquisition apparatus 1A. The communication I / F 24 functions as a bright field image and fluorescent image input unit.

The storage unit 25 includes, for example, an HDD (Hard Disk Drive), a semiconductor nonvolatile memory, or the like. As described above, the storage unit 25 stores various programs, various data, and the like.
In addition, the image processing apparatus 2A may include a LAN adapter, a router, and the like and be connected to an external device via a communication network such as a LAN.

The image processing apparatus 2A in the present embodiment performs analysis using the bright field image (HE-stained image) and the fluorescence image transmitted from the microscope image acquisition apparatus 1A.
The bright field image is a microscope image obtained by enlarging and photographing a tissue section stained with a HE (hematoxylin-eosin) staining reagent in a bright field in the microscope image acquisition apparatus 1A. Hematoxylin is a blue-violet pigment that stains cell nuclei, bone tissue, part of cartilage tissue, serous components, etc. (basophilic tissue, etc.). Eosin is a red to pink pigment that stains cytoplasm, soft tissue connective tissue, red blood cells, fibrin, endocrine granules, and the like (eosinophilic tissue, etc.). FIG. 3 shows an example of a bright field image obtained by photographing a tissue section subjected to HE staining. As shown in FIG. 3, in the bright field image obtained by photographing the tissue section subjected to HE staining, the morphology of the cells in the tissue section appears. That is, the bright field image is a cell morphology image representing the morphology of cells in the tissue section. In such a bright-field image, the cell nucleus appears in a darker color (blue purple) than the surrounding cytoplasm and distinguished from the surrounding, and the morphology of the cell nucleus can be clearly captured.
The fluorescent image is stained using a staining reagent including nanoparticles (referred to as fluorescent substance-containing nanoparticles) encapsulating a fluorescent substance bound with a biological substance recognition site that specifically binds and / or reacts with a specific biological substance. This is a microscope image obtained by irradiating the tissue section with the excitation light of a predetermined wavelength in the microscope image acquisition device 1A to emit fluorescent substance-containing nanoparticles (fluorescence), and enlarging and photographing this fluorescence. . That is, the fluorescence that appears in the fluorescence image indicates the expression of a specific biological material corresponding to the biological material recognition site in the tissue section. FIG. 4 shows an example of the fluorescence image.

<Acquisition of fluorescence image>
Here, a method for acquiring a fluorescent image will be described in detail, including a staining reagent (fluorescent substance-containing nanoparticles) used for acquiring the fluorescent image, a method for staining a tissue section with the staining reagent, and the like.

[Fluorescent substance]
Examples of the fluorescent substance used in the staining reagent for obtaining a fluorescent image include fluorescent organic dyes and quantum dots (semiconductor particles). When excited by ultraviolet to near infrared light having a wavelength in the range of 200 to 700 nm, it preferably emits visible to near infrared light having a wavelength in the range of 400 to 1100 nm.

  Examples of fluorescent organic dyes include fluorescein dye molecules, rhodamine dye molecules, Alexa Fluor (Invitrogen) dye molecules, BODIPY (Invitrogen) dye molecules, cascade dye molecules, coumarin dye molecules, and eosin dyes. Examples include molecules, NBD dye molecules, pyrene dye molecules, Texas Red dye molecules, and cyanine dye molecules.

  Specifically, 5-carboxy-fluorescein, 6-carboxy-fluorescein, 5,6-dicarboxy-fluorescein, 6-carboxy-2 ′, 4,4 ′, 5 ′, 7,7′-hexachlorofluorescein, 6 -Carboxy-2 ', 4,7,7'-tetrachlorofluorescein, 6-carboxy-4', 5'-dichloro-2 ', 7'-dimethoxyfluorescein, naphthofluorescein, 5-carboxy-rhodamine, 6-carboxy Rhodamine, 5,6-dicarboxy-rhodamine, rhodamine 6G, tetramethylrhodamine, X-rhodamine, and Alexa Fluor 350, Alexa Fluor 405, Alexa Fluor 430, Alexa Fluor 488, Alexa Fluor 500, Alexa Fluor 514, Alexa Fluor 532, Alexa Fluor 546, Alexa Fluor 555, Alexa Fluor 568, Alexa Fluor 594, Alexa Fluor 610, Alexa Fluor 633, Alexa Fluor 633, Alexa Fluor 633 , Alexa Fluor 750, BODIPY FL, BODIPY TMR, BODIPY 493/503, BODIPY 530/550, BODIPY 558/568, BODIPY 564/570, BODIPY 576/589, BODIPY 581/591, 30/30 ODIPY 650/665 (all manufactured by Invitrogen), methoxy coumarin, eosin, NBD, pyrene, Cy5, Cy5.5, can be mentioned Cy7 like. You may use individually or what mixed multiple types.

  Quantum dots include quantum dots containing II-VI group compounds, III-V group compounds, or IV group elements as components ("II-VI group quantum dots", "III-V group quantum dots", " Any of the “Group IV quantum dots” can be used. You may use individually or what mixed multiple types.

  Specific examples include, but are not limited to, CdSe, CdS, CdTe, ZnSe, ZnS, ZnTe, InP, InN, InAs, InGaP, GaP, GaAs, Si, and Ge.

It is also possible to use a quantum dot having the above quantum dot as a core and a shell provided thereon. Hereinafter, as a notation of a quantum dot having a shell in this specification, when the core is CdSe and the shell is ZnS, it is expressed as CdSe / ZnS. For example, CdSe / ZnS, CdS / ZnS, InP / ZnS, InGaP / ZnS, Si / SiO ₂ , Si / ZnS, Ge / GeO ₂ , Ge / ZnS, and the like can be used, but are not limited thereto.
As the quantum dots, those subjected to surface treatment with an organic polymer or the like may be used as necessary. Examples thereof include CdSe / ZnS having a surface carboxy group (manufactured by Invitrogen), CdSe / ZnS having a surface amino group (manufactured by Invitrogen), and the like.

[Fluorescent substance-containing nanoparticles]
In the present embodiment, the fluorescent substance-encapsulating nanoparticles are those in which the fluorescent substance is dispersed inside the nanoparticles, whether the fluorescent substance and the nanoparticles themselves are chemically bonded or not. Good.
The material constituting the nanoparticles is not particularly limited, and examples thereof include polystyrene, polylactic acid, and silica.

  The fluorescent substance-containing nanoparticles used in the present embodiment can be produced by a known method. For example, silica nanoparticles encapsulating a fluorescent organic dye can be synthesized with reference to the synthesis of FITC-encapsulated silica particles described in Langmuir 8, Vol. 2921 (1992). Various fluorescent organic dye-containing silica nanoparticles can be synthesized by using a desired fluorescent organic dye in place of FITC.

  Silica nanoparticles encapsulating quantum dots can be synthesized with reference to the synthesis of CdTe-encapsulated silica nanoparticles described in New Journal of Chemistry, Vol. 33, p. 561 (2009).

  Polystyrene nanoparticles encapsulating a fluorescent organic dye may be copolymerized using an organic dye having a polymerizable functional group described in US Pat. No. 4,326,008 (1982) or polystyrene described in US Pat. No. 5,326,692 (1992). It can be produced using a method of impregnating nanoparticles with a fluorescent organic dye.

  The polymer nanoparticles encapsulating the quantum dots can be prepared by using the method of impregnating the quantum nanoparticles into polystyrene nanoparticles described in Nature Biotechnology, Vol. 19, page 631 (2001).

  Although the average particle diameter of the fluorescent substance-containing nanoparticles used in the present embodiment is not particularly limited, those having a size of about 30 to 800 nm can be used. Further, the coefficient of variation (= (standard deviation / average value) × 100%) indicating the variation in particle diameter is not particularly limited, but it is preferable to use a coefficient of 20% or less. The average particle diameter is obtained by taking an electron micrograph using a scanning electron microscope (SEM), measuring the cross-sectional area of a sufficient number of particles, and taking each measured value as the area of the circle. As sought. In the present application, the arithmetic average of the particle sizes of 1000 particles is defined as the average particle size. The coefficient of variation was also a value calculated from the particle size distribution of 1000 particles.

[Bonding of biological substance recognition sites and fluorescent substance-containing nanoparticles]
The biological material recognition site according to the present embodiment is a site that specifically binds and / or reacts with the target biological material. The target biological substance is not particularly limited as long as a substance that specifically binds to the target biological substance exists, but typically, protein (peptide), nucleic acid (oligonucleotide, polynucleotide), antibody, etc. Is mentioned. Accordingly, substances that bind to the target biological substance include antibodies that recognize the protein as an antigen, other proteins that specifically bind to the protein, and nucleic acids having a base sequence that hybridizes to the nucleic acid. Is mentioned. Specifically, an anti-HER2 antibody that specifically binds to HER2, which is a protein present on the cell surface, an anti-ER antibody that specifically binds to an estrogen receptor (ER) present in the cell nucleus, and actin that forms a cytoskeleton And an anti-actin antibody that specifically binds to. Among them, those in which anti-HER2 antibody and anti-ER antibody are bound to fluorescent substance-encapsulating nanoparticles can be used for selection of breast cancer medication, and are preferable.

  The mode of binding between the biological substance recognition site and the fluorescent substance-encapsulating nanoparticles is not particularly limited, and examples thereof include covalent bonding, ionic bonding, hydrogen bonding, coordination bonding, physical adsorption, and chemical adsorption. A bond having a strong bonding force such as a covalent bond is preferred from the viewpoint of bond stability.

  Moreover, there may be an organic molecule that connects between the biological substance recognition site and the fluorescent substance-encapsulating nanoparticles. For example, in order to suppress nonspecific adsorption with a biological substance, a polyethylene glycol chain can be used, and SM (PEG) 12 manufactured by Thermo Scientific can be used.

  When the biological substance recognition site is bound to the fluorescent substance-encapsulating silica nanoparticles, the same procedure can be applied regardless of whether the fluorescent substance is a fluorescent organic dye or a quantum dot. For example, a silane coupling agent that is a compound widely used for bonding an inorganic substance and an organic substance can be used. This silane coupling agent is a compound having an alkoxysilyl group that gives a silanol group by hydrolysis at one end of the molecule and a functional group such as a carboxyl group, an amino group, an epoxy group, an aldehyde group at the other end, Bonding with an inorganic substance through an oxygen atom of the silanol group. Specifically, mercaptopropyltriethoxysilane, glycidoxypropyltriethoxysilane, aminopropyltriethoxysilane, a silane coupling agent having a polyethylene glycol chain (for example, PEG-silane no. SIM6492.7 manufactured by Gelest), etc. Is mentioned. When using a silane coupling agent, you may use 2 or more types together.

  A known procedure can be used for the reaction procedure of the fluorescent organic dye-encapsulated silica nanoparticles and the silane coupling agent. For example, the obtained fluorescent organic dye-encapsulated silica nanoparticles are dispersed in pure water, aminopropyltriethoxysilane is added, and the mixture is reacted at room temperature for 12 hours. After completion of the reaction, fluorescent organic dye-encapsulated silica nanoparticles whose surface is modified with an aminopropyl group can be obtained by centrifugation or filtration. Subsequently, by reacting an amino group with a carboxyl group in the antibody, the antibody can be bound to the fluorescent organic dye-encapsulated silica nanoparticles via an amide bond. If necessary, a condensing agent such as EDC (1-Ethyl-3- [3-Dimethylaminopropyl] carbohydrate Hydrochloride: manufactured by Pierce (registered trademark)) can also be used.

  If necessary, a linker compound having a site capable of directly binding to a fluorescent organic dye-encapsulated silica nanoparticle modified with an organic molecule and a site capable of binding to a molecular target substance can be used. As a specific example, sulfo-SMCC having both a site that selectively reacts with an amino group and a site that reacts selectively with a mercapto group (Sulfosuccinimidyl 4 [N-maleimidemethyl] -cyclohexane-1-carboxylate: manufactured by Pierce) When used, by binding the amino group of the fluorescent organic dye-encapsulated silica nanoparticles modified with aminopropyltriethoxysilane and the mercapto group in the antibody, antibody-bound fluorescent organic dye-encapsulated silica nanoparticles can be produced.

  When the biological substance recognition site is bound to the fluorescent substance-encapsulated polystyrene nanoparticles, the same procedure can be applied regardless of whether the fluorescent substance is a fluorescent organic dye or a quantum dot. That is, by impregnating polystyrene nanoparticles having functional groups such as amino groups with fluorescent organic dyes and quantum dots, it is possible to obtain fluorescent substance-containing polystyrene nanoparticles having functional groups, and thereafter using EDC or sulfo-SMCC. Thus, antibody-bound fluorescent substance-encapsulated polystyrene nanoparticles can be produced.

  Examples of antibodies that recognize specific antigens include M.I. Actin, M.I. S. Actin, S. M.M. Actin, ACTH, Alk-1, α1-antichymotrypsin, α1-antitrypsin, AFP, bcl-2, bcl-6, β-catenin, BCA225, CA19-9, CA125, calcitonin, calretinin, CD1a, CD3, CD4, CD5, CD8, CD10, CD15, CD20, CD21, CD23, CD30, CD31, CD34, CD43, CD45, CD45R, CD56, CD57, CD61, CD68, CD79a, "CD99, MIC2", CD138, chromogranin, c-KIT, c-MET, collagen type IV, Cox-2, cyclin D1, keratin, cytokeratin (high molecular weight), pankeratin, pankeratin, cytokeratin 5/6, cytokeratin 7, cytokeratin 8, cytokeratin 8/1 8, cytokeratin 14, cytokeratin 19, cytokeratin 20, CMV, E-cadherin, EGFR, ER, EMA, EBV, factor VIII related antigen, fascin, FSH, galectin-3, gastrin, GFAP, glucagon, glycophorin A, granzyme B, hCG, hGH, Helicobacter pylori, HBc antigen, HBs antigen, hepatocyte specific antigen, HER2, HSV-I, HSV-II, HHV-8, IgA, IgG, IgM, IGF-1R, inhibin, insulin, Kappa light chain, Ki67, lambda light chain, LH, lysozyme, macrophage, melan A, MLH-1, MSH-2, myeloperoxidase, myogenin, myoglobin, myosin, neurofilament, NSE, p27 (Kip1), p53, P63 PAX5, PLAP, Pneumocystis carini, podoplanin (D2-40), PGR, prolactin, PSA, prostatic acid phosphatase, Renal Cell Carcinoma, S100, somatostatin, spectrin, synaptophysin, TAG-72, TdT, thyroglobulin, TSH, TTF -1, TRAcP, tryptase, villin, vimentin, WT1, and Zap-70.

[Dyeing method]
Hereinafter, a method for staining tissue sections will be described. The staining method described below is not limited to a pathological section tissue and can also be applied to cell staining.
Moreover, the preparation method of the section | slice which can apply the dyeing | staining method demonstrated below is not specifically limited, The thing produced by the well-known method can be used.

1) Deparaffinization process The pathological section is immersed in a container containing xylene to remove paraffin. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, xylene may be exchanged during the immersion.
Next, the pathological section is immersed in a container containing ethanol to remove xylene. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. Further, if necessary, ethanol may be exchanged during the immersion.
Next, the pathological section is immersed in a container containing water to remove ethanol. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. Moreover, you may exchange water in the middle of immersion as needed.

2) Activation process The activation process of the target biological substance is performed according to a known method. The activation conditions are not particularly defined, but as the activation liquid, 0.01 M citrate buffer (pH 6.0), 1 mM EDTA solution (pH 8.0), 5% urea, 0.1 M Tris-HCl buffer, etc. Can be used. As the heating device, an autoclave, a microwave, a pressure cooker, a water bath, or the like can be used. The temperature is not particularly limited, but can be performed at room temperature. The temperature can be 50-130 ° C. and the time can be 5-30 minutes.
Next, the section after the activation treatment is immersed in a container containing PBS (Phosphate Buffered Saline) and washed. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, the PBS may be replaced during the immersion.

3) Staining using fluorescent substance-encapsulated nanoparticles combined with biological substance recognition sites Place PBS dispersion of fluorescent substance-encapsulated nanoparticles combined with biological substance recognition sites on a pathological section and react with the target biological material . By changing the biological material recognition site to be combined with the fluorescent substance-containing nanoparticles, staining corresponding to various biological materials becomes possible. In the case of using fluorescent substance-encapsulating nanoparticles to which several kinds of biological substance recognition sites are bound, each fluorescent substance-encapsulating nanoparticle PBS dispersion may be mixed in advance or separately placed on a pathological section separately. Also good.
The temperature is not particularly limited, but can be performed at room temperature. The reaction time is preferably 30 minutes or more and 24 hours or less.
It is preferable to drop a known blocking agent such as BSA-containing PBS before staining with fluorescent substance-encapsulating nanoparticles.
Next, the stained section is immersed in a container containing PBS, and unreacted fluorescent substance-containing nanoparticles are removed. The temperature is not particularly limited, but can be performed at room temperature. The immersion time is preferably 3 minutes or longer and 30 minutes or shorter. If necessary, the PBS may be replaced during the immersion. A cover glass is placed on the section and sealed. A commercially available encapsulant may be used as necessary.
In addition, when dyeing | staining using a HE dyeing reagent, HE dyeing is performed before enclosure with a cover glass.

(Fluorescence image acquisition)
A wide-field microscope image (fluorescence image) is acquired from the stained pathological section using the microscope image acquisition device 1A. In the microscope image acquisition apparatus 1A, an excitation light source and a fluorescence detection optical filter corresponding to the absorption maximum wavelength and fluorescence wavelength of the fluorescent material used for the staining reagent are selected.
The visual field of the fluorescent image is preferably 3 mm ² or more, more preferably 30 mm ² or more, and further preferably 300 mm ² or more.

<Relationship between fluorescent bright spot and FISH score>
Here, as described below, the Applicant produced Cy5 encapsulated silica nanoparticles (hereinafter referred to as “Nanoparticles 1”) and bound anti-HER2 antibody to the nanoparticles 1 as an example. A labeling material A was prepared. Further, CdSe / ZnS-encapsulated silica nanoparticles (hereinafter referred to as “nanoparticles 2”) were produced, and a labeling material B in which an anti-HER2 antibody was bound to the nanoparticles 2 was produced. A plurality of fluorescent images are obtained by performing immunostaining using adjacent sections of human breast tissue whose FISH score has been measured in advance using the produced labeling materials A and B and the labeling materials C and D as comparative examples, and changing the visual field. Was obtained, and the number of fluorescent bright spots appearing in each fluorescent image was measured to examine the relationship with the FISH score.

[Synthesis of phosphor-containing nanoparticles]
Synthesis Example 1: Fluorescent Organic Dye-Incorporated Silica: Synthesis of Cy5 Encapsulated Silica Nanoparticles
Cy5-encapsulated silica nanoparticles (nanoparticles 1) were prepared by the following steps (1) to (5).
Step (1): 1 mg (0.00126 mmol) of an N-hydroxysuccinimide ester derivative of Cy5 (manufactured by GE Healthcare) and 400 μL (1.796 mmol) of tetraethoxysilane were mixed.
Process (2): Ethanol 40mL and 14% ammonia water 10mL were mixed.
Step (3): The mixture prepared in Step (1) was added to the mixture prepared in Step (2) while stirring at room temperature. Stirring was performed for 12 hours from the start of addition.
Step (4): The reaction mixture was centrifuged at 10,000 G for 60 minutes, and the supernatant was removed.
Step (5): Ethanol was added to disperse the sediment, followed by centrifugation again. Washing with ethanol and pure water was performed once by the same procedure.
The obtained nanoparticles 1 were observed with a scanning electron microscope (SEM; Model S-800, manufactured by Hitachi (registered trademark)). As a result, the average particle size was 110 nm and the variation coefficient was 12%.

(Synthesis Example 2: Quantum dot-encapsulated silica: synthesis of CdSe / ZnS-encapsulated silica nanoparticles having an emission wavelength of 655 nm)
CdSe / ZnS-encapsulated silica nanoparticles (hereinafter referred to as nanoparticles 2) were produced by the following steps (1) to (5).
Step (1): 10 μL of CdSe / ZnS decane dispersion (Invitrogen Qdot655) and 40 μL of tetraethoxysilane were mixed.
Step (2): 4 mL of ethanol and 1 mL of 14% aqueous ammonia were mixed.
Step (3): The mixture prepared in Step (1) was added to the mixture prepared in Step (2) while stirring at room temperature. Stirring was performed for 12 hours from the start of addition.
Step (4): The reaction mixture was centrifuged at 10,000 G for 60 minutes, and the supernatant was removed.
Step (5): Ethanol was added to disperse the sediment, followed by centrifugation again. Washing with ethanol and pure water was performed once by the same procedure.
When the obtained nanoparticles 2 were observed with a scanning electron microscope, the average particle size was 130 nm and the coefficient of variation was 13%.

[Binding of antibody to fluorescent substance-encapsulated silica nanoparticles]
By the method of the following steps (1) to (12), an antibody was bound to the fluorescent substance-containing silica nanoparticles. Here, although the example using the nanoparticle 1 is shown, the same applies to the nanoparticle 2.
Step (1): 1 mg of nanoparticles 1 was dispersed in 5 mL of pure water. Next, 100 μL of an aminopropyltriethoxysilane aqueous dispersion was added and stirred at room temperature for 12 hours.
Step (2): The reaction mixture was centrifuged at 10,000 G for 60 minutes, and the supernatant was removed.
Step (3): Ethanol was added to disperse the sediment, followed by centrifugation again. Washing with ethanol and pure water was performed once by the same procedure.
When the FT-IR measurement of the obtained amino group-modified silica nanoparticles was performed, absorption derived from the amino group was observed, and it was confirmed that the amino group was modified.

Step (4): The amino group-modified silica nanoparticles obtained in step (3) were adjusted to 3 nM using PBS containing 2 mM of EDTA (ethylenediaminetetraacetic acid).
Step (5): SM (PEG) 12 (manufactured by Thermo Scientific, succinimidyl-[(N-maleidopropionamid) -dodecaethyleneglycol] ester) is mixed with the solution prepared in step (4) to a final concentration of 10 mM. The reaction was performed for 1 hour.
Step (6): The reaction mixture was centrifuged at 10,000 G for 60 minutes, and the supernatant was removed. Step (7): PBS containing 2 mM of EDTA was added to disperse the precipitate, and then centrifuged again. The washing | cleaning by the same procedure was performed 3 times. Finally, it was redispersed with 500 μL of PBS.

Step (8): When 100 μg of the anti-HER2 antibody was dissolved in 100 μL of PBS, 1 M dithiothreitol (DTT) was added and reacted for 30 minutes.
Step (9): Excess DTT was removed from the reaction mixture with a gel filtration column to obtain a reduced anti-HER2 antibody solution.

Step (10): Using the nanoparticle 1 as a starting material, the particle dispersion obtained in step (7) and the reduced anti-HER2 antibody solution obtained in step (9) are mixed in PBS and allowed to react for 1 hour. It was.
Step (11): 4 μL of 10 mM mercaptoethanol was added to stop the reaction.
Step (12): The reaction mixture was centrifuged at 10,000 G for 60 minutes, and the supernatant was removed. Then, PBS containing 2 mM of EDTA was added, the precipitate was dispersed, and centrifuged again. The washing | cleaning by the same procedure was performed 3 times. Finally, 500 μL of PBS was redispersed to obtain fluorescent substance-encapsulated silica nanoparticles to which the anti-HER2 antibody was bound.

  Fluorescent substance-encapsulated silica nanoparticles bound with anti-HER2 antibody obtained from nanoparticle 1 as a starting material are labeled material A, and phosphor-encapsulated silica nanoparticles bound with anti-HER2 antibody obtained from nanoparticle 2 as a starting material. This is labeled material B.

  In addition, as a comparative example, an anti-HER2 antibody was bound to Cy5 to obtain a reduced anti-HER2 antibody solution (labeling material D). Similarly, a labeling material C was prepared by binding an anti-HER2 antibody to CdSe.

[Tissue staining using fluorescent substance-containing nanoparticles]
Using the produced antibody-binding labeling materials A to D by the methods of the following steps (1) to (10), immunostaining was performed using adjacent sections of human breast tissue whose FISH score was measured in advance. As a stained section, a tissue array slide (CB-A712) manufactured by Cosmo Bio was used. Twenty-four sections with a FISH score of 1-9 were used.

Step (1): The pathological section was immersed in a container containing xylene for 30 minutes. The xylene was changed three times during the process.
Step (2): The pathological section was immersed in a container containing ethanol for 30 minutes. The ethanol was changed three times during the process.
Step (3): The pathological section was immersed in a container containing water for 30 minutes. The water was changed three times along the way.
Step (4): The pathological section was immersed in 10 mM citrate buffer (pH 6.0) for 30 minutes.
Step (5): Autoclaving was performed at 121 degrees for 10 minutes.
Step (6): The section after autoclaving was immersed in a container containing PBS for 30 minutes.
Step (7): PBS containing 1% BSA was placed on the tissue and left for 1 hour.
Step (8): Labeling materials A to D bound with anti-HER2 antibody diluted to 0.05 nM with PBS containing 1% BSA were placed on each tissue section and left for 3 hours.
Step (9): The stained sections were each immersed for 30 minutes in a container containing PBS.
Step (10): Aquex manufactured by Merck Chemicals was dropped, and then a cover glass was placed and sealed.

〔Experimental result〕
For tissue sections stained with each of the labeling materials A to D, a plurality of fluorescent images are obtained by changing the field of view (observation area), and the number of fluorescent bright spots (the number of bright spots) is obtained from each fluorescent image using image analysis software. Measured.
The microscope is an upright microscope Axio Imager M2 manufactured by Carl Zeiss, the objective lens is set to 20 times, and excitation light having a wavelength of 630 to 670 nm is irradiated to form an image of fluorescence emitted from the tissue section. Then, a fluorescence image (image data) was acquired with a microscope installation camera (monochrome), and the number of bright spots was measured with image analysis software. The camera has a pixel size of 6.4 μm × 6.4 μm, a vertical pixel count of 1040, and a horizontal pixel count of 1388 (imaging area 8.9 mm × 6.7 mm).
For each of the labeling materials A to D, the correlation coefficient R between the measured number of bright spots and the FISH score was calculated in each visual field. The FISH score corresponds to the overexpression level of the HER2 gene, and the larger the value of the FISH score, the higher the overexpression level of the HER2 gene.

FIG. 5 shows the number of bright spots measured from fluorescent images of a plurality of different visual fields (0.3 mm ² , 3 mm ² , 32 mm ² , 324 mm ² ) and FISH when using the labeling material A (Cy5 inclusion labeling material). It is a figure which shows the relationship with a score. The value of R2 shown in the figure is the square value of the correlation coefficient between the number of bright spots and the FISH score.

FIG. 6 shows the number of bright spots measured from fluorescent images of a plurality of different visual fields (0.3 mm ² , 3 mm ² , 32 mm ² , 324 mm ² ) and FISH when the labeling material B (CdSe inclusion labeling material) is used. It is a figure which shows the relationship with a score.

FIG. 7 shows the number of bright spots measured from fluorescence images of a plurality of different visual fields (0.3 mm ² , 3 mm ² , 32 mm ² , 324 mm ² ) and the FISH score when the labeling material C (CdSe) is used. It is a figure which shows a relationship.

FIG. 8 shows the number of bright spots measured from fluorescent images of a plurality of different visual fields (0.3 mm ² , 3 mm ² , 32 mm ² , 324 mm ² ) and the FISH score when the labeling material D (Cy5) is used. It is a figure which shows a relationship.

Table 1 and FIG. 9 show the square value (R ² ) of the correlation coefficient between the number of bright spots and the FISH score measured from the fluorescence image of each visual field (observation area) for each of the labeling materials A to D.

When a tissue section is stained with the labeling material A and the number of bright spots is measured from a fluorescent image of a visual field of 0.3 mm ² , the square value (R ² ) of the correlation coefficient between the number of bright spots and the FISH score is The correlation was not found between the number of bright spots and the FISH score. This is probably because the field of view of 0.3 mm ² is too narrow and the variation is large.

When a tissue section is stained with the labeling material A and the number of bright spots is measured from a fluorescence image of a 3 mm ² visual field, the square value (R ² ) of the correlation coefficient between the number of bright spots and the FISH score is 0. 5387. This value is approximately 0.734 when converted to the correlation coefficient R, and it can be said that there is a strong correlation between the number of bright spots and the FISH score.

When a tissue section is stained with the labeling material A and the number of bright spots is measured from a fluorescence image of a field of view of 32 mm ² , the square value (R ² ) of the correlation coefficient between the number of bright spots and the FISH score is 0. 9011. When the visual field is 32 mm ² , it can be said that the correlation is stronger than when the visual field is 3 mm ² .

When a tissue section is stained with the labeling material A and the number of bright spots is measured from a fluorescent image of a field of view of 324 mm ² , the square value (R ² ) of the correlation coefficient between the number of bright spots and the FISH score is 0. It was 9887. When the field of view is 324 mm ² , it can be said that the correlation is stronger than when the field of view is 32 mm ² .

Similarly, when the labeling material B was used, it was found that there was a correlation between the number of bright spots and the FISH score in the field of view of 3 mm ² or more, and the correlation coefficient increased as the field of view increased.

Further, from the results when the labeling materials A and B were used, it was found that the square value (R ² ) of the correlation coefficient between the number of bright spots and the FISH score was sufficiently close to 1 in the field of view of 324 mm ² .

  On the other hand, when the tissue section was stained using the labeling material C or the labeling material D, no correlation was found between the number of bright spots and the FISH score.

  Moreover, even when the focus of the microscope was slightly shifted above or below the thickness (usually several μm) of the tissue section to be observed, no significant difference was observed in the above situation.

From the above results, when the tissue sections are observed with a wide field of view using the labeling materials A and B, the correlation between the number of bright spots and the FISH score is good, and the expression level of HER2 is evaluated based on the number of bright spots. Was found to be possible. That is, without using a time-consuming method such as the FISH method, a tissue section is stained with a fluorescent substance-encapsulating nanoparticle staining reagent to which a biological substance recognition site that recognizes a specific biological substance is bound. By measuring the number of bright spots from an image with a field of view of 3 mm ² or more obtained by taking an enlarged image, it is possible to evaluate the expression level of a specific biological substance, which is an effective alternative to the FISH method. is there.

  Since the labeling materials A and B use particles containing a fluorescent substance and are brighter than the labeling materials C and D using a single fluorescent substance, each point of the bright spot is captured from the image. Easy to measure the number of bright spots with high accuracy.

<Operation of Pathological Diagnosis Support System 100 (Including Pathological Diagnosis Support Method)>
Hereinafter, an operation of acquiring and analyzing the above-described fluorescence image and bright field image in the pathological diagnosis support system 100 will be described. Here, a staining reagent including fluorescent substance-encapsulating nanoparticles bound to a biological substance recognition site that recognizes a specific protein (here, Ki67 protein or HER2 protein in breast cancer tissue, hereinafter referred to as a specific protein) is used. However, the present invention is not limited to this example.

First, the operator uses two types of staining reagents, a HE staining reagent and a staining reagent using fluorescent substance-encapsulated nanoparticles bound with a biological substance recognition site that recognizes a specific protein as a fluorescent labeling material. Stain.
Thereafter, in the microscope image acquisition apparatus 1A, a bright field image and a fluorescence image are acquired by the procedures (a1) to (a5).
(A1) The operator places the tissue sections stained with the HE staining reagent and the staining reagent containing fluorescent substance-containing nanoparticles on the slide, and places the slide on the slide fixing stage of the microscope image acquisition apparatus 1A. .
(A2) Set to the bright field unit, adjust the imaging magnification and focus, and place the observation target area on the tissue in the field of view.
(A3) Shooting is performed by the imaging unit to generate bright field image data, and the image data is transmitted to the image processing apparatus 2A.
(A4) Change the unit to a fluorescent unit.
(A5) Shooting is performed by the imaging means without changing the field of view and the shooting magnification to generate image data of a fluorescent image, and the image data is transmitted to the image processing apparatus 2A.

According to the study by the inventors of the present application, when HE staining and staining with fluorescent substance-containing nanoparticles are performed at the same time, the tissue autofluorescence and eosin emission (background) are 10% (1.1 times). As long as the fluorescence emission point of the phosphor-containing nanoparticle has the above-mentioned difference in light emission amount, the microscope can be used in any processing system of 8 bits (0 to 255 gradations) and 12 bits (0 to 4095 gradations). It was possible to automatically detect fluorescent luminescent spots from an image (fluorescent image). In the case of only staining with fluorescent substance-containing nanoparticles, if the fluorescent substance-containing nanoparticles have a difference in light emission amount of 10% (1.1 times) or more with respect to the autofluorescence of the tissue, 8 bits (0 ˜255 gradations) and 12-bit (0-4095 gradations) processing system can automatically detect fluorescent luminescent spots. Therefore, it is preferable to select the excitation light wavelength in the fluorescent unit in the range of 560 to 630 nm. Further, as the fluorescent substance, it is preferable to use a substance that emits fluorescence having a peak in the range of 580 to 690 nm, more preferably in the range of 600 to 630 nm by the excitation light. If a fluorescent substance having a peak in this range is selected, when the excitation light in the above range is selected, the difference between the autofluorescence of the tissue including the emission of eosin and the fluorescence from the fluorescent substance-containing nanoparticles is secured. This is because it is possible to ensure that they can be distinguished and recognized (difference of 10% (1.1 times) or more).
In the case where HE staining is not performed at the same time, the autofluorescence of the tissue is weak. Therefore, the wavelength range of the excitation light is not particularly limited to a general range of 200 nm to 700 nm, and the autofluorescence and fluorescent substance-containing nanoparticles are not particularly limited. Thus, it is possible to secure a difference in fluorescence emission from the light source so that both can be distinguished and recognized (a difference in light quantity between the two is 10% (1.1 times or more)).

In the image processing apparatus 2A, image analysis processing is executed based on the bright field image and the fluorescence image.
FIG. 10 shows a flowchart of image analysis processing in the image processing apparatus 2A. The image analysis processing shown in FIG. 10 is executed in cooperation with the control unit 21 and a program stored in the storage unit 25.

First, when a bright field image is input from the microscope image acquisition apparatus 1A through the communication I / F 24 (step S1), a nucleus region indicating a cell nucleus region is extracted from the bright field image (step S2).
FIG. 11 shows a detailed flow of the process in step S2. The process of step S2 is executed in cooperation with the control unit 21 and the program stored in the storage unit 25.

In step S2, first, a bright-field image is converted into a monochrome image (step S201). FIG. 12A shows an example of a monochrome image.
Next, threshold processing is performed on the monochrome image using a predetermined threshold, and the value of each pixel is binarized (step S202). FIG. 12B shows an example of a binary image after threshold processing.

  Next, noise processing is performed (step S203). Specifically, the noise process can be performed by performing a closing process on the binary image. The closing process is a process in which the contraction process is performed the same number of times after the expansion process is performed. The expansion process is a process of replacing a target pixel with white when at least one pixel in the range of n × n pixels (n is an integer of 2 or more) from the target pixel is white. The contraction process is a process of replacing a target pixel with black when at least one pixel in the range of n × n pixels from the target pixel contains black. By the closing process, a small area such as noise can be removed. FIG. 12C shows an example of an image after noise processing. As shown in FIG. 12C, after noise processing, an image (cell nucleus image) from which cell nuclei are extracted is obtained.

Next, a labeling process is performed on the image after the noise process, and a label Label_nucleus is assigned to each of the extracted cell nuclei (step S204). The labeling process is a process for identifying an object in an image by assigning the same label (number) to connected pixels. By labeling, each cell nucleus can be identified from the image after noise processing and a label can be applied.
In addition, in order to distinguish from the label number in the fluorescence point extraction described later, if the maximum value that can be held by the computer is MAX, and the number of labeling performed so far is Label_temp, the label Label_nucleus is added to the new cell nucleus as MAX. -Label_temp is assigned. For example, when labeling the 101st cell nucleus, since Label_temp = 100, when MAX = 65536, 65436 is assigned as Label_nucleus.

On the other hand, when a fluorescent image is input from the microscope image acquisition apparatus 1A through the communication I / F 24 (step S3), fluorescent bright spots are extracted from the fluorescent image (step S4).
FIG. 13 shows a detailed flow of the process in step S4. The process of step S4 is executed in cooperation with the control unit 21 and the program stored in the storage unit 25.

First, the R component is extracted from the fluorescence image (step S401).
Next, Topat conversion is performed on the image from which the R component has been extracted (step S402). The Tophat conversion is a process of subtracting the value of the corresponding pixel of the image obtained by applying the minimum value filter and the maximum value filter to the input image in this order from the value of each pixel of the input image. The minimum value filter replaces the value of the target pixel with the minimum value of pixels in the vicinity of the target pixel (for example, 3 × 3 pixels). The maximum value filter replaces the value of the target pixel with the maximum value among the pixels in the vicinity of the target pixel (for example, 3 × 3 pixels). By the Tophat conversion, small protrusions (areas with higher brightness than neighboring pixels) on the light and shade profile can be extracted. Thereby, a fluorescent luminescent spot candidate image can be obtained. FIG. 14A shows an example of the fluorescent bright spot candidate image.

  Next, noise is removed from the fluorescent luminescent spot candidate image to obtain an image (fluorescent luminescent spot image) from which fluorescent luminescent spots are extracted (step S403). FIG. 14B shows a fluorescent luminescent spot image obtained after noise removal from the fluorescent luminescent spot candidate image shown in FIG. 14A.

  Then, a labeling process is performed on the image after noise removal, and a label Label_point is assigned to each of the extracted fluorescent luminescent spots (step S404). Label_point is assigned in order from 1. After the labeling process ends, the process proceeds to step S5 in FIG.

In step S <b> 5, the control unit 21 aligns each image of the bright field image and the fluorescence image based on the cell morphology observed in the bright field image and the fluorescent bright spot observed in the fluorescence image.
The bright field image used for the alignment is the nuclear region image obtained in step S2 (see FIG. 12C), and the fluorescent image used for the alignment is the fluorescent bright spot image obtained in step S4 (see FIG. 14B). ). In step S5, at least one of bright field image and fluorescent image is subjected to coordinate transformation using the following expression to perform at least one image processing of translation, rotation, enlargement, or reduction. The control unit 21 changes the parameters dx, dy, θ, α, and β to search for a combination of parameters (optimal parameters) that best matches the positions of the bright field image and the fluorescence image.

(X, y): Bright spot coordinates after conversion (dx, dy): Parallel movement amount θ: Rotational movement distance (α, β): Zoom ratio (x ₀ , y ₀ ): Bright spot coordinates before conversion

  In the search for the optimum parameter, the determination that the bright-field image and the fluorescence image are in position may be made by setting an arbitrary standard according to the type of the specific protein and the tissue specimen. Any of the determination methods can be used. Note that the optimum parameter search method may be a brute force search or an iterative search method such as ICP.

(1-1) Determination method 1 when the specific protein is a nucleoprotein
In the case where the specific protein is a nuclear protein expressed in the cell nucleus such as Ki67, for example, when the number of bright spots existing inside the stained cell nucleus is maximum, the control unit 21 performs bright field image and fluorescence. Judge that the position of the image is correct.
Specifically, for example, it is determined whether or not the fluorescent luminescent spot extracted in step S4 overlaps any of the nuclear regions extracted in step S2, and if it overlaps, the score of the luminescent spot is +1, If they do not overlap, the score of the bright spot is set to 0, and the total score of the entire image is calculated. In the schematic diagram of FIG. 15 (a), 13 out of the 16 fluorescent bright spots 30 represented by ● overlap with either one of the nuclear regions 31A or 31B, so The total score of bright spots is calculated as 13. Further, in FIG. 15B in which the fluorescent luminescent spots in FIG. 15A are translated, the total score is calculated as 9. By searching for a parameter that maximizes the sum of the scores and performing coordinate conversion using it as an optimum parameter, the bright-field image and the fluorescence image are accurately aligned.

(1-2) Determination method 2 when the specific protein is a nucleoprotein
When the specific protein is a nucleoprotein, the control unit 21 determines that the bright field image and the fluorescence image are aligned when the sum of the distances between the stained cell nucleus and the bright spot is minimized, for example. You may judge.
Specifically, first, for each fluorescent luminescent spot extracted in step S4, the distance from the nuclear region extracted in step S4 is calculated, and the closest nuclear region is searched and associated with the fluorescent luminescent spot. The definition of the distance to the nucleus region is arbitrary. For example, the distance between the centroid 32 of the nucleus region and the corresponding fluorescent bright spot 30 as shown by the dotted line in FIG. 15C, or the nucleus regions 31A and 31B. The distance between the contour line and each fluorescent luminescent spot 30 can be used.
By performing coordinate conversion using the parameter that minimizes the sum of the distances from the cell nuclei calculated for all fluorescent luminescent spots as the optimum parameter, the bright field image and the fluorescent image are accurately aligned.

The method for calculating the total sum of distances is arbitrary, but for example, the following method can be used.
If the simple sum obtained by adding the absolute values of the distances is used as the sum, the sum can be calculated by the simplest calculation method.
Further, if the sum of the squares of the distances is taken as the sum, the longer the distance, the greater the influence of the sum when the parameters are changed. Therefore, a parameter that greatly reduces the number of fluorescent bright spots that deviate from the region can be set as the optimum parameter. .
In addition, an average value of distances from all associated fluorescent luminescent spots may be calculated for each nucleus region, and a simple sum or a square sum of the average values may be used as the sum. As a result, even when there is a large variation in the number of fluorescent luminescent spots corresponding to the nucleus region, it is possible to calculate the sum total that equally considers all the nucleus regions.
Further, each distance may be weighted by the reciprocal of the area of the corresponding nucleus region, and the simple sum or the square sum may be used as the sum.
Also, each distance may be weighted by the number of fluorescent bright spots associated with the corresponding nucleus region, and the simple sum or the square sum may be used as the sum. As a result, a nucleus region having a large number of associated fluorescent luminescent spots can be preferentially used for alignment.

(2-1) Determination method 1 when the specific protein is a membrane protein
When the specific protein is a membrane protein expressed on the cell membrane such as HER2, it is difficult to directly extract the cell membrane to which the membrane protein is attached from the bright field image. Therefore, in the tissue section obtained by slicing the tissue specimen, the control unit 21 is based on the fact that the fluorescent bright spots indicating the expression of the membrane protein are considered to be distributed outside the nucleus region without overlapping the nucleus region. When the number of bright spots existing inside the stained nucleus region is minimized, it is determined that the positions of the bright field image and the fluorescent image are in alignment.
Specifically, a score based on the overlap between the nucleus region and the fluorescent bright spot is calculated in the same manner as in the method (1-1) described above. By searching for a parameter that minimizes the sum of the scores of all bright spots in the fluorescent image and performing coordinate conversion using the optimum parameter, the bright-field image and the fluorescent image are aligned with high accuracy.

(2-2) Determination method 2 when the specific protein is a membrane protein
When the specific protein is a membrane protein, the control unit 21 aligns the bright-field image and the fluorescence image when, for example, the sum of the distance between the outline of the stained nuclear region and the bright spot is minimized. It may be determined that
Specifically, first, for each fluorescent luminescent spot extracted in step S4, the distance from the outline of the nuclear region extracted in step S2 is calculated, and the closest nuclear region is searched and associated with the fluorescent luminescent spot.
The bright field image and the fluorescence image are accurately obtained by calculating the distance from the contour of the corresponding nuclear region for all the bright spots, searching for the parameter that minimizes the sum, and performing coordinate conversion using it as the optimum parameter. To be aligned. In FIG. 16A, ● represents the fluorescent bright spot 30 associated with the nucleus regions 31C and 31D, and the length of the dotted line represents the distance to the contour of the corresponding nucleus region. When there is a fluorescent bright spot that overlaps the nucleus area as in the nucleus area 31D, the distance between the fluorescence starting point that overlaps each area and the outline of the nucleus area is replaced with a predetermined fixed value, and the sum is calculated. Also good. If a predetermined fixed value is set to a large value, if there is a fluorescent bright spot that overlaps the nucleus region, the sum of the distances between the fluorescent bright point and the contour of the nuclear region always increases. There is a low possibility that a parameter in the case of overlapping with an area is determined as an optimum parameter.

  In addition, the arbitrary methods similar to (1-2) mentioned above can be used for calculation of sum total.

(2-3) Determination method 3 when the specific protein is a membrane protein
In addition, when it is assumed that the specific protein is a membrane protein and the cell nucleus is present at substantially the center of the cell, the control unit 21 displays the center of gravity 32 of the nucleus region 31C as shown in FIG. For example, the sum of the standard deviation of the distance between the center of gravity of the nuclear region and the fluorescent luminescent spot extracted in step S2 is minimized, assuming that the distance from the fluorescent luminescent spot associated with the nuclear region is substantially constant. Sometimes it may be determined that the bright field image and the fluorescence image are in alignment.
Specifically, first, for each fluorescent luminescent spot extracted in step S4, the distance from the center of gravity of the nuclear region extracted in step S2 is calculated, and the closest nuclear region is searched and associated with the fluorescent luminescent spot. Next, for each nucleus region, the standard deviation of the distance from the associated fluorescent bright spot is calculated, and by performing coordinate conversion using the parameter that minimizes the sum of the standard deviations as the optimum parameter, The fluorescence image is accurately aligned.

In addition, although the calculation method of the sum total of a standard deviation is arbitrary, the following methods can be used, for example.
If the simple sum obtained by adding the absolute values of the standard deviation is defined as the sum, the sum can be calculated by the simplest calculation method.
If the sum of the squares of the standard deviation is the sum, the larger the standard deviation, the greater the influence on the sum. Therefore, searching for the optimum parameter that minimizes the sum can reduce the core region where the standard deviation is large. it can.
Alternatively, the standard deviation may be weighted by the reciprocal of the area of the core region, and the simple sum or the square sum may be used as the sum.
Further, the standard deviation may be weighted by the number of fluorescent luminescent spots corresponding to the nucleus region and used for calculating the sum. Further, a nucleus region in which the number of fluorescent bright spots associated with the nucleus region is smaller than a predetermined value may not be used for calculating the total standard deviation. Thereby, a nucleus area | region with many associated fluorescence luminescent spots can be used for position alignment preferentially.

Note that in the alignment in step S5 described above, when the bright-field image and the fluorescence image are acquired by the method of the present embodiment, there is usually little image shift due to the optical axis shift or vibration of the microscope. Therefore, in order to prevent the optimum parameter from being erroneously recognized due to excessive image processing, it is preferable to provide a predetermined limit value for each parameter.
By providing a limit value for each parameter, for example, when the biological material is a nuclear protein, the parameter when all the fluorescent spots overlap one nuclear region due to excessive enlargement of the bright field image is the optimal parameter. In addition, when the biological material is a membrane protein, it prevents false recognition that the parameter when all fluorescent luminescent spots are outside the nucleus region due to excessive translation of one image is the optimum parameter. be able to.

In addition, when the specific protein is a nucleoprotein, when the bright-field image and the fluorescence image are superimposed after alignment, the fluorescent bright spot existing outside the nucleus region is a noise that is not related to the expression of the specific protein. It may be deleted as if it were present and not used in the processing after step S6.
In addition, when the specific protein is a membrane protein, when the bright field image and the fluorescent image are superimposed after alignment, the fluorescent bright spot that overlaps the nucleus region is noise that is unrelated to the expression of the specific protein. It is good also as not deleting it considering that it deletes, and processing after Step S6.

After completion of the alignment in the step S5, the bright field image (nucleus region image obtained in step S2) and the fluorescence image (fluorescent bright spot image obtained in step S4) superimposed and aligned are aligned. Thus, the expression status of the specific protein in each nucleus region is determined (step S6).
Specifically, in a state where the bright field image and the fluorescence image are aligned and superimposed, for example, the number of fluorescent bright spots corresponding to each of the nuclear regions extracted in step S2 is calculated, and the malignancy and progression of cancer are calculated. It is an indicator.
Statistical values used for determining malignancy and treatment planning are not particularly limited. For example, a value obtained by dividing the total number of fluorescent bright spots corresponding to the nuclear region by the number of nuclear regions, or a fluorescent bright spot corresponding to the nuclear region. In addition, the ratio of the number of nuclei regions having a certain number or more of fluorescent luminescent spots is preferably used.

Next, an analysis result screen 231 is generated and displayed on the display unit 23 based on the expression state of the specific protein corresponding to the determined nuclear region (step S7).
For example, in step S7, based on whether or not the number of fluorescent bright spots corresponding to one nuclear region exceeds a plurality of predetermined thresholds, the expression status of the specific protein is classified into a plurality of stages, and the high expression region, They are classified in the form of medium expression area, low expression area, and extremely low expression area. Thereafter, an image (protein expression status display image) classified (for example, color-coded) in a different display mode according to the classification result is generated on the bright field image, and is displayed and output on the display unit 23 as the analysis result screen 231. . In the protein expression status display image, for example, the nuclear region portion may be color-coded according to the division, or a region including a fluorescent luminescent spot corresponding to each nuclear region is created, and the nuclear region is displayed. It may be displayed in different colors according to the classification.

FIG. 17 shows an example of the analysis result screen 231. As shown in FIG. 17, the analysis result screen 231 displays a bright field image 231a, a fluorescent bright spot image 231b, and a protein expression status display 231c.
In the protein expression state display image 231c, a high expression region, a medium expression region, a low expression region, and an extremely low expression region of a specific protein are displayed in a color-coded manner on the bright field image. Because it is displayed in different colors according to the expression status of specific proteins, doctors can efficiently grasp the overexpression and spread of specific proteins that are an indicator of cancer malignancy, and make appropriate treatment plans. It is possible to stand.

The analysis result screen 231 is not limited to that shown in FIG.
For example, only the protein expression status display image 231c may be displayed. Further, the bright field image 231a and the protein expression status display image 231c may be switched and displayed in response to a switching instruction from the operation unit 22. The bright field image 231a, the fluorescent bright spot image 231b, and the protein expression status display image 231c displayed on the analysis result screen 231 may be displayed in an enlarged or reduced manner so that they can be easily observed.
Further, an image obtained by simply superimposing the bright field image 231a and the fluorescent bright spot image 231b may be displayed in accordance with an instruction from the operation unit 22, and in such a case, the specific protein may be displayed based on the superimposed image. It is also possible to visually present the state of the occurrence of the disease to a doctor and prompt judgment.

The analysis result can be printed out or output to an external device by pressing the print button 231d or the send button 231e.
When the print button 231d is pressed by the operation unit 22, the data of the analysis result is transmitted by the control unit 21 to a printer (not shown) via a communication network such as the communication I / F 24 or LAN, and the analysis result is printed out. When the transmission button 231e is pressed by the operation unit 22, the control unit 21 transmits the analysis result data to an external device (for example, PACS (Picture Archiving and Communication System for PACS) via a communication network such as the communication I / F 24 or a LAN). medical application)).
In addition, by operating the microscope image acquisition device 1A to change the field of view range and magnification, even in the case of fluorescent images and bright field images continuously acquired from the same specimen, due to vibration during operation to shift the field of view range, Since irregular deviation may occur and the degree of deviation may vary depending on the visual field range, it is preferable to perform the alignment shown in the flowchart of FIG. 5 each time the visual field range is changed.

  According to the present embodiment described above, at least one of translation, rotation, enlargement, or reduction is performed on at least one of the fluorescence image and the bright field image, and the expression region of the specific protein is specified. By performing the alignment, the fluorescent luminescent spots of the fluorescent image are accurately superimposed on the cells whose form is shown in the bright field image. That is, according to the method of the present invention, in addition to the staining usually performed in the conventional diagnosis, the bright field image and the fluorescence image can be obtained without providing a marker substance or separately providing a step for fluorescent staining of the cell membrane. Alignment is possible. By aligning the bright field image and the fluorescence image, the expression state of the specific protein in the cell can be accurately diagnosed.

  Further, if a predetermined limit value is provided for image processing for translation, enlargement, or reduction, it is possible to prevent the optimum parameter from being erroneously recognized due to excessive image processing.

  In addition, even when the specific protein is expressed in the cell membrane, which is difficult to automatically extract from the cell morphology image, based on the positional relationship between the nuclear region and the fluorescent bright spot that can be easily extracted from the cell morphology image, Alignment is possible.

In addition, when a specific protein is expressed in the cell nucleus, the bright spots that exist outside the nucleus region after alignment are considered to be noise and are deleted, so that noise is removed and accurate diagnosis is performed. Can do.
In addition, when a specific protein is expressed on the cell membrane, the bright spot existing in the nuclear region after positioning is considered to be noise and deleted, so that noise is removed and accurate diagnosis is performed. Can do.

  Further, if the alignment is performed based on the number of fluorescent bright spots in the nucleus region, the alignment process is relatively simple and the calculation amount is small, so that the measurement can be easily performed in a short time. In addition, all fluorescent luminescent spots outside the nuclear region are treated equally as being not fluorescent luminescent spots within the nuclear region, regardless of the distance from the nuclear region. ) Exists outside the nucleus region, it is difficult to affect the number of fluorescent luminescent spots in the nucleus region even if the positional relationship between the noise and the nucleus region changes due to image processing such as translation, rotation, enlargement, or reduction of the image. . That is, the alignment based on the number of fluorescent bright spots in the nucleus region is considered to be an alignment method that is not easily affected by noise.

  In addition, by performing alignment based on the distance from the fluorescent luminescent spot to the nuclear area closest to each fluorescent luminescent spot, for example, fluorescent luminescent spots that are considered to exist outside the nuclear area due to uneven staining of cell nuclei. Even in the case where there is, the alignment is performed taking into account the distance from the nucleus region with respect to all the fluorescent luminescent spots, so that the alignment can be performed with high accuracy.

  In addition, in cells with cell nuclei in the approximate center of the cell, the distance from the fluorescence bright spot on the cell membrane to the center of gravity of the nucleus region is almost constant, and the alignment is performed with high accuracy based on the small standard deviation. be able to.

  Further, by assigning weights based on the number of fluorescent luminescent spots corresponding to the nucleus region and the size of the nucleus region, it is possible to appropriately select a preferred nucleus region as a diagnosis target and perform alignment.

  In addition, when acquiring a bright field image and a fluorescence image from a plurality of visual field ranges of the same specimen, for example, irregular shift may occur due to vibrations when changing the visual field. If alignment is performed, it is possible to perform alignment for all acquired bright field images and fluorescent images.

  As for the expression status of a specific protein, the expression status of the specific protein is classified into a plurality of stages based on whether or not the number of fluorescent bright spots corresponding to one nuclear region exceeds a plurality of predetermined thresholds, An image divided in a manner according to the classification result is output as an analysis result. Therefore, the nuclear region is displayed in different display modes depending on the expression status of the specific protein, and the doctor can efficiently grasp the overexpression of the specific protein, which is an index of cancer malignancy, and its spread, It is possible to make an appropriate treatment plan.

In addition, the description content in the said embodiment is a suitable example of this invention, and is not limited to this.
For example, in the above-described embodiment, Ki67 protein and HER2 protein in breast cancer are exemplified as specific proteins, but the specific protein is not limited thereto. Depending on the type of lesion (cancer) to be diagnosed, if the biological material recognition site when acquiring a fluorescence image is different, the feature quantity that quantitatively indicates the expression level of the specific protein corresponding to the lesion type It can be provided to a doctor.

In the above description, an example in which an HDD or a semiconductor nonvolatile memory is used as a computer-readable medium for the program according to the present invention is disclosed, but the present invention is not limited to this example. As another computer-readable medium, a portable recording medium such as a CD-ROM can be applied. Further, a carrier wave (carrier wave) is also applied as a medium for providing program data according to the present invention via a communication line.

  In addition, the detailed configuration and detailed operation of each device constituting the pathological diagnosis support system 100 can be changed as appropriate without departing from the spirit of the invention.

1A Microscope image acquisition device 2A Image processing device 3A Cable 21 Control unit (positioning means, calculating means, determining means, deleting means)
22 Operation part 23 Display part 24 Communication I / F (input means)
25 storage unit 26 bus 30 fluorescent bright spots 31A, 31B, 31C, 31D nucleus region 32 center of gravity of nucleus region 100 pathological diagnosis support system

Claims (18)

An input means for inputting a fluorescence image representing the expression of a specific biological substance in a specimen with a fluorescent bright spot, and a cell morphology image representing a cell morphology based on an information source different from the fluorescent bright spot;
An alignment means for aligning the cell morphology image and the fluorescent image based on a positional relationship between the cell morphology in the cell morphology image and the fluorescent bright spot in the fluorescence image;
An image processing apparatus comprising:   The alignment means performs alignment by performing at least one image processing of translation, rotation, enlargement, or reduction on at least one of the cell morphology image and the fluorescence image. The image processing apparatus according to claim 1.   The image processing apparatus according to claim 1, wherein the cell morphology in the cell morphology image is a nucleus region indicating a region of a stained cell nucleus.   The image processing apparatus according to claim 3, wherein the specific biological substance is expressed in a cell nucleus.   The image processing apparatus according to claim 3, wherein the specific biological substance is expressed in a cell membrane. Having a calculation means for calculating the number of the fluorescent bright spots overlapping the nucleus region;
The image processing apparatus according to claim 4, wherein the alignment unit performs alignment so that the number of the fluorescent bright spots is maximized. Having a calculation means for calculating the number of the fluorescent bright spots overlapping the nucleus region;
The image processing apparatus according to claim 5, wherein the alignment unit performs alignment so that the number of fluorescent bright spots is minimized. Based on the positional relationship between the nuclear region and the fluorescent luminescent spot, there is a determining means for determining which of the nuclear regions each of the fluorescent luminescent points corresponds to,
5. The alignment means calculates a distance between the fluorescent luminescent spot and a centroid of a corresponding nuclear region, and performs alignment so that a total sum of the distances with the centroid is minimized. An image processing apparatus according to 1. Based on the positional relationship between the nuclear region and the fluorescent luminescent spot, there is a determining means for determining which of the nuclear regions each of the fluorescent luminescent points corresponds to,
The alignment means calculates the shortest distance between the fluorescent luminescent spot and the contour of the corresponding nuclear region, and if the fluorescent luminescent spot overlaps the corresponding nuclear region, the shortest distance from the contour is calculated. 6. The image processing apparatus according to claim 5, wherein after the replacement with a predetermined fixed value, alignment is performed so that the sum of the shortest distances to the contour is minimized. Based on the positional relationship between the nuclear region and the fluorescent luminescent spot, there is a determining means for determining which of the nuclear regions each of the fluorescent luminescent points corresponds to,
The alignment means calculates the distance between the fluorescent luminescent spot and the center of gravity of the corresponding nuclear region, calculates the standard deviation of the distance from the center of gravity for each of the core regions, and the sum of the standard deviations is the smallest The image processing apparatus according to claim 5, wherein alignment is performed so that   The alignment means performs alignment so that the total sum of the standard deviations calculated in the nucleus region in which the number of corresponding fluorescent bright spots is larger than a predetermined value is minimized. The image processing apparatus according to 10.   12. The alignment according to claim 2, wherein the alignment unit performs alignment by performing parallel movement within a range of a predetermined limit value on at least one of the cell morphology image and the fluorescence image. The image processing apparatus according to any one of the above.   13. The alignment unit performs alignment by performing enlargement or reduction within a predetermined limit value range on at least one of the cell morphology image and the fluorescence image. The image processing apparatus according to any one of the above.   9. The image processing according to claim 4, further comprising a deletion unit that deletes the fluorescent bright spot that does not overlap the nucleus region after the alignment by the alignment unit. apparatus.   The image processing apparatus according to claim 1, wherein the alignment unit performs alignment each time the cell morphology image and the fluorescence image are acquired. The image processing apparatus according to any one of claims 1 to 15,
An image acquisition device used in the image processing device for acquiring the cell morphology image and the fluorescence image;
An image processing system comprising: Computer
An input means for inputting a fluorescence image representing the expression of a specific biological substance in a specimen with a fluorescent bright spot and a cell morphology image representing a cell morphology based on an information source different from the fluorescent bright spot;
An alignment means for aligning the cell morphology image and the fluorescent image based on a positional relationship between the cell morphology in the cell morphology image and the fluorescent bright spot in the fluorescence image;
Image processing program to function as An input step of inputting a fluorescence image representing the expression of a specific biological substance in a specimen with a fluorescent bright spot, and a cell morphology image representing a cell morphology based on an information source different from the fluorescent bright spot;
An alignment step of aligning the cell morphology image and the fluorescent image based on the positional relationship between the cell morphology in the cell morphology image and the fluorescent bright spot in the fluorescence image;
An image processing method comprising: