WO2020262117A1 - Image processing system, image processing method, and program - Google Patents

Abstract

According to this image processing system 1, a control unit 61 is configured to acquire histological images captured of a tissue specimen, and to select, from among the acquired histological images, an image of an area of interest for the purpose of re-imaging. The control unit 61 acquires a cell-type identification image for identifying at least one cell type in a tissue specimen 50 and at least one cell analysis image for analyzing the identified cells, and then selects an area of interest on the basis of a feature quantity pertaining to the cell type and being calculated from the cell-type identification image and a feature quantity pertaining to at least one target substance and being calculated from the cell analysis image.

Description

Image processing system, image processing method and program

The present invention relates to an image processing system, an image processing method and a program.

In recent years, virtual microscopes have been attracting attention as advanced technology in fields such as pathology. The virtual microscope is a system that digitizes an image observed by an optical microscope and allows the tissue sample to be observed on a display as if the optical microscope was actually used (see, for example, Patent Document 1).

In detail, the entire tissue sample on the slide glass is photographed, the obtained image is converted into digital data and saved in a database, and observation is performed using viewer software installed on a personal computer or the like. At this time, the reason why it is called a virtual microscope is that it can be observed while performing operations such as moving up and down, left and right, and scaling, as in the case of observation using an optical microscope. Digital image data of the entire tissue specimen, called a whole slide image (WSI), is stored in a database. Since the stored digital image data can be accessed via the Internet or the like, for example, a rapid pathological diagnosis by a pathologist at a remote location and a rare tissue specimen can be made available to anyone.

JP-A-2018-72253

By the way, in order to perform an analysis for diagnosis or the like, the image quality that can withstand the analysis must be guaranteed. Therefore, in order to perform highly accurate analysis, it is necessary to take a tissue sample at a high magnification.
However, as described in Patent Document 1, when the entire pathological specimen is photographed in a high-magnification field of view, the capacity of the acquired image becomes enormous, and analysis using the image requires an enormous amount of time. However, the efficiency of analysis is reduced.

The present invention has been made in view of the above problems, and an object of the present invention is to provide an image processing system, an image processing method, and a program capable of efficiently performing high-precision analysis.

In order to solve the above problems, the image processing system of the present invention
An acquisition method for acquiring a tissue image of a tissue sample,
A selection means for selecting an image of a region of interest for re-imaging by the acquisition means from the tissue image acquired by the acquisition means.
Have,
The acquisition means acquires a cell type identification image for identifying one or more cell types in the tissue specimen and one or more cell analysis images for analyzing the identified cells.
The selection means selects the region of interest based on the feature amount related to the cell type calculated from the cell type identification image and the feature amount related to one or more target substances calculated from the cell analysis image. To do.

Moreover, the image processing method of the present invention
The acquisition process to acquire the tissue image of the tissue sample,
A selection step of selecting an image of a region of interest for rephotographing in the acquisition step from the tissue image acquired in the acquisition step, and a selection step.
Have,
In the acquisition step, an image for identifying one or more cell types in the tissue sample and an image for analyzing one or more cells for analyzing the identified cells are acquired.
The selection step is based on the feature amount related to one or more cell types calculated from the cell type identification image and the feature amount related to one or more target substances calculated from the cell analysis image. Select the area of interest.

In addition, the program of the present invention
Computer,
Acquisition method for acquiring tissue images of tissue specimens,
A selection means for selecting an image of a region of interest for re-imaging by the acquisition means from the tissue image acquired by the acquisition means.
To function as
The acquisition means acquires a cell type identification image for identifying one or more cell types in the tissue specimen and one or more cell analysis images for analyzing the identified cells.
The selection means is based on the feature amount related to one or more cell types calculated from the cell type identification image and the feature amount related to one or more target substances calculated from the cell analysis image. This is a program that selects the area of interest.

According to the present invention, it is possible to efficiently perform high-precision analysis, that is, it is possible to simultaneously improve the accuracy and efficiency of analysis.

It is a figure which shows the schematic structure of the image processing system which concerns on this invention. It is a figure which shows the schematic structure of the 1st image acquisition part. It is a figure which shows the schematic structure of the 2nd image acquisition part. It is a flowchart which shows the whole flow of operation of an image processing system. It is a flowchart which shows the control at the time of focusing. It is a figure which shows the setting method of the imaging area. It is a figure which shows the setting method of the focus measurement position in a bright-field image. It is a figure which shows an example of a focus map. It is a figure which shows the setting method of the focus measurement position in a fluorescence image. It is a flowchart which shows the control at the time of the fluorescence image acquisition. It is a figure which shows an example of the partial image acquisition method. It is a flowchart which shows the control of the attention area selection process. This is an example of a cell type identification image. This is an example of an image for cell analysis. This is an example of an image showing area information. It is a figure which shows an example of the selected area of interest. It is a flowchart which shows the control of the attention area selection process of the modification 1. It is a figure which shows an example of the selected area of interest. It is a flowchart which shows the control of the attention area selection process of the modification 2.

Hereinafter, preferred embodiments of the present invention will be described with reference to page 1.
[Image processing system]
FIG. 1 shows a schematic configuration of the image processing system 1 (WSI creation system) in the present invention. As shown in FIG. 1, the image processing system 1 includes a microscope device 10, a control device 60, a display device 70, and a database 80.

The microscope device 10 includes a first image acquisition unit 20, a second image acquisition unit 30, and a stage 40.
The tissue specimen 50 after immunostaining is placed in the stage 40. The tissue sample 50 is an example of a biological sample.

FIG. 2 shows a schematic configuration of the first image acquisition unit 20.
The first image acquisition unit 20 acquires a bright field image of the tissue sample 50. The first image acquisition unit 20 includes a bright field light source 21, a first image sensor 22, and a light guide lens 23.

The bright-field light source 21 is a light source that irradiates the tissue sample 50 with light for generating a light image for acquiring a bright-field image, and is installed so as to irradiate the light from below the stage 40. When the tissue sample 50 is irradiated by the bright-field light source 21 and an optical image is generated, the optical image is guided to the first image sensor 22 via the light guide lens 23, and the light image of the tissue sample 50 is brightened by the first image sensor 22. A field image is taken.
The first image sensor 22 is an image sensor such as a two-dimensional CCD sensor capable of acquiring a two-dimensional image of the optical image of the tissue specimen 50.

FIG. 3 shows a schematic configuration of the second image acquisition unit 30.
The second image acquisition unit 30 acquires a fluorescence image of the tissue sample 50.
The second image acquisition unit 30 includes a transmission light source 31, an excitation light source 32, a second image sensor 33, an objective lens 34, a fluorescence cube 35, and an imaging lens 36. The fluorescent cube 35 includes an excitation filter 351, a dichroic mirror 352, and an absorption filter 353.
The transmission light source 31 is a light source used when acquiring a transmission observation image of the tissue specimen 50, and is installed so as to irradiate light from below the stage 40.

The excitation light source 32 is a lamp that emits excitation light by a light source such as a discharge tube. The excitation filter 351 is a filter that transmits only excitation light. The dichroic mirror 352 is a mirror that reflects or transmits light having a predetermined wavelength as a boundary, and here, it reflects excitation light and transmits fluorescence. The absorption filter 353 is a filter that blocks excitation light and transmits only fluorescence.

In the second image acquisition unit 30, when the excitation light source 32 is turned on, the excitation light passes through the excitation filter 351 and is reflected by the dichroic mirror 352, passes through the objective lens 34, and irradiates the tissue sample 50. As a result, fluorescence is emitted from the tissue sample 50, and the fluorescence is focused by the objective lens 34 and transmitted through the dichroic mirror 352 and the absorption filter 353. After that, the fluorescence is guided to the second image sensor 33 as a fluorescence image via the image pickup lens 36, and is imaged by the second image sensor 33. The objective lens 34 includes a low-magnification objective lens (for example, 20 times) and a high-magnification objective lens (for example, 40 times).
The second image sensor 33 is an image sensor such as a one-dimensional CCD camera capable of acquiring a one-dimensional image or a two-dimensional image having a predetermined direction as the longitudinal direction, and acquires a high-resolution fluorescent image of the tissue sample 50. be able to.

A control device 60 for controlling these is connected to the microscope device 10. The control device 60 includes a control unit (acquisition means, selection means, identification means, analysis means) 61, a storage unit 62, an image processing unit 63, and a communication unit 64.

The control unit 61 is configured to include a CPU (Central Processing Unit), a RAM (Random Access Memory), and the like, and executes various processes in cooperation with various programs stored in the storage unit 62 to execute various processes of the microscope device 10. Control the operation comprehensively.
The control unit 61 is connected to the stage 40 and can control the ascent and descent of the stage 40 to control the focusing position (Z coordinate) of the tissue sample 50 installed on the stage 40. Further, the control unit 61 is connected to the first image acquisition unit 20 and controls the bright field light source 21 and the first image sensor 22 to take a bright field image. Further, the control unit 61 is connected to the second image acquisition unit 30 and controls the transmission light source 31, the excitation light source 32, and the second image sensor 33 to take a fluorescence image.

The storage unit 62 is composed of, for example, an HDD (Hard Disk Drive), a semiconductor non-volatile memory, or the like. The storage unit 62 stores a program for taking a bright field image and a fluorescent image.

The image processing unit 63 performs image processing on the fluorescent image taken by the microscope device 10 to create a hall slide image (WSI). As will be described later, according to the instruction of the control unit 61, the captured partial images are combined to create an entire image of the tissue sample 50, and the image data is A / D converted into a digital image to create a WSI. .. In addition, based on the created WSI, a fluorescence brightness map used for quantitative analysis of the target substance is created.

The communication unit 64 is an interface for transmitting and receiving data to and from an external device such as a personal computer. A user who wants to refer to the WSI can read the WSI stored in the database 80 into a personal computer or the like via the communication unit 64 and observe it on the display.

A display device 70 is connected to the control device 60.
The display device 70 is configured to include, for example, a monitor such as a CRT (Cathode Ray Tube) or an LCD (Liquid Crystal Display), and displays various screens according to an instruction of a display signal input from the control unit 61. .. In the present embodiment, the display device 70 functions as an output means for outputting a captured fluorescent image or the like.

A database 80 is further connected to the control device 60. The database 80 includes, for example, an HDD (Hard Disk Drive) and stores the WSI synthesized by the image processing unit 63.
In this embodiment, the WSI is stored in the database 80 as described above. However, if the WSI can be stored, the storage area is not limited to the database 80, and the database 80 is not provided. There may be.
For example, the WSI may be stored in the storage unit 62, or may be stored in an external server (not shown) to form a database.

[Tissue specimen]
Subsequently, the tissue sample 50 will be described.
The tissue sample 50 is a tissue section containing the target substance and is stained with an immunostaining agent, and the stained tissue sample 50 is placed on the stage 40.

(1) Target substance The target substance is a substance that is present in a tissue section and is intended for immunostaining using a fluorescent label, mainly for detection or quantification from the viewpoint of pathological diagnosis. In particular, it is a protein (antigen).
Typical target substances include biological substances that are expressed in the cell membranes of various cancer tissues and can be used as biomarkers for proteins, RNA, and the like.
Further, the target substance may be a substance introduced from outside the body, such as a drug. In addition, units smaller than proteins such as peptides can be immunostained.

(2) Immunostaining agent (antibody-fluorescent nanoparticle conjugate)
As an immunostaining agent, in order to improve the efficiency of fluorescent labeling and suppress the passage of time leading to deterioration of fluorescence as much as possible, the primary antibody and fluorescent nanoparticles indirectly, that is, using an antigen-antibody reaction or the like, other than covalent bonds. It is preferable to use a complex linked by the binding of. In order to simplify the staining operation, a complex in which fluorescent nanoparticles are directly linked to the primary antibody or the secondary antibody can also be used as the immunostaining agent.

Examples of the immunostaining agent include [primary antibody against the target substance] ... [antibody against the primary antibody (secondary antibody)] to [fluorescent nanoparticles].
“…” Indicates that the bond is bound by an antigen-antibody reaction, and the mode of binding indicated by “～” is not particularly limited. For example, covalent bond, ionic bond, hydrogen bond, coordination bond, antigen-antibody bond, Examples thereof include biotin avidin reaction, physical adsorption, and chemisorption, and may be mediated by a linker molecule if necessary.

(3) Antibody As the primary antibody, an antibody (IgG) that specifically recognizes and binds to the target substance as an antigen can be used. For example, when HER2 is the target substance, an anti-HER2 antibody can be used, and when HER3 is the target substance, an anti-HER3 antibody can be used.
As the secondary antibody, an antibody (IgG) that specifically recognizes and binds to the primary antibody as an antigen can be used.
Both the primary antibody and the secondary antibody may be polyclonal antibodies, but monoclonal antibodies are preferable from the viewpoint of quantitative stability. The type of animal (immune animal) that produces an antibody is not particularly limited, and may be selected from mice, rats, guinea pigs, rabbits, goats, sheep, and the like as in the conventional case.

(4) Fluorescent nanoparticles Fluorescent nanoparticles are nano-sized particles that fluoresce when irradiated with excitation light, and emit fluorescence of sufficient intensity to represent the target substance as a bright spot one by one. Fluorescent particles.
As the fluorescent nanoparticles, in the present invention, fluorescent substance integrated nanoparticles (PID: Phosphor Integrated Dot nanoparticles) are used.

(4.1) Fluorescent substance-accumulated nanoparticles PID is based on particles made of organic or inorganic substances, and a plurality of fluorescent substances (for example, the quantum dots, organic fluorescent dyes, etc.) are contained therein and /. Alternatively, it is a nano-sized particle having a structure adsorbed on its surface. As the PID, quantum dot integrated nanoparticles, fluorescent dye integrated nanoparticles and the like are used.
The fluorescent substance used for PID shows emission of visible to near infrared light having a wavelength in the range of 400 to 900 nm when excited by ultraviolet to near infrared light having a wavelength in the range of 200 to 700 nm. It is preferable that the mother body and the fluorescent substance have substituents or sites having opposite charges, and an electrostatic interaction acts.

The average particle size of the PID used in the present invention is not particularly limited, but one having a diameter of about 30 to 800 nm can be used. When the average particle size is less than 30 nm, the amount of fluorescent substances contained in the accumulated particles is small, making it difficult to quantitatively evaluate the target substance, and when it exceeds 800 nm, it is difficult to bind to the target substance in the pathological tissue. This is to become. The average particle size is more preferably in the range of 40 to 500 nm. Here, the reason why the average particle size is set to 40 to 500 nm is that if it is less than 40 nm, an expensive detection system is required, and if it exceeds 500 nm, the quantification range is narrowed due to the physical size. ..

The coefficient of variation (= (standard deviation / average value) x 100%) indicating the variation in particle size is not particularly limited, but it is desirable to use one of 15% or less. The smaller the variation in particle size, the smaller the variation in the brightness of the fluorescence bright spot, and as will be described later, the expression level of the target substance can be quantitatively evaluated based on the fluorescence brightness. The average particle size is determined by taking an electron micrograph using a scanning electron microscope (SEM), measuring the cross-sectional area of a sufficient number of particles, and using each measured value as the area of a circle. Can be obtained as.

(4.1.1) Mother Body Among the mother bodies, the organic substances are resins generally classified as thermosetting resins such as melamine resin, urea resin, aniline resin, guanamine resin, phenol resin, xylene resin, and furan resin. Resins generally classified as thermoplastic resins such as styrene resin, acrylic resin, acrylonitrile resin, AS resin (acrylonitrile-styrene copolymer), ASA resin (acrylonitrile-styrene-methyl acrylate copolymer); poly Other resins such as styrene; polysaccharides can be exemplified.
Examples of the inorganic substance in the mother body include silica and glass.

(4.1.2) Quantum Dot Accumulated Nanoparticles Quantum dot integrated nanoparticles have a structure in which the quantum dots are contained in the mother body and / or adsorbed on the surface thereof.
When the quantum dots are contained in the mother body, the quantum dots need only be dispersed inside the mother body, and may or may not be chemically bonded to the mother body itself.

As the quantum dots, semiconductor nanoparticles containing a group II-VI compound, a group III-V compound or a group IV element are used. For example, CdSe, CdS, CdTe, ZnSe, ZnS, ZnTe, InP, InN, InAs, InGaP, GaP, GaAs, Si, Ge and the like can be mentioned.

It is also possible to use a quantum dot having the above quantum dot as a core and a shell provided on the core. Hereinafter, as the notation of the quantum dot having a shell in the present specification, when the core is CdSe and the shell is ZnS, it is described as CdSe / ZnS. For example, CdSe / ZnS, CdS / ZnS, InP / ZnS, InGaP / ZnS, Si / SiO2, Si / ZnS, Ge / GeO2, Ge / ZnS and the like can be used, but are not limited thereto.

If necessary, the quantum dots may be surface-treated with an organic polymer or the like. For example, CdSe / ZnS having a surface carboxy group (manufactured by Invitrogen), CdSe / ZnS having a surface amino group (manufactured by Invitrogen), and the like can be mentioned.

Quantum dot integrated nanoparticles can be produced by a known method. For example, the silica nanoparticles containing quantum dots can be synthesized with reference to the synthesis of CdTe-encapsulating silica nanoparticles described in New Journal of Chemistry, Vol. 33, p. 561 (2009).
For silica nanoparticles encapsulating quantum dots, refer to the synthesis of silica nanoparticles in which particles of CdSe / ZnS capped with 5-amino-1-pentanol and APS described on page 2670 (2009) of Chemical Communication are integrated on the surface. Can be synthesized into.
Polymer nanoparticles encapsulating quantum dots can be produced by using the method of impregnating polystyrene nanoparticles with quantum dots described in Nature Biotechnology, Vol. 19, p. 631 (2001).

(4.1.3) Fluorescent dye-accumulated nanoparticles The fluorescent dye-accumulated nanoparticles have a structure in which a fluorescent dye is contained in the mother body and / or is adsorbed on the surface thereof.
Examples of the fluorescent dye include organic fluorescent dyes such as rodamine-based dye molecule, squarylium-based dye molecule, cyanine-based dye molecule, aromatic ring-based dye molecule, oxazine-based dye molecule, carbopyronine-based dye molecule, and pyromesene-based dye molecule. it can.

Specifically, Alexa Fluor (registered trademark, manufactured by Invigen) dye molecule, BODIPY (registered trademark, manufactured by Invigen) dye molecule, Cy (registered trademark, manufactured by GE Healthcare) dye molecule, HiLite (registered) Trademarks, Anaspec) dye molecules, DyLight (registered trademark, Thermoscientific) dye molecules, ATTO (registered trademark, ATTO-TEC) dye molecules, MFP (registered trademark, Mobitec) ) Dye molecule, CF (registered trademark, manufactured by Biotium) dye molecule, DY (registered trademark, manufactured by DYOMICS) dye molecule, CAL (registered trademark, manufactured by BioSearch Technologies) dye molecule and the like can be used. ..

When the fluorescent dye is contained in the mother body, the fluorescent dye may or may not be chemically bonded to the mother body itself as long as it is dispersed inside the mother body.

Fluorescent dye-accumulated nanoparticles can be produced by a known method. For example, silica nanoparticles containing a fluorescent dye can be synthesized with reference to the synthesis of FITC-encapsulating silica particles described in Langmuir Vol. 8, p. 2921 (1992). By using a desired fluorescent dye instead of FITC, various fluorescent dye-accumulated nanoparticles can be synthesized.
Polystyrene nanoparticles containing a fluorescent dye can be obtained by a copolymerization method using an organic dye having a polymerizable functional group described in US Pat. No. 4,326,008 (1982) or polystyrene nanoparticles described in US Pat. No. 5,326,692 (1992). It can be prepared by using a method of impregnating particles with a fluorescent dye.

(5) Staining Method for Tissue Section An example of the staining method will be described.
The method for preparing a tissue section to which this staining method can be applied (also referred to simply as a “section” and including a section such as a pathological section) is not particularly limited, and a tissue section prepared by a known procedure can be used.

(5.1) Specimen preparation step (5.1.1) Deparaffinization treatment The section is immersed in a container containing xylene to remove paraffin. The temperature is not particularly limited, but it can be carried out at room temperature. The immersion time is preferably 3 minutes or more and 30 minutes or less. If necessary, xylene may be replaced during immersion.

Next, the section is immersed in a container containing ethanol to remove xylene. The temperature is not particularly limited, but it can be carried out at room temperature. The immersion time is preferably 3 minutes or more and 30 minutes or less. If necessary, ethanol may be replaced during immersion.

Immerse the section in a container filled with water and remove ethanol. The temperature is not particularly limited, but it can be carried out at room temperature. The immersion time is preferably 3 minutes or more and 30 minutes or less. If necessary, the water may be replaced during immersion.

(5.1.2) Activation treatment The activation treatment of the target substance is performed according to a known method.
The activation conditions are not particularly specified, but the activation solution is 0.01 M citrate buffer (pH 6.0), 1 mM EDTA solution (pH 8.0), 5% urea, and 0.1 M Tris-hydrochloric acid buffer. A liquid or the like can be used.
The pH condition is such that a signal is output from the range of pH 2.0 to 13.0 depending on the tissue section to be used and the tissue roughness is such that the signal can be evaluated. Normally, it is performed at pH 6.0 to 8.0, but for special tissue sections, it is also performed at pH 3.0, for example.
As the heating device, an autoclave, a microwave, a pressure cooker, a water bath, or the like can be used. The temperature is not particularly limited, but it can be carried out at room temperature. The temperature can be 50 to 130 ° C. and the time can be 5 to 30 minutes.

Next, the section after activation treatment is immersed in a container containing PBS and washed. The temperature is not particularly limited, but it can be carried out at room temperature. The immersion time is preferably 3 minutes or more and 30 minutes or less. If necessary, PBS may be replaced during immersion.

(5.2) Immunostaining Step In the immunostaining step, in order to stain the target substance, a solution of an immunostaining agent containing fluorescent nanoparticles having a site capable of directly or indirectly binding to the target substance is applied to a section. Place and react with the target substance. The solution of the immunostaining agent used in the immunostaining step may be prepared in advance before this step.

When a plurality of target substances are to be detected, immunostaining is performed with a plurality of immunostaining agents corresponding to the target substances. The plurality of immunostaining agents used in this case may be those containing at least one immunostaining agent using PID (PID staining agent), and if the antibody and the fluorescent substance (fluorescent wavelength) are different from each other, PID staining may be performed. It is also possible to detect a plurality of target substances by multiple staining using a plurality of agents or by a combination of a PID staining agent and an immunostaining agent using a fluorescent label such as an organic fluorescent substance or a quantum dot. .. In this case, a solution of each immunostaining agent is prepared, placed on a section, and reacted with the target substance. However, the solution of each immunostaining agent may be mixed in advance when the solution is placed on the section, or separately. It may be placed sequentially in.
When a plurality of immunostaining agents are used, it is desirable that the excitation / emission wavelengths of the PID and the excitation / emission wavelengths of the fluorescent labels of other immunostaining agents are separated to a extent that crosstalk can be ignored.

The conditions for performing the immunostaining step, that is, the temperature and the immersion time when immersing the tissue section in the solution of the immunostaining agent, should be appropriately adjusted so as to obtain an appropriate signal according to the conventional immunostaining method. Can be done.
The temperature is not particularly limited, but it can be carried out at room temperature. The reaction time is preferably 30 minutes or more and 24 hours or less.
It is preferable to drop a known blocking agent such as PBS containing BSA or a surfactant such as Tween 20 before performing the treatment as described above.

(5.3) Specimen post-treatment step It is preferable that the tissue specimen after the immunostaining step is subjected to treatments such as immobilization / dehydration, permeation, and encapsulation so as to be suitable for observation.

For the immobilization / dehydration treatment, the tissue section may be immersed in a fixation treatment solution (crosslinking agent such as formalin, paraformaldehyde, glutaaldehyde, acetone, ethanol, methanol). In the permeation treatment, the tissue section that has been immobilized and dehydrated may be immersed in a permeation solution (xylene or the like). For the encapsulation treatment, the tissue section that has undergone the permeation treatment may be immersed in the encapsulation liquid.
The conditions for performing these treatments, for example, the temperature and the immersion time when immersing the tissue section in a predetermined treatment solution, may be appropriately adjusted so as to obtain an appropriate signal according to the conventional immunostaining method. it can.

(5.4) Morphological observation staining step Separate from the immunostaining step, morphological observation staining is performed so that the morphology of cells, tissues, organs, etc. can be observed. The morphological observation staining is not particularly limited as long as it can express the morphology of cells, tissues, organs and the like. For example, in addition to the dyeing methods listed below as conventional methods, a dyeing method using a fluorescent dye or the like can also be used.
The morphological observation dyeing step can be performed according to a conventional method.
For morphological observation of the tissue specimen 50, staining with eosin, in which cytoplasm, interstitium, various fibers, erythrocytes, and keratinocytes are stained in red to deep red, is standardly used. Staining with hematoxylin, in which the cell nucleus, lime, cartilage tissue, bacteria, and mucus are stained in blue to pale blue, is also standardly used (the method of performing these two stainings at the same time is hematoxylin / eosin staining). Known as (HE staining)).
When the morphological observation staining step is included, it may be performed after the immunostaining step or before the immunostaining step.

[Operation of image processing system]
FIG. 4 shows the overall flow of the operation of the image processing system 1 in the present invention.
As shown in FIG. 4, the operation of the image processing system 1 includes a focusing step (step S101), a low-magnification image acquisition step (step S102), a region of interest selection step (step S103), and an image acquisition step for analysis (step S104). ) And an analysis step (step S105).
Hereinafter, each step will be described.

[Focusing process]
First, the focusing process will be described.
In the present embodiment, the first image acquisition unit 20 acquires a bright-field image of the tissue sample 50, sets an imaging region to be created for WSI based on the bright-field image, and performs focusing based on the bright-field image. Further, the second image acquisition unit 30 irradiates the tissue specimen 50 with the fluorescence-labeled PID with excitation light, and performs more rigorous focusing with reference to the fluorescent bright spot of the detected PID.

Specific control will be described with reference to the flowchart of FIG.
First, the control unit 61 controls the first image acquisition unit 20 to acquire a bright field image for focusing of the entire slide glass (step S1). This bright-field image is used for setting high-resolution imaging conditions such as a fluorescence image described later, and is a low-magnification image using a low-magnification objective lens. Based on the obtained bright-field image, the control unit 61 sets an imaging region R including the tissue sample 50 as shown in FIG. 6 (step S2).
Specifically, the control unit 61 binarizes the entire image of the tissue sample 50 depending on the presence or absence of the tissue sample 50, and detects the region where the tissue sample 50 exists in each of the X-axis direction and the Y-axis direction. The imaging region R is determined. The imaging region R may be manually set on the display device 70 while the user is observing the bright field image of the entire tissue specimen 50, but it is preferably set automatically.

Next, the tissue sample 50 is focused based on the bright field image. Focusing based on the bright-field image may be performed manually by the user, but it is preferably performed automatically under the control of the control unit 61. Hereinafter, a method of automatically creating a focus map under the control of the control unit 61 and performing focusing will be described.

First, the first focus measurement position P1 is set on the imaging region R (step S3).
As shown in FIG. 7, the control unit 61 divides the imaging region R into the X-axis direction and the Y-axis direction to set a small region, and obtains the XY coordinates of each small region. Here, the XY coordinates are the center coordinates of each subregion, but the coordinates are not limited to this, and for example, the coordinates of the upper left end of each subregion can be the XY coordinates.
Further, as shown in FIG. 7, the control unit 61 assigns numbers such as 1, 2, 3, ... In each of the X-axis direction and the Y-axis direction to each small area, and sets the array number. That is, the sequence number is, for example, (X-axis, Y-axis) = (1,1) in the small area at the upper left end of the imaging region R.
Next, the control unit 61 sets the first focus measurement position P1 for each small area. In the present embodiment, P1 is the center coordinate position of each small area, but the present invention is not limited to this, and for example, the upper left end of each small area can be set as the first focus measurement position P1. In addition, the tissue sample 50 may not exist on the center coordinates, for example, as in the region of the sequence number (1, 1) in FIG. In this case, it is possible to move the first focus measurement position P1 to an arbitrary coordinate on the tissue sample 50.

Subsequently, focusing is performed on the first focus measurement position P1 in each small area (step S4). Here, the control unit 61 adjusts the optical axis position to the first focus measurement position P1 while moving the stage 40 in the XY directions, and focuses the bright field by actual measurement with respect to each first focus measurement position P1. Find the position (Z coordinate).

Based on the bright field focus position obtained in this way, the control unit 61 creates a focus map as shown in FIG. 8 (step S5). The focus map stores the array number of each small area and the corresponding stage coordinates. The stage coordinates correspond to the center coordinates of each small area for the X-axis and the Y-axis, and the bright-field focusing position for the Z-axis.
This completes the focusing of the tissue sample 50 based on the bright-field image.

From step S6 onward, focusing is performed on the PID bright spot based on the focusing information obtained based on the bright field image. That is, as described above, the bright-field focusing position of the tissue sample 50 based on the bright-field image is specified by the processing of steps S1 to S3, and focusing is further performed on the PID bright spot based on this. By performing the above, a more precisely focused fluorescence image can be obtained.

First, a fluorescence image of the PID for focusing is acquired (step S6). That is, the control unit 61 controls the excitation light source 32 to irradiate the tissue sample 50 with the excitation light of the PID, and acquires the fluorescence image of the PID by the second image sensor 33.

Next, the fluorescence bright spot of an arbitrary PID on the obtained fluorescence image is selected, and the second focus measurement position P2 is set (step S7). Here, it is assumed that the user manually sets the second focus measurement position P2, and as shown in FIG. 9, one or a plurality of second focus measurement positions P2 on the tissue sample 50 are set. The second focus measurement position P2 may be automatically set.

Next, focusing is performed on the set second focus measurement position P2 (step S8). Specifically, the control unit 61 adjusts the optical axis position to the second focus measurement position P2 while moving the stage 40 in the XY directions, and refers to the focus position of the focus map created in step 3. , Finely adjust in the Z coordinate direction to obtain the fluorescence focusing position (Z coordinate) with respect to the second focus measurement position P2.

When this is performed for all the second focus measurement positions P2, the control unit 61 modifies the focus map created in step S3 by using the obtained new focus position (step S9).
As described above, the focusing on the tissue sample 50 is completed.
The focusing method described above is merely an example, and the method applicable to the present invention is not limited to this.

[Low magnification image acquisition process]
Next, a low-magnification image acquisition process for creating a WSI by low-magnification imaging will be described.
When Focusing is completed as described above, the process proceeds to the preparation of WSI for the entire tissue sample 50.
Hereinafter, a method for creating a WSI of a fluorescence image will be described with reference to the flowchart of FIG.

First, the fluorescently labeled body labeled on the tissue specimen 50 is excited (step S10). Specifically, the control unit 61 controls the excitation light source 32 to irradiate the tissue specimen 50 with excitation light that excites the labeled PID.

Next, a partial image of the tissue sample 50 is acquired (step S11).
Here, based on the information of the focus map completed in step S9, the control unit 61 moves and controls the stage 40, and controls the second image acquisition unit 30 to acquire a partial fluorescence image. That is, the optical axis position and the focusing position are moved to the XYZ coordinates indicated by the stage coordinates stored in the focus map, and the second image sensor 33 is controlled to capture an image for each small area. Here, by using a high-magnification objective lens as the objective lens 34, a high-resolution image can be acquired.

Specifically, a strip-shaped scan image as shown in FIG. 11 is acquired as a partial image. First, imaging of the tissue specimen 50 is started from the upper left corner. The control unit 61 irradiates the excitation light and scans the image pickup position by the second image pickup element 33 while moving in the positive direction of the Y axis of the tissue section 51 to acquire a partial image A. Subsequently, the control unit 61 moves the image pickup position by the second image pickup element 33 in the positive direction of the X axis to acquire the partial image B. Similarly, when the partial images are acquired in the order of the partial image C, the partial image N, the imaging is completed.

Next, the control unit 61 controls the image processing unit 63 as the creating means to synthesize the captured partial images to create the entire fluorescent image of the imaging region R (step S12). That is, by arranging and pasting the partial images A to N in the X-axis direction, a high-resolution fluorescence image of the entire tissue sample 50 can be obtained.
Further, the image processing unit 63 A / D-converts the entire fluorescent image of the obtained imaging region R into a digital image (step S13). With the above, the creation of WSI is completed.
The created WSI is stored by the database 80 as a storage means. A user who wants to refer to the WSI can read the image data into a personal computer or the like via the communication unit 64 and observe it on the display.

The method for creating a WSI for a fluorescent image has been described above, but the WSI is not particularly limited to a fluorescent image as long as the target for creating the WSI (for example, the entire tissue sample 50) can be grasped. For example, WSI may be acquired as a bright field image. Further, the WSI of the bright field image can be created in the same manner as the creation of the WSI of the fluorescence image described above.

Further, in the present invention, a WSI of an image for specifying the cell type (image for specifying the cell type) and an image for analyzing the specified cell (image for cell analysis) is created.
The target cell types include, for example, classification by differentiation of hepatocytes, glial cells, T cells, etc., pathological classification such as canceration and inflammation, and classification under specific conditions such as cell cycle and necrosis. , Spatial arrangement and shape feature classification such as infiltration and protrusions are also included.

[Production area selection process]
Subsequently, the process of selecting the region of interest will be described.
The region of interest selection step is a step of selecting a region of interest (FOV: Field Of View) using the created WSI (image for specifying cell type, image for cell analysis).

The “region of interest” is not limited as long as it is a region useful for pathological diagnosis, and may include a normal tissue region as well as a lesion portion such as a cancer region. In addition, a region in which the effect of the drug appears or a region in which the drug is present can also be included in the region of interest because it is considered to be useful for pathological diagnosis.
Further, a region specified by the color of the bright-field image or a region specified by the shade of the fluorescence image may also be useful, and the size of each region can be appropriately specified, preferably about 100 um square. Is.
In particular, specifying a region using the number of fluorescent bright spots as an index may be useful for pathological diagnosis, and identifying and extracting a region having a large number, a region having a medium number, and a region having a small number, respectively. You can also. Areas with a large number are sometimes called hot spots, and areas with a small number are sometimes called cold spots.

Hereinafter, the region of interest selection process will be described with reference to the flowchart of FIG.

First, the control unit 61 reads two predetermined WSIs (image for cell type identification and image for cell analysis) from the database 80 with image processing software, and for the read image, for example, noise removal and area setting (non-specific). Pretreatment such as fluorescence exclusion) is performed to acquire two images for selecting a region of interest (step S21).
As an example of the cell type identification image, FIG. 13A is an H-stained bright field image. Further, as an example of the image for cell analysis, FIG. 13B is a PD-L1 stained fluorescent image. The lower right image in FIGS. 13A and 13B is an enlarged image obtained by enlarging a part of each image.

(Noise removal processing)
The noise removal process is, for example, a process of suppressing the autofluorescence brightness of the read fluorescence image. Specifically, the control unit 61 generates a frequency spectrum by DFT processing (high-pass filter), and multiplies the frequency spectrum by a high-pass filter image. Next, a low-frequency component-suppressed image is generated by IDFT.
Since the size of the fluorescent image is huge, it is preferable to divide and process the fluorescent image according to the memory load used when the processing by the DFT is executed. That is, the above-mentioned processing may be executed for each of the divided images of the fluorescence image, and finally they may be combined.

Here, the fluorescence signal by PID and the fluorescence signal by autofluorescence have different spatial frequency profiles. In general, autofluorescence contains a large amount of low-frequency components, PID fluorescence has a steep peak shape, and low-frequency components are less than autofluorescence. For example, by obtaining the frequency information of the fluorescence image by frequency spectrum analysis using DFT or FFT, the brightness signal of the PID and the brightness signal due to autofluorescence can be separated, and the brightness component due to autofluorescence can be selectively suppressed. ..
The frequency spectrum can be reversibly converted into a spatial signal by IDFT / IFFT, and as described above, the frequency spectrum in a state where the brightness component due to autofluorescence is selectively suppressed is inversely transformed into spatial information by IDFT / IFFT, thereby autofluorescent. A fluorescent image with suppressed is obtained.

(Region setting (non-specific fluorescence exclusion) processing)
In addition, in the area setting (non-specific fluorescence exclusion) processing, the fluorescence at the peripheral edge of the tissue may become stronger due to uneven staining of the automatic dyeing machine. It is a process to make it a necessary area.
Specifically, if uneven staining density occurs in the tissue section due to factors such as non-uniformity of the flow of the staining solution by the automatic dyeing machine and dominant dye adsorption at the edges due to the thickness of the tissue section, the above-mentioned factors cause artifacts. Since the frequency of non-specific fluorescence is high in a certain part and the brightness of the fluorescence cannot be analyzed equally, it is necessary to exclude it from the necessary region.

Therefore, the control unit 61 specifies the search start point at the end of the tissue section image by a conventionally known method, and obtains the ROI information of the edge of the section by the contour tracking algorithm that allows the pixel value change within a specific range. Generate. As a conventionally known method, for example, Imagej's WandTool (National Institutes of Health, MD, USA) can be mentioned.
Next, the control unit 61 performs a reduction process of several hundred pixels (about one field of view of the microscope) on the generated ROI information (that is, the designated pixel area is reduced inward).
Next, the control unit 61 excludes the outside of the required region by superimposing the generated ROI information on the HPF image and deleting the fluorescence pixel value outside the region (luminance is zero). That is, the required area is narrowed down. However, narrowing down the area is not essential in cases where non-specific uneven staining around the tissue does not occur.
The autofluorescent region of cells without nuclei such as erythrocytes may be excluded from the required region. Further, in order to narrow down the required region, various known methods can be applied in addition to the above-mentioned method of deleting the fluorescent pixel value outside the required region (luminance is zero).
Next, the control unit 61 cuts out a necessary region from the autofluorescence suppression image and generates an image for selecting a region of interest.

Returning to FIG. 12, next, the control unit 61 machine-learns or identifies a characteristic part of the image with respect to the cell type identification image for selecting the region of interest acquired in step S21. By applying the classification method and identifying one or more analysis target regions in the cell, the cell type is specified, and region information indicating the analysis target region is created (step S22).
Specific examples of the machine learning or image feature classification method include cell type recognition by machine learning, dye extraction from a bright-field H-stained image, and tumor marker-stained cell extraction.
The analysis target region is a region that is a specific type of cell and is the target of analysis, and in step S23 described later, feature amount data regarding the target substance in the analysis target region is calculated. Specifically, the analysis target region is a specific site in a specific type of cell, for example, the cell nucleus, cytoplasm, cell membrane, specific organelle of a specific type of cell, or an arbitrary defined by image processing, calculation, or the like. Area of.
As the region information, for example, as shown in FIG. 14, an image in which the shape of the cell can be understood can be mentioned.

Next, the control unit 61 identifies the analysis target region from the fluorescence image for selecting the region of interest acquired in step S21 based on the region information created in step S22, and features related to the target substance in each region. Quantitative data (first feature quantity data) is calculated (step S23).
For example, the average luminance value in each region is calculated as the first feature amount data. By creating a heat map or a histogram in which the average luminance value is represented by shades of color, the user can recognize the distribution of the target substance in the fluorescence image.

Next, the control unit 61 identifies the analysis target region from the bright field image for selecting the region of interest acquired in step S21 based on the region information created in step S22, and relates to the cell type in each region. The feature amount data (second feature amount data) is calculated (step S24).
For example, the cell density in each region is calculated as the second feature amount data. The user can recognize the distribution of the cells in the bright-field image by creating a heat map or a histogram in which the value of the cell density is represented by a shade of color or the like.

Next, the control unit 61 performs arithmetic processing using the first feature amount data calculated in step S23 and the second feature amount data calculated in step S24, and calculates the composite feature amount data (step). S25).
Specifically, the control unit 61 performs arithmetic processing such as multiplication using the average luminance value (first feature amount data) and the cell density (second feature amount data) for each analysis target region. The calculation result is a composite feature quantity.
In addition to using the values as they are for the first feature data and the second feature data, certain offset values are added, normalized, or converted to another dimension such as n-step stratification, etc. The arithmetic processing may be performed after the processing of.

Next, the control unit 61 selects a region of interest based on the calculated composite feature amount data (step S26).
For example, a region in which the value of the composite feature amount data is equal to or higher than a predetermined first threshold value can be selected as a region of interest (hot spot). Further, a region smaller than the first threshold value and less than the second threshold value may be selected as a region of interest (cold spot). In addition, the top N elements having high values of the composite feature amount data may be selected as the region of interest.
Thereby, for example, as shown in FIG. 15, the result data in which the region of interest SP is selected can be obtained.
By using the complex feature amount in this way, it is possible to quantitatively evaluate only the target substance existing in the target cell type, not the arbitrary target substance existing in the tissue sample, and thus the distribution of the target substance. Etc. can be grasped more accurately, and analysis and the like can be performed with high accuracy.

(Modification example 1)
Next, a modification 1 of the region of interest selection process will be described with reference to the flowchart of FIG.
In the above-mentioned attention area selection process, in step S25, composite feature amount data is calculated from the first feature amount data and the second feature amount data, and in step S26, the attention area is selected from the values.
However, as shown in FIG. 16, a predetermined spot is selected (specified) from the fluorescence image using the first feature data, and a predetermined spot is selected (specified) from the bright field image using the second feature data. (Specification) may be performed (step S27), and the region of interest may be selected from the result of superimposing these results (step S28).
As a spot selection method using the first feature amount data or the second feature amount data, for example, a region in which the average brightness value or the cell density is equal to or higher than a predetermined first threshold value is selected as a predetermined spot. be able to. Alternatively, a region smaller than the first threshold value and less than the second threshold value may be selected as the predetermined spot, or a region between the first threshold value and the second threshold value may be selected as the predetermined spot. Is also good. Further, the top N spots having a high average brightness value or cell density value may be selected as predetermined spots.
Then, after spot selection using the first feature amount data or the second feature amount data, a region in which these overlap each other is selected as a region of interest.
For example, as shown in FIG. 17, attention is paid to the region SP3 in which the predetermined spot SP1 (solid line) using the first feature data and the predetermined spot SP2 (broken line) using the second feature data overlap. Select as an area.

(Modification 2)
Next, a modification 2 of the region of interest selection process will be described with reference to the flowchart of FIG.
Further, in the above-described region of interest selection process, the case where one image for specifying the cell type and one image for cell analysis are acquired in step S21 has been described as an example, but a plurality of these images can also be used. ..
Hereinafter, the region of interest selection process using two images for cell type identification and two images for cell analysis will be described.

Examples of the cell type identification image include, although not shown, an Epcam-stained fluorescence image and a DAB (CD8) -stained bright-field image.
Although not shown, examples of the cell analysis image include a PD-L1 stained fluorescent image and a PD-1 stained fluorescent image.
The control unit 61 performs preprocessing on such an image in the same manner as in step S21 to acquire an image for selecting a region of interest (step S31).

Next, the control unit 61 identifies different analysis target regions from each of the two cell type identification images in the same manner as in step S22, and creates region information for each region (step S32).
For example, region information for identifying a cancer cell region is created from an Epcam-stained fluorescent image. Further, from the DAB (CD8) stained bright field image, region information for identifying the CD8-positive T cell region is created.

Next, the control unit 61 calculates the first feature amount data from each of the two cell analysis images in the same manner as in step S23 (step S33).
Next, the control unit 61 calculates the second feature amount data from each of the two cell type identification images in the same manner as in step S24 (step S34).

Next, the control unit 61 calculates the related information between the two cell types using the two second feature amount data calculated in step S34 (step S35).
Specifically, the density ratio between the two cell types and the distance between the dense regions of the two cell types are calculated as related information.

Next, the control unit 61 narrows down the area for each of the two cell analysis images based on the related information calculated in step S35, and uses the first feature amount data in the narrowed down area. To select a predetermined spot (step S36).
As a result, a predetermined spot is selected in the narrowed area in each of the two cell analysis images. The selection of the predetermined spot using the first feature amount data can be performed in the same manner as in step S27.

Next, the control unit 61 selects a region of interest from the result of superimposing the results of selecting a predetermined spot in step S36 (step S37).

From the above, for example, when a region having a high cancer cell / T cell density ratio is narrowed down, it is possible to select a region of interest in the (non-infiltrating) cancer cell region according to the expression level of cancer cell PD-L1. it can.
Further, when the region having a low cancer cell / T cell density ratio is narrowed down, the region of interest in the (non-infiltrating) T cell region can be selected according to the T cell PD-1 expression level.
In addition, when the region where the cancer cell / T cell density ratio is within the threshold is narrowed down, (1) the region of interest according to the expression level of PD-L1 of cancer cells in the region of T cell infiltration into cancer cells, (2). ) A region of interest according to the expression level of T cell PD-1, and (3) a region of interest according to co-expression such as multiplication of the expression levels of (1) and (2) above can be selected.

[Image acquisition process / analysis process for analysis]
The analysis image acquisition step is a step of acquiring an analysis image (re-photographed image) by photographing the area of interest selected as described above at a magnification higher than that of the tissue image. Specifically, the position of the selected region of interest is observed with the high-magnification objective lens of the second image acquisition unit 30, and an image is acquired again. In addition to the high-magnification objective lens of the second image acquisition unit 30, a high-magnification microscope such as BX63 + DP80 (Olympus) can also be used.
By taking an enlarged image of the selected region of interest in this way, more accurate analysis becomes possible. It should be noted that such an embodiment can be taken in the case of a hall slide scanner having a low magnification (low sensitivity). Accurate analysis can be realized by automatically inheriting the position coordinates of the region of interest to the high-magnification microscope, and alignment can be achieved by driving the tissue sample slide on an automatic stage.

In the analysis image acquisition step, it suffices if an analysis image for the selected region of interest can be acquired, and in addition to acquiring an analysis image by photographing at a magnification higher than that of the tissue image as described above. , An image acquisition with an increased bit depth, or a three-dimensional stack image (re-photographed image) of the selected region of interest may be acquired.

Further, the analysis step is a step in which the control unit 61 analyzes using an image (re-photographed image) photographed at a high magnification as described above. For example, a step of evaluating the fluorescence bright spot derived from PID, which will be described later, and a step of quantitatively evaluating the target substance can be mentioned. Examples of the step of evaluating the fluorescent bright spots derived from PID include a step of measuring the number of fluorescent bright spots and a step of measuring the number of PID particles corresponding to the number of fluorescent bright spots. Examples of the step of quantitatively evaluating the target substance include a step of calculating a PID score.

Here, in order to perform an analysis for diagnosis or the like, the image quality that can withstand the analysis must be ensured. Therefore, in order to perform highly accurate analysis, it is necessary to take a tissue sample at a high magnification. However, when the entire pathological specimen is photographed in a high-magnification field of view, the capacity of the acquired image becomes enormous, and the analysis using the image takes an enormous amount of time, which reduces the efficiency of the analysis.
In particular, when a fluorescent image is used, it is difficult to acquire a fluorescent image with good image quality (a fluorescent image that can withstand high-precision analysis) as WSI, which is particularly problematic.
From this, by acquiring a fluorescence image only for the selected region of interest, it becomes relatively easy to acquire a fluorescence image with good image quality (a fluorescence image that can withstand high-precision analysis). Even when it is used, a fluorescent image with good image quality can be efficiently obtained, and the effect of the present invention is particularly remarkable. That is, for example, by going through each step of the above-described embodiment, it is possible to suppress unnecessarily high-magnification imaging, and highly accurate and highly efficient analysis becomes possible.

Further, it is preferable that the analysis is performed using a plurality of images (re-photographed images) corresponding to the cell type identification image used for selecting the region of interest and the cell analysis image. By using a plurality of images, it is possible to obtain more detailed analysis data. Specifically, it is possible to obtain statistical values of the expression level of the target substance per cell and analysis data such as a position image of the target substance.

[Effect of Embodiment]
As described above, according to the image processing system 1 of the present embodiment, the control unit 61 acquires a tissue image obtained by photographing the tissue sample 50, and from the acquired tissue image, an image of a region of interest for re-imaging. Select. Further, the control unit 61 acquires a cell type identification image for identifying one or more cell types in the tissue sample 50 and one or more cell analysis images for analyzing the identified cells. , The region of interest is selected based on the feature amount related to the cell type calculated from the cell type identification image and the feature amount related to one or more target substances calculated from the cell analysis image.
For this reason, it is possible to efficiently perform high-precision analysis by suppressing unnecessary high-magnification imaging by the low-magnification image acquisition step, the specific region extraction step, and the analysis image acquisition step. It is possible to improve the accuracy and efficiency of analysis at the same time.
In addition, by selecting the region of interest based on the feature amount related to the cell type calculated from the image for specifying the cell type and the feature amount related to the target substance calculated from the image for cell analysis, the region of interest suitable for the purpose can be selected. Can be automatically selected based on descriptive criteria.
As a result, compared to the case where the observer visually selects the region of interest, for example, it is possible to perform an analysis excluding arbitrariness, so that the evaluation of the fluorescent bright spot and the target substance can be performed with higher accuracy. It becomes possible to do.

Further, according to the present embodiment, the tissue specimen is morphologically observed and stained so that one or more cell types can be identified by the control unit 61.
This makes it possible to identify one or more cell types using one tissue specimen.

Further, according to the present embodiment, the control unit 61 identifies the analysis target region in the cell type identification image by machine learning or an image feature classification method.
As a result, a more accurate image for identifying the cell type can be obtained.

Further, according to the present embodiment, the control unit 61 calculates the feature amount related to the cell type from the analysis target region of the cell type identification image, and relates to the target substance from the region corresponding to the analysis target region of the cell analysis image. Calculate the feature amount.
Therefore, since the feature amount is calculated only from the analysis target region in the cell type identification image and the cell analysis image, more accurate analysis can be performed efficiently.

Further, according to the present embodiment, the control unit 61 selects a region of interest based on the complex feature amount calculated from the feature amount related to the cell type and the feature amount related to the target substance.
Therefore, by using the complex feature amount, it is possible to quantitatively evaluate only the target substance existing in the target cell type, not the arbitrary target substance existing in the tissue sample, and the distribution of the target substance, etc. It becomes possible to grasp the above more accurately and perform analysis and the like with high accuracy.

Further, according to the present embodiment, the control unit 61 identifies the first spot from the cell type identification image based on the feature amount related to the cell type, and from the cell analysis image based on the feature amount related to the target substance. The second spot is specified, and the region where the first spot and the second spot overlap each other is selected as the region of interest.
Therefore, the method of selecting the region of interest is expanded, and the region of interest desired by the user can be selected.

Further, according to the present embodiment, the control unit 61 has a plurality of cell type identification images for identifying different cell types and a plurality of cell analysis images for analyzing the identified plurality of cell types. And get.
Therefore, more detailed analysis becomes possible.

Further, according to the present embodiment, the tissue image is a whole slide image obtained by photographing the entire tissue sample 50.
Therefore, in pathological diagnosis using a hole slide image, it is possible to efficiently perform highly accurate analysis.

Further, according to the present embodiment, the acquisition means are a first photographing means for acquiring a tissue image (a low-magnification objective lens of the first image acquisition unit 20 or the second image acquisition unit 30) and a first image. A second photographing means (such as a high-magnification objective lens of the second image acquisition unit 30) having a higher photographing magnification than the photographing means is provided, and the re-photographing is performed by the second photographing means in the extracted specific area.
Therefore, an enlarged image of the selected region of interest can be taken, and more accurate analysis becomes possible.
A low-magnification microscope may be used in addition to the whole slide scanner as long as the WSI can be obtained at a low magnification.
Further, the first photographing means and the second photographing means may be configured as different photographing means in the same photographing device, or may be configured as two different photographing devices.

Further, according to the present embodiment, an analysis means for analyzing the re-photographed image obtained by re-photographing the region of interest by the second photographing means is further provided.
Therefore, it is possible to analyze the enlarged image of the selected region of interest.

[Other]
In addition, the detailed configuration of each device constituting the image processing system 1 and the detailed operation of each device can be appropriately changed without departing from the gist of the present invention.

For example, in the above embodiment, a tissue section is targeted as a biological sample, and the tissue sample 50 is stained with an immunostaining agent containing fluorescent substance-accumulated nanoparticles as a fluorescent label. The target of the biological sample may be a cultured cell or a gene (DNA).

The present invention can be used in an image processing system, an image processing method, and a program that enable highly accurate analysis to be performed efficiently.

1 Image processing system 10 Microscope device 20 First image acquisition unit (first imaging means)
21 Bright field light source 22 First image sensor 30 Second image acquisition unit (first imaging means, second imaging means)
31 Transmission light source 32 Excitation light source 33 Second image sensor 40 Stage 50 Tissue specimen 60 Control device 61 Control unit (acquisition means, selection means, identification means, analysis means)
62 Storage unit 63 Image processing unit 64 Communication unit 70 Display device 80 Database

Claims (14)

1. An acquisition method for acquiring a tissue image of a tissue sample,
   A selection means for selecting an image of a region of interest for re-imaging by the acquisition means from the tissue image acquired by the acquisition means.
   Have,
   The acquisition means acquires a cell type identification image for identifying one or more cell types in the tissue specimen and one or more cell analysis images for analyzing the identified cells.
   The selection means selects the region of interest based on the feature amount related to the cell type calculated from the cell type identification image and the feature amount related to one or more target substances calculated from the cell analysis image. Image processing system.
2. The image processing system according to claim 1, wherein the tissue specimen is morphologically observed and stained so that the one or more cell types can be identified by the acquisition means.
3. The image processing system according to claim 1 or 2, further comprising an identification means for identifying an analysis target region in the cell type identification image by machine learning or an image feature classification method.
4. The image processing system according to claim 3, wherein the selection means calculates a feature amount related to the cell type from the analysis target region of the cell type identification image.
5. The image processing system according to claim 4, wherein the selection means calculates a feature amount related to the target substance from a region corresponding to the analysis target region of the cell analysis image.
6. The selection means according to any one of claims 1 to 5, wherein the selection means selects the region of interest based on the complex feature amount calculated from the feature amount related to the cell type and the feature amount related to the target substance. Image processing system.
7. The selection means identifies the first spot from the cell type identification image based on the feature amount related to the cell type, and selects the second spot from the cell analysis image based on the feature amount related to the target substance. The image processing system according to any one of claims 1 to 5, wherein a region in which the first spot and the second spot overlap each other is specified and selected as the region of interest.
8. From claim 1, the acquisition means acquires a plurality of the cell type identification images for identifying different cell types and a plurality of cell analysis images for analyzing the specified plurality of cell types. 7. The image processing system according to any one of 7.
9. The image processing system according to any one of claims 1 to 8, wherein the tissue image is a whole slide image obtained by photographing the entire tissue sample.
10. The acquisition means
    A first photographing means for acquiring the tissue image and
    A second photographing means for re-photographing the region of interest and acquiring a re-photographed image by a method different from that of the first photographing means.
    The image processing system according to any one of claims 1 to 9.
11. The image processing system according to claim 10, wherein the second photographing means re-photographs at a higher photographing magnification than the first photographing means.
12. The image processing system according to claim 10 or 11, further comprising an analysis means for analyzing the re-photographed image acquired by the second image-taking means.
13. The acquisition process to acquire the tissue image of the tissue sample,
    A selection step of selecting an image of a region of interest for rephotographing in the acquisition step from the tissue image acquired in the acquisition step, and a selection step.
    Have,
    In the acquisition step, an image for identifying one or more cell types in the tissue sample and an image for analyzing one or more cells for analyzing the identified cells are acquired.
    The selection step is based on the feature amount related to one or more cell types calculated from the cell type identification image and the feature amount related to one or more target substances calculated from the cell analysis image. An image processing method that selects the area of interest.
14. Computer,
    Acquisition method for acquiring tissue images of tissue specimens,
    A selection means for selecting an image of a region of interest for re-imaging by the acquisition means from the tissue image acquired by the acquisition means.
    To function as
    The acquisition means acquires a cell type identification image for identifying one or more cell types in the tissue specimen and one or more cell analysis images for analyzing the identified cells.
    The selection means is based on the feature amount related to one or more cell types calculated from the cell type identification image and the feature amount related to one or more target substances calculated from the cell analysis image. A program that selects the area of interest.

Images