WO2019225325A1 - Observation device, method for operating observation device, and observation control program - Google Patents

Abstract

The present invention provides an observation device, a method for operating an observation device and an observation control program which allow for the detection of the excess or shortage of the volume of a medium. The observation device comprises: a reception unit for receiving the shape information of a container storing an observation target including a medium; an acquisition unit for acquiring phase-difference image information indicating an observation target stored in the container corresponding to the shape information received by the reception unit; and a detection unit for detecting whether or not the medium volume estimated on the basis of the phase-difference image information acquired by the acquisition unit deviates from the threshold range corresponding to the shape information received by the reception unit. The threshold range is selected from a plurality of threshold ranges for the volume of a medium which is predetermined for each type of shape information corresponding to different types of containers.

Description

Observation apparatus, operation method of observation apparatus, and observation control program

The present disclosure relates to an observation apparatus that observes an observation object accommodated in a container, an operation method of the observation apparatus, and an observation control program.

Conventionally, pluripotent stem cells such as ES (Embryonic Stem) cells and iPS (Induced Pluripotent Stem) cells and differentiation-induced cells are photographed with a microscope, etc. A determination method has been proposed. And pluripotent stem cells such as ES cells and iPS cells have the ability to differentiate into cells of various tissues, and are attracting attention as cells that can be applied in regenerative medicine, drug development, disease elucidation, etc. Yes. When photographing cells with a microscope, in order to obtain a wide-field image with a high magnification, for example, the inside of a culture vessel such as a well plate is scanned with an imaging optical system, and an image for each observation position is photographed. It has been proposed to perform so-called tiling photography that combines images for each observation position.

As described above, there is a phase contrast microscope, for example, as a microscope used when photographing cells. In a general phase contrast microscope, ring-shaped illumination light is irradiated onto an observation target, and direct light and diffracted light that have passed through the observation target are incident on a phase plate. The direct light is attenuated by the ring part of the phase plate, and the diffracted light passes through the transparent part of the phase plate, and the direct light and the diffracted light are imaged to form an image with a contrast of light and dark. You can shoot.

When an observation target including a culture medium is observed with a phase contrast microscope, for example, when the culture medium is a culture solution, the shape pattern of the meniscus formed on the liquid surface of the culture solution varies depending on the amount of the culture solution. If the shape pattern of the meniscus is different, the effect on the direct light and diffracted light incident on the phase plate will be different when observing the observation target with a phase contrast microscope. Will change.

However, when the culture solution is put into the culture container, the amount of the medium may not be constant due to variations in humans when manually placed by the user and due to a malfunction of the electric pipette when placed with the electric pipette. In addition, the amount of the medium may vary due to evaporation of the medium. As described above, since there are a plurality of factors that change the amount of the medium, it is very important to grasp whether or not the amount of the medium stored in the culture container is an appropriate amount.

On the other hand, in order to grasp the amount of medium contained in the culture vessel, there is a method of measuring and monitoring the weight, but since a sensor for measuring the weight is required, the structure of the microscope apparatus becomes complicated. .

Therefore, for example, in Patent Document 1, the height of the culture medium is calculated by extracting the contour of the bottom surface of the culture vessel and the contour of the surface of the culture medium in the photographed image, so that the amount of the medium can be determined without adding a new configuration. A method of grasping is disclosed. Further, in Patent Document 2, by adjusting the refraction of light caused by the liquid surface shape by estimating the liquid surface shape of the liquid in the container based on the luminance distribution of the phase difference image to be observed, A method for capturing a phase difference image in which the influence of a meniscus is suppressed is disclosed. Patent Document 3 discloses a method of estimating a medium amount based on RGB (Red-Green-Blue) signal values of a color image, and outputting an abnormal signal when the estimated value of the medium amount is out of the set range. Yes.

International Publication WO2008 / 146474 JP 2017-151132 A International Publication No. WO2007 / 055317

However, in Patent Document 1, the amount of the medium is grasped without adding a new configuration, but it is necessary to acquire an image for grasping the amount of the medium separately from the cell observation image. Further, Patent Document 2 describes a method for estimating the liquid level shape in the container, but does not describe estimating the liquid amount in the container, and an appropriate amount in which the medium amount is set in advance. It is not described about detecting whether or not. Patent Document 3 discloses a container observation system for acquiring a color image representing an entire observation region used for estimating a medium amount in addition to a microscopic observation system for acquiring a sample observation image for observing a sample with a microscope. is required.

The present disclosure has been made in view of the above circumstances, and an observation apparatus, an operation method of the observation apparatus, and observation control that can detect a container having an excessive or insufficient amount of a medium without acquiring an image other than a phase difference image. The purpose is to provide a program.

The observation device of the present disclosure includes a receiving unit that receives shape information of a container that contains an observation target including a culture medium;

An acquisition unit for acquiring phase difference image information representing an observation object accommodated in a container corresponding to the shape information received by the reception unit;

A detection unit that detects whether or not the amount of the medium estimated based on the phase difference image information acquired by the acquisition unit is out of a threshold range corresponding to the shape information received by the reception unit;

The threshold range is a threshold range selected from a plurality of threshold ranges of a predetermined amount of medium for each shape information of different types of containers.

In the present disclosure, “container shape information” means information that can specify the amount of the storage unit that stores the observation target.

Further, the observation device of the present disclosure adjusts the medium amount of the container detected to be out of the threshold range to an appropriate amount when the detection unit detects that the detection unit is out of the threshold range, or the medium in the container You may provide the adjustment part which performs the presentation for adjusting quantity to an appropriate quantity.

Moreover, the observation apparatus of this indication may be provided with the alerting | reporting part which alert | reports that the amount of culture media is excessive or insufficient when it detects that a detection part remove | deviates from the range of a threshold value.

In the present disclosure, “notify that the amount of medium is excessive or insufficient” indicates that the medium amount is increased, the medium amount is decreased, and the medium is replaced. To include.

Further, in the present disclosure, the “notification unit” includes a display that displays a message or the like visually, a sound reproduction device that displays an audible display by outputting sound, a printer that records on a recording medium such as paper, and displays it permanently and visually. Means a communication means such as a telephone, a display lamp, etc., and at least two or more of the display, the sound reproduction device, the printer, the communication means and the display light may be combined.

In addition, the observation device of the present disclosure again displays image information representing an observation target housed in a container that is detected to be out of the threshold range by the detection unit after the medium amount is adjusted to an appropriate amount. You may get it.

In the observation device of the present disclosure, the detection unit may acquire luminance distribution information based on the phase difference image information, and estimate the medium amount based on a relationship between the preset luminance distribution information and the medium amount. it can.

In the observation device according to the present disclosure, the threshold range may be provided for each imaging condition for imaging the observation target.

Further, in the observation device of the present disclosure, the relationship between the phase difference image information and the amount of medium is set for each shape information of different types of containers and for each coordinate position of the observation region in the container,

The detection unit may estimate the medium amount from the set relationship.

Further, in the observation apparatus according to the present disclosure, the detection unit uses the phase difference image information representing the entire container,

May be estimated.

In the observation device of the present disclosure, the detection unit may estimate the amount of the culture medium using phase difference image information that represents the central region of the container.

The operation method of the observation device of the present disclosure is an operation method of the observation device including a reception unit, an acquisition unit, and a detection unit,

The accepting unit accepts shape information of the container containing the observation object including the culture medium,

The acquisition unit acquires phase difference image information representing an observation object accommodated in a container corresponding to the shape information received by the reception unit,

The detection unit detects whether or not the amount of the medium estimated based on the phase difference image information acquired by the acquisition unit is out of the threshold range corresponding to the shape information received by the reception unit. , Including a threshold range selected from a plurality of threshold ranges of a predetermined amount of medium for each shape information of different types of containers.

An observation control program of the present disclosure includes a computer,

It functions as a reception unit, an acquisition unit, and a detection unit included in the observation apparatus.

Note that the operation method of the observation apparatus according to the present disclosure may be provided as a program that causes a computer to execute the method.

Another observation device according to the present disclosure includes a memory for storing instructions for causing a computer to execute,

A processor configured to execute stored instructions, the processor comprising:

Accept the shape information of the container containing the observation target including the culture medium,

Obtain phase difference image information representing an observation object accommodated in a container corresponding to the received shape information,

A process for detecting whether or not the amount of the medium estimated based on the acquired phase difference image information is out of the threshold range corresponding to the received shape information is performed. It is the range of the threshold value selected from the range of the some threshold value of the culture medium amount preset for every shape information.

According to the present disclosure, the shape information of the container containing the observation target including the culture medium is received, the phase difference image information representing the observation target stored in the container corresponding to the received shape information is acquired, and the acquired phase difference is acquired. The amount of the medium estimated based on the image information is a threshold range corresponding to the received shape information, and from a plurality of threshold ranges of the medium amount set in advance for each shape information of different types of containers Since it is detected whether or not the selected threshold value is deviated, a container deviating from the threshold value can be detected as an excessive or insufficient medium amount container without acquiring an image other than the phase difference image. As a result, by adjusting the amount of medium or exchanging the medium in a container with an excessive or insufficient amount of medium, the observation object can be imaged with an appropriate amount of medium, which reduces the image quality of the captured image. Can be suppressed.

The figure which shows schematic structure of one Embodiment of the microscope observation system to which the observation apparatus of this indication is applied. Diagram showing the scanning position of the observation position on the well plate with a solid line Schematic block diagram showing the configuration of the medium amount detection unit The flowchart which shows the process by the culture medium amount detection part of FIG. The figure for explaining the processing of the microscope observation system Diagram for explaining the medium amount estimation method The schematic block diagram which shows the structure of the culture medium amount detection part of 2nd Embodiment. The flowchart which shows the process by the culture medium amount detection part of FIG. Conceptual diagram showing an example of an aspect in which an observation control program is installed in a microscope control device from a storage medium in which the observation control program is stored

Hereinafter, a microscope observation system to which an embodiment of the observation device, the operation method of the observation device, and the observation control program of the present disclosure is applied will be described in detail with reference to the drawings. A microscope observation system according to an embodiment of the present disclosure includes a microscope device 1, a microscope control device 2, a display device 3, and an input device 4, as shown in FIG. Note that the microscope control device 2 includes the observation device of the present disclosure.

In the present embodiment, the microscope apparatus 1 is a phase contrast microscope, and acquires, for example, a phase contrast image of cultured cells as an observation target. Specifically, the microscope apparatus 1 includes an illumination light irradiation unit 10, an imaging optical system 30, a stage 61, and a photographing unit 40, as shown in FIG.

On the stage 61, an observation container S such as cells and a culture vessel 71 containing a culture medium C as a medium are installed. A rectangular opening is formed at the center of the stage 61. The culture vessel 71 is installed on the member forming the opening, and the phase difference image of the observation target S in the culture vessel 71 is configured to pass through the opening.

In the culture vessel 71 installed on the stage 61, a cultured cell group (cell colony) is arranged as the observation target S. The cultured cells include pluripotent stem cells such as iPS cells and ES cells, nerves induced to differentiate from stem cells, skin, myocardium and liver cells, and skin, retina, heart muscle, blood cells, nerves and the like removed from the human body. There are organ cells.

As the culture container 71, for example, a plurality of wells (corresponding to the container of the present disclosure) arranged on a well plate are used, but not limited thereto, a petri dish, a flask, a dish, or the like may be used. In the present embodiment, a well plate 70 in which a plurality of wells are arranged is used, and each well of the well plate 70 corresponds to the culture vessel 71. Hereinafter, the culture vessel 71 may be collectively referred to as the well 71.

In the culture container 71 installed on the stage 61, the bottom surface in the culture container 71 is the installation surface P1 of the observation object S, and the observation object S is arranged on the installation surface P1. The culture solution 71 is filled in the culture vessel 71. In the present embodiment, the cells to be cultured in the culture medium are the observation objects S. However, the observation objects S are not limited to those in the culture liquid, but water, formalin, ethanol, methanol, and the like. The cells fixed in the liquid may be the observation object S.

The illumination light irradiation unit 10 irradiates the observation target S accommodated in the culture vessel 71 on the stage 61 with illumination light for so-called phase difference measurement, and in this embodiment, the phase difference Ring-shaped illumination light is irradiated as illumination light for measurement.

Specifically, the illumination light irradiation unit 10 of the present embodiment has a white light source 11 that emits white light for phase difference measurement and a ring-shaped slit, and the white light emitted from the white light source 11 is incident thereon. The slit plate 12 that emits ring-shaped illumination light, and the ring-shaped illumination light emitted from the slit plate 12 are incident, and the condenser lens 13 that irradiates the incident ring-shaped illumination light to the observation object S is provided.

The slit plate 12 is provided with a ring-shaped slit that transmits white light to the light-shielding plate that blocks the white light emitted from the white light source 11, and the ring shape is obtained when the white light passes through the slit. Illumination light is formed. The condenser lens 13 converges the ring-shaped illumination light emitted from the slit plate 12 toward the observation target S.

The imaging optical system 30 forms an image of the observation target S in the culture vessel 71 on the photographing unit 40, and includes an objective lens 31, a phase plate 32, and an imaging lens 33.

The phase plate 32 is obtained by forming a phase ring on a transparent plate that is transparent with respect to the wavelength of the ring-shaped illumination light. Note that the size of the slit of the slit plate 12 described above is in a conjugate relationship with this phase ring.

In the phase ring, a phase film that shifts the phase of incident light by ¼ wavelength and a neutral density filter that attenuates incident light are formed in a ring shape. When the direct light incident on the phase plate 32 passes through the phase ring, the phase is shifted by ¼ wavelength and its brightness is weakened. On the other hand, most of the diffracted light diffracted by the observation object S passes through the transparent plate portion of the phase plate 32, and its phase and brightness do not change.

The imaging lens 33 receives direct light and diffracted light that have passed through the phase plate 32, and forms an image of these lights on the imaging unit 40.

The imaging unit 40 receives the image of the observation target S formed by the imaging lens 33 and receives the observation target S.

An image sensor that captures S and outputs a phase difference image is provided. As the image sensor, a charge-coupled device (CCD) image sensor, a complementary metal-oxide semiconductor (CMOS) image sensor, or the like can be used.

The stage 61 is driven by the drive unit 62 and moves in the X direction and the Y direction orthogonal to each other in the horizontal plane. FIG. 2 is a diagram showing the scanning position of the observation position with a solid line when the well plate 70 having six wells 71 is used. As shown in FIG. 2, the imaging optical system 30 moves along a solid line 77 from the scanning start point 75 to the scanning end point 76. That is, the observation position of the observation region on the culture vessel 50 by the imaging optical system 30 is scanned in the positive direction in the X direction (right direction in FIG. 2) and then moved in the Y direction (downward direction in FIG. 2). , Reverse negative direction

Scanning is performed in the left direction in FIG. Next, the observation position moves again in the Y direction and is scanned again in the positive direction. As described above, the imaging optical system 30 scans the inside of the culture vessel 50 in a two-dimensional manner by repeatedly performing reciprocal movement in the X direction and movement in the Y direction.

That is, as the stage 61 moves, the observation position of the observation region smaller than the well 71 in each well 71 of the well plate 70 moves along the solid line 77, so that phase difference images of the observation regions at different observation positions are taken. Acquired by the unit 40. The phase difference image for each observation area is output to the microscope control device 2 by the control unit 21 and stored in the secondary storage unit 25. Note that the phase difference image acquired by the imaging unit 40 is stored in association with the coordinate position on the well plate 70 so that it can be determined which well 71 is imaged in the well plate 70.

In the present embodiment, the phase difference image for each observation region in the well plate 70 is acquired by moving the stage 61. However, the present invention is not limited to this, and the imaging optical system 30 is moved with respect to the stage 61. By doing so, a phase difference image for each observation region may be acquired. Alternatively, both the stage 61 and the imaging optical system 30 may be moved. In the present embodiment, scanning is performed using the scanning locus shown in FIG. 2, but the present disclosure is not limited thereto, and scanning may be performed using another scanning locus such as a spiral shape.

The microscope control device 2 includes a computer including a CPU (Central Processing Unit) 20, a primary storage unit 24, a secondary storage unit 25, an external I / F (Interface) 28, and the like. The CPU 20 includes a control unit 21, a medium amount detection unit 22, and an image processing unit 23, and controls the entire microscope observation system.

The control unit 21 acquires the phase difference image of the observation target S by controlling the illumination light irradiation unit 10, the drive unit 62 that drives the stage 61, the imaging optical system 30, and the imaging unit 40. The control unit 21 also functions as a display control unit that causes the display device 3 to display one composite phase difference image generated by combining the phase difference images of the respective observation positions photographed by the microscope device 1. . Further, the control unit 21 causes the photographing unit 40 to photograph the observation target S. In the present embodiment, since the culture container 71 is a plurality of wells 71 arranged in the well plate 70, the control unit 21 causes the imaging unit 40 to image each observation region in each well 71 to represent the well 71. A plurality of phase difference images are acquired.

The medium amount detection unit 22 detects excessive medium amount and insufficient medium amount. The medium amount detection unit 22 will be described in detail later. In the present embodiment, the medium amount detection unit 22 can detect both an excessive amount of medium and an insufficient amount of medium. However, the present disclosure is not limited to this, and the medium amount detection unit 22 has an excessive amount of medium. Only a shortage of the medium may be detected.

The image processing unit 23 performs various processes such as gamma correction, luminance / color difference conversion, and compression processing on the image signal acquired by the photographing unit 40. Further, the image processing unit 23 outputs an image signal obtained by performing various processes to the control unit 21 for each frame at a specific frame rate. Further, the image processing unit 23 generates one composite image representing the whole well 71 by combining the phase difference images of the respective observation regions photographed by the microscope apparatus 1. The image processing unit 23 according to the present embodiment performs a process of creating a luminance profile representing a luminance distribution with respect to the position of the image based on the composite image. As the luminance distribution, any one of an absolute value, a relative value, and a variance value of the entire image can be used, and a luminance profile can be created using a known technique.

The display device 3 displays the composite image generated by the image processing unit 23 and includes, for example, a liquid crystal display. Further, the display device 3 may be configured by a touch panel and may also be used as the input device 4.

The input device 4 includes a mouse and a keyboard, and receives various setting inputs by the user.

The primary storage unit 24 is a volatile memory used as a work area or the like when executing various programs. An example of the primary storage unit 24 is a RAM (Random Access Memory). The secondary storage unit 25 is a non-volatile memory that stores various programs, various parameters, and the like in advance, and stores appropriate range information 26 of the present disclosure. In the secondary storage unit 25, an embodiment of the observation control program 27 of the present disclosure is installed. The microscope control apparatus 2 functions when the observation control program 27 is executed by the CPU 20. An example of the secondary storage unit 25 is an EEPROM (Electrically Erasable Programmable Read-Only Memory) or a flash memory. The external I / F 28 controls transmission / reception of various information between the microscope apparatus 1 and the microscope control apparatus 2. The CPU 20, the primary storage unit 24, and the secondary storage unit 25 are connected to the bus line 29. External I / F

28 is also connected to the bus line 29.

The observation control program 27 is recorded and distributed on a recording medium such as a DVD (Digital Versatile Disc) and a CD-ROM (Compact Disc Read Only Memory), and is installed in the computer from the recording medium. Alternatively, the observation control program 27 is stored in a storage device or network storage of a server computer connected to the network in a state where it can be accessed from the outside, and is installed after being downloaded to the computer in response to a request from the outside. You may be made to do.

The appropriate range information 26 is information on a range of a plurality of threshold values for a predetermined amount of medium for each shape information of different types of culture vessels 71. In the present embodiment, a table that associates the shape information of the culture vessel 71 with the appropriate range information is stored in the secondary storage unit 25. For example, when the culture container 71 is each well 71 arranged on the well plate 70, the accommodation amount of each well accommodation unit varies depending on the number of the wells 71 arranged on the well plate 70. Further, even if the number of wells 71 is the same in the well plate 70, the shape of each well may differ depending on the manufacturer. Further, the shape of the well 71 may differ depending on the position of the well 71 of the well plate 70. If the shape of the well is different, for example, when the culture solution C is contained in the well as a medium, the shape pattern of the meniscus formed on the liquid surface of the culture solution C is different. If the meniscus shape pattern is different, the effect on the direct light and the diffracted light incident on the phase plate 32 will be different when observing the observation object with the microscope apparatus 1, so that the image quality of the phase difference image will be different. End up.

Therefore, for example, the control unit 21 is provided for each type of the well plate 70 in which 96, 48, 24, 12, and 6 wells 71 are arranged for each manufacturer. For each position of each well 71 on the plate 70, an appropriate medium amount threshold range is stored as the appropriate range information 26. The control unit 21 stores a different amount of medium in advance in the photographing unit 40 for each manufacturer and for each type of well plate 70 and for each position of each well 71 on the well plate 70. The phase difference image is acquired by photographing the well 71, and the lower limit value and the upper limit value of the amount of culture medium in which the phase difference image having an appropriate image quality for observing the observation target can be acquired from the acquired phase difference image. It is memorized as “threshold range of medium amount”.

The “medium amount threshold range” is provided for each imaging condition for imaging an observation target. When the magnification of the objective lens 31 of the microscope apparatus 1 is high, the size of the slit of the phase plate 32 becomes narrow, so that the axis shift with respect to the change of the medium amount, that is, the optical axis of the observation light transmitted through the culture vessel and the imaging optics. The sensitivity of deviation between the optical axis of the system becomes high. In other words, the amount of light that passes through the phase plate 32 increases, and the luminance of the acquired phase difference image increases. Therefore, the influence of the change in the medium amount on the phase difference image is increased. Therefore, the range of the medium amount threshold is set for each magnification of the objective lens 31.

Here, the control unit 21 is based on the phase difference image representing the well 71 acquired by photographing the well 71 containing different amounts of the medium, which is performed to determine the range of the medium amount threshold value as described above. The image processing unit 23 creates a luminance profile, and the control unit 21 stores the medium amount and the luminance profile in the phase difference image in association with each other in the secondary storage unit 25 (see FIG. 6). Here, the medium amount and the luminance profile are stored as a table in which each is associated. In the present embodiment, a table in which the medium amount and the luminance profile are associated with each other is stored. However, the present disclosure is not limited to this, and the table may not be used as long as the medium amount can be derived from the luminance profile.

In the present embodiment, a table in which the shape information of the well 71 is associated with the appropriate range information 26 is stored. However, the present disclosure is not limited to this, and if the appropriate range information 26 can be derived from the shape information of the well 71, It is not necessary to use a table. And the detection part 52 mentioned later reads the appropriate range information 26 of the well 71 which has the shape information from the secondary memory | storage part 25 based on the shape information of the well 71 which the reception part 50 mentioned later received.

In the above description, the case where the general-purpose computer functions as the microscope control apparatus 2 has been described. However, the general-purpose computer may be implemented by a dedicated computer. The dedicated computer may be firmware that executes a program recorded in a nonvolatile memory such as a built-in ROM (Read Only Memory) or a flash memory. Furthermore, a dedicated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (field programmable gate arrays) that permanently stores a program for executing at least a part of the functions of the microscope control device 2 May be provided. Alternatively, the program instructions stored in the dedicated circuit may be combined with the program instructions executed by a general-purpose CPU programmed to use the program of the dedicated circuit. As described above, program instructions may be executed by any combination of hardware configurations of computers.

Here, the medium amount detection unit 22 will be described in detail. FIG. 3 is a schematic block diagram showing the configuration of the medium amount detection unit 22. As shown in FIG. 3, the medium amount detection unit 22 includes a reception unit 50, an acquisition unit 51, a detection unit 52, and a notification unit 53.

The receiving unit 50 receives the shape information of the well 71 that contains the observation target including the culture medium. The receiving unit 50 receives the shape information of the well plate 70 installed on the stage 61, which is input from the input device 4 by the user, as the shape information of the well 71. Further, the shape information of the well 71 is not input by the user, but a barcode or the like is given to the well plate 70, and the reception unit 50 can read the barcode, for example, and is connected to the external I / F 28. The shape information may be received by reading a barcode given to the well plate 70 via the external device. Note that the shape information of the well 71 may be information that can identify the amount of accommodation of the accommodation unit that accommodates the observation target, and may be, for example, a manufacturer's model number as long as it is associated with the accommodation amount in advance. It may be a production number.

The acquisition unit 51 acquires a phase difference image representing an observation object accommodated in the well 71 corresponding to the shape information received by the reception unit 50. In the present embodiment, the acquisition unit 51 is a phase difference image acquired by the imaging unit 40, and is a well plate 70 that has received shape information by the reception unit 50, and is a well plate 70 disposed on the stage 61. Phase difference image information representing the observation object accommodated in each well 71 is acquired.

In the detection unit 52, the amount of the medium estimated based on the phase difference image information acquired by the acquisition unit 51 is within a threshold range corresponding to the shape information received by the reception unit 50, and different types of wells 71 are used. It is detected whether or not the shape information is out of the threshold range selected from the plurality of threshold ranges of the medium amount set in advance for each shape information. The detection process performed by the detection unit 52 as to whether or not the predetermined amount of the medium amount is out of the predetermined range will be described in detail later.

The notification unit 53 notifies that the medium amount is excessive or insufficient when the detection unit 52 detects that the medium amount is outside the preset threshold range of the medium amount. In the present embodiment, the notification unit 53 includes the display device 3, and specifically displays the words “excess medium amount” or “insufficient medium amount” as an example in response to a command from the control unit 21. In the present embodiment, the notification unit 53, that is, the display device 3 displays the words “excess medium amount” or “insufficient medium amount”, but the present disclosure is not limited to this, "Please increase the media", "Please reduce the amount of medium", "Please replace the medium", etc. may be displayed. Further, the notification unit 53 is not limited to the one configured by the display device 3, and the notification unit 53 is a communication device such as an audio reproduction device, a printer, an email, a telephone, or the like connected via the external I / F 28. It may be configured by means, an indicator lamp, or the like, or may be configured by combining at least two or more of the display, the sound reproduction device, the printer, the communication means, and the indicator lamp.

Next, processing by the medium amount detection unit 22 of the present embodiment will be described. FIG. 4 is a flowchart showing processing by the medium amount detection unit 22 of FIG. 3, FIG. 5 is a diagram for explaining processing of the microscope observation system, and FIG. 6 is a diagram for explaining a medium amount estimating method.

First, the receiving unit 50 acquires the shape information of the well 71 as described above (step S1). In the present embodiment, the accepting unit 50 accepts the shape of each of the 96 wells 71 arranged on the well plate 70 shown in FIG. 5 as shape information (step S1).

Next, the acquisition unit 51 reads out and acquires the phase difference images for each of the plurality of observation regions stored in the secondary storage unit 25 by causing the imaging unit 40 to image the well plate 70 by the control unit 21 (step S2). In the present embodiment, one well 71 is represented by nine observation areas A1 to A9 as shown in FIG. The acquisition unit 51 acquires a combined image obtained by combining the phase difference images representing the nine observation regions as a phase difference image representing the well 71. Note that the number of observation regions is not limited to nine, and may be, for example, 16 observation regions, 25 observation regions, or depending on the performance of the microscope apparatus 1 and imaging conditions. It is changed appropriately.

Next, the detection unit 52 estimates the amount of medium (step S3). For example, as illustrated in FIG. 5, the acquisition unit 51 acquires a phase difference image of the entire well 71 </ b> A in the fifth row from the top and the fifth column from the left in FIG. 5 of the well plate 70. The detection unit 52 causes the image processing unit 23 to create a luminance profile based on the acquired phase difference image via the control unit 21, and acquires the generated luminance profile.

Next, the detection unit 52 refers to the luminance profile stored in the secondary storage unit 25 in association with the medium amount, and estimates the medium amount of the luminance profile acquired above. As a method for obtaining the correlation between the brightness profile stored in the secondary storage unit 25 and the brightness profile acquired above, for example, a correlation function may be used to obtain the correlation between these brightness profiles. . In addition to the method using the correlation function, for example, the inflection points in the luminance profiles acquired as described above are obtained when stored in the secondary storage unit 25, and the luminance values of the inflection points corresponding to these luminance profiles are obtained. Find the total difference. And the total value mentioned above is calculated | required about each of several brightness | luminance profile for every different culture medium amount previously memorize | stored in the secondary memory | storage part 25, and the brightness | luminance profile with the smallest total value is specified as a brightness | luminance profile with the highest correlation. It may be.

And the detection part 52 specifies the culture medium quantity with the highest correlation with the brightness | luminance profile acquired above, and presumes the specified culture medium quantity as the culture medium quantity in the well 71A.

Next, the detection unit 52 has a medium amount threshold range corresponding to the shape information of the well 71 received by the receiving unit 50 from the medium amount threshold range stored in the appropriate range information 26, and Further, in FIG. 5 of the well plate 70, the range of the medium amount threshold of the well 71 corresponding to the position of the well 71A in the fifth row from the top and the fifth column from the left is read, and the estimated medium amount is within the above threshold range. It is detected whether it is outside (step S4).

When the detection unit 52 detects that the estimated medium amount is outside the above-described threshold range (step S4; YES), the detection unit 52 detects an excessive medium amount or an insufficient medium amount (step S5). That is, the detecting unit 52 detects an excessive amount of the medium if the estimated amount of the medium is larger than the upper limit value of the threshold range, and detects an insufficient amount of the medium if the estimated amount is smaller than the lower limit value of the threshold range.

Next, the notification unit 53 notifies the display device 3 of the words “excessive amount of medium” or “insufficient amount of medium” via the control unit 21, thereby notifying that the amount of medium is excessive or insufficient (step S6), the processing by the medium amount detection unit 22 ends.

On the other hand, when the detection unit 52 detects in step S4 that the estimated medium amount is within the above-described threshold range (step S4; NO), the medium amount is not excessive or insufficient, and is an appropriate amount, so the medium amount detection unit The process by 22 is completed.

Similarly, the above process is performed on all 96 wells 71 to detect whether or not the medium amount is out of the threshold range, thereby detecting whether or not the medium amount is an appropriate amount.

According to the above, since the user can know that the medium amount is excessive and the medium amount is insufficient in the well 71A shown in FIG. 5, the medium in the well 71A is reduced or added to adjust the medium amount, or the medium amount is exchanged. By doing so, the well 71A can be adjusted to an appropriate medium amount. As a result, the observation target can be photographed with an appropriate amount of medium, so that deterioration of the image quality of the phase difference image can be suppressed.

The acquisition unit 51 may acquire again the phase difference image information representing the observation target accommodated in the well 71A after the medium amount in the well 71A is adjusted to an appropriate amount. In this case, for example, the user operates the input device 4 to input that the well 71A has an appropriate amount of medium. When the control unit 21 detects that the well 71A has an appropriate amount of medium, that is, detects that the amount of medium in the well 71 is within the above-described threshold value range, the control unit 21 causes the photographing unit 40 to image the well 71A again. The acquisition unit 51 acquires the captured phase difference image. Thereby, the phase difference image in which the image quality is lowered due to the excessive amount of medium or the insufficient amount of the medium can be changed to a phase difference image with improved image quality, which is taken with an appropriate amount of medium.

In the above-described embodiment, the detection unit 52 detects whether or not the medium amount is out of the threshold range, but the present disclosure is not limited to this, and the detection unit 52 has a medium whose estimated medium amount is appropriate. You may detect how much more and less than the amount threshold range.

In this case, the notification unit 53 notifies how much or how much is insufficient. Thereby, since the user can know how much the amount of the medium is reduced or can be adjusted to an appropriate amount of the medium, the amount of the medium can be easily adjusted.

In the above embodiment, when the detection unit 52 estimates the medium amount, the luminance profile based on the image information representing the entire well 71 is used. However, the present disclosure is not limited to this, and for example, the well 71 The luminance profile based on the phase difference image information of the central region of the observation region A5 in FIG. 5 can be used. In this case, the control unit 21 causes the image processing unit 23 to create a luminance profile based on the phase difference image representing the central region of the well 71, that is, the observation region A5, and the control unit 21 determines the medium amount and the luminance in the phase difference image. The profile is associated with and stored in the secondary storage unit 25.

The detection unit 52 is not limited to the phase difference image information of the center region of the well 71, that is, the observation region A5 in FIG. Any one of the phase difference image information of 9 may be used. In addition, phase difference image information of two or more observation regions among the observation regions A1 to 9 representing the well 71 may be used. However, since the phase difference image information in the central region of the well 71, that is, the observation region A5 in FIG. 5 represents the change in the medium amount in the well 71, it is more preferable to use the phase difference image information in the central region of the well 71. preferable.

Further, in the detection unit 52, it is more accurate to estimate the medium amount based on the phase difference image information representing the whole well 71 than to estimate the medium amount based on the phase difference image information representing the central region of the well 71. It is more preferable because it is good. However, the process of creating a luminance profile, for example, in which the medium amount is estimated based on the phase difference image information representing the central region of the well 71 rather than the medium amount is estimated based on the phase difference image information representing the entire well 71. Since the speed is high, the calculation cost can be reduced. In the present disclosure, a medium amount estimation method can be selected according to the user's desire.

Moreover, in the said embodiment, although the culture medium amount and luminance profile in the phase difference image showing the well 71 are matched and memorize | stored in the secondary memory | storage part 25, this indication is not restricted to this. As an example, as shown in FIG. 6, the medium amount and the phase difference image in the phase difference image representing the well 71 may be associated with each other and stored in the secondary storage unit 25. In this case, when the detection unit 52 estimates the medium amount, a luminance profile based on the phase difference image may be created from the phase difference image stored in the secondary storage unit 25 and referred to.

Next, another embodiment of the medium amount detection unit 22 will be described. FIG. 7 is a schematic block diagram showing the configuration of the medium amount detection unit of the second embodiment. In addition, since the culture medium amount detection part of this embodiment is provided with the adjustment part 54 instead of the alerting | reporting part 53 of the culture medium amount detection part 22 of embodiment mentioned above, since it is the same as that of the said embodiment about other structures, The description here is omitted, and only different parts will be described in detail.

When the detecting unit 52 detects whether or not the medium amount is out of the threshold range, the adjusting unit 54 of the present embodiment sets the appropriate amount of the medium in the culture vessel 71 that is detected to be out of the threshold range. Make a presentation for adjustment. Specifically, the detection unit 52 of the present embodiment detects how much the estimated medium amount is larger or insufficient than the appropriate medium amount threshold range, and the detection unit 52 detects the above threshold range. When it is detected that the medium is out of the range, the adjustment unit 54 outputs the excess or deficiency of the medium detected by the detection unit 52 to the control unit 21.

Next, processing by the medium amount detection unit 22 of the present embodiment will be described. FIG. 8 is a flowchart showing processing by the medium amount detection unit of FIG. In FIG. 8, the processing from step S21 to step S25 is the same as the processing from step S1 to step S5 in FIG.

When the detection unit 52 detects how much the medium amount estimated in step S25 is larger or insufficient than the appropriate medium amount threshold range and detects an excessive medium amount or an insufficient medium amount, the adjustment unit 54 outputs an appropriate amount of the medium amount by outputting the excess or deficiency of the medium detected by the detection unit 52 to the control unit 21 (step S26), and the processing by the medium amount detection unit 22 ends. To do.

According to the above, since the control unit 21 can know how much medium is insufficient or insufficient in the well 71A shown in FIG. 5, how much the control unit 21 is in the display device 3, for example, It can be displayed whether it is insufficient. Thereby, since the user can know how much the amount of the medium is increased or added, an appropriate amount of the medium can be obtained, so that the amount of the medium can be easily adjusted.

Further, when a device for electrically injecting the medium into the well 71 such as an electric micropipette is connected to the external I / F 28, the control unit 21 outputs an excessive or insufficient amount of the medium to this device. Thus, the well 71A can be automatically set to an appropriate medium amount.

In addition, the adjustment part 54 of this embodiment may be comprised by the apparatus which inject | pours a culture medium electrically into wells 71, such as an electric micropipette mentioned above. In this case, when detecting how much the medium amount estimated in step S25 is larger than or less than the appropriate medium amount threshold range and detecting an excessive medium amount or an insufficient medium amount, the adjustment unit 54 It is possible to automatically adjust the medium amount of the well 71 in which the medium amount is excessive or the medium amount is insufficient to an appropriate amount.

In addition, although this embodiment is provided with the adjustment part 54 instead of the alerting | reporting part 53 of embodiment mentioned above, this indication is not restricted to this, Both the alerting | reporting part 53 and the adjustment part 54 are provided. Also good.

In each of the above embodiments, the case where the observation control program 27 is read from the secondary storage unit 25 is exemplified, but it is not always necessary to store the observation control program 27 in the secondary storage unit 25 from the beginning. For example, as shown in FIG. 9, an SSD (Solid State Drive), a USB (Universal Serial Bus) memory, or a DVD-ROM (Digital)

First, the observation control program 27 may be stored in an arbitrary portable storage medium 250 such as versatile disc-Read Only Memory). In this case, the observation control program 27 of the storage medium 250 is installed in the microscope control apparatus 2, and the installed observation control program 27 is executed by the CPU 21.

In addition, the observation control program 27 is stored in a storage unit such as another computer or server device connected to the microscope apparatus 1 via a communication network (not shown), and the observation control program 27 responds to the request of the microscope apparatus 1. It may be downloaded in response. In this case, the downloaded observation control program 27 is executed by the CPU 21.

In addition, the observation control process described in the above embodiments is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, and the processing order may be changed within a range not departing from the spirit.

Further, in each of the above embodiments, the case where the observation control process is realized by a software configuration using a computer is illustrated, but the technology of the present disclosure is not limited to this. For example, instead of a software configuration using a computer, the observation control process may be executed only by a hardware configuration such as an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The observation control process may be executed by a combination of a software configuration and a hardware configuration.

All documents, patent applications and technical standards mentioned in this specification are to the same extent as if each individual document, patent application and technical standard were specifically and individually stated to be incorporated by reference. Incorporated by reference in the book.

1 Microscope device

2 Microscope control device

3 Display device

4 input devices

10 Illumination light irradiation part

11 White light source

12 Slit plate

13 Condenser lens

21 Control unit

22 Medium volume detector

23 Image processing section

24 Primary storage unit

25 Secondary storage

26 Appropriate range information

27 Observation control program

28 External I / F

29 Bus line

30 Imaging optics

31 Objective lens

32 Phase plate

33 Imaging lens

40 Shooting Department

50 reception desk

50 culture vessels

51 Acquisition Department

52 Detector

53 Notification Department

54 Adjustment unit

61 stages

62 Drive unit

70 well plate

71 each well

71 Culture vessel

71 well

75 Scanning start point

76 Scan end point

77 Solid line

250 storage media

A1 to A9 Observation area

C culture solution

P1 installation surface

S Observation target

Claims (11)

1. 
   A reception unit for receiving shape information of a container containing an observation target including a culture medium;
   
   An acquisition unit for acquiring phase difference image information representing an observation object accommodated in a container corresponding to the shape information received by the reception unit;
   
   A detection unit that detects whether or not the amount of medium estimated based on the phase difference image information acquired by the acquisition unit is out of a threshold range corresponding to the shape information received by the reception unit;
   
   The threshold value range is an observation apparatus that is a threshold value range selected from a plurality of threshold value ranges of a predetermined amount of medium for each shape information of different types of containers.
   
2. 
   When the detection unit detects that the detection unit is out of the threshold range, the medium amount in the container that is detected to be out of the threshold range is adjusted to an appropriate amount, or the medium amount in the container is set appropriately. The observation apparatus according to claim 1, further comprising an adjustment unit that makes a presentation for adjusting the amount.
   
3. 
   The observation apparatus according to claim 1, further comprising a notification unit that notifies that the amount of the medium is excessive or insufficient when the detection unit detects that the medium is out of the threshold range.
   
4. 
   The acquisition unit re-acquires image information representing an observation target stored in a container that is detected to be out of the threshold range by the detection unit after the medium amount is adjusted to an appropriate amount. 4. The observation apparatus according to any one of items 3.
   
5. 
   The detection unit obtains luminance distribution information based on the phase difference image information,
   
   The observation apparatus according to any one of claims 1 to 4, wherein the culture medium amount is estimated based on a relationship between preset luminance distribution information and the culture medium amount.
   
6. 
   The observation device according to claim 1, wherein the threshold range is provided for each imaging condition for imaging an observation target.
   
7. 
   For each shape information of different types of containers, and for each coordinate position of the observation region in the container, the relationship between the phase difference image information and the amount of medium is set,
   
   The observation device according to claim 1, wherein the detection unit estimates a medium amount from the set relationship.
   
8. 
   The observation device according to claim 7, wherein the detection unit estimates a medium amount using phase difference image information representing the entire container.
   
9. 
   The observation device according to claim 7, wherein the detection unit estimates a medium amount using phase difference image information representing a central region of the container.
   
10. 
    An operation method of an observation apparatus including a reception unit, an acquisition unit, and a detection unit,
    
    The reception unit receives shape information of a container containing an observation target including a culture medium,
    
    The acquisition unit is an observation object stored in a container corresponding to the shape information received by the reception unit.
    
    Phase difference image information representing
    
    The detection unit detects whether the amount of medium estimated based on the phase difference image information acquired by the acquisition unit is out of a threshold range corresponding to the shape information received by the reception unit, The method of operating an observation apparatus, wherein the threshold range includes a threshold range selected from a plurality of threshold ranges of a predetermined amount of medium for each shape information of different types of containers.
    
11. 
    Computer
    
    The observation control program for functioning as the said reception part, the said acquisition part, and the said detection part which are contained in the observation apparatus of any one of Claims 1-9.

Images