WO2019180808A1

WO2019180808A1 - Cell observation device

Info

Publication number: WO2019180808A1
Application number: PCT/JP2018/011002
Authority: WO
Inventors: 倫誉山川
Original assignee: 株式会社島津製作所
Priority date: 2018-03-20
Filing date: 2018-03-20
Publication date: 2019-09-26
Also published as: JP7036192B2; JPWO2019180808A1

Abstract

A display processing unit (27) presents a hologram of an entire observation target area on a display unit (202) after hologram data of an entire culture plate (13) is acquired via a microscopic observation unit (10) that is a holographic microscope. When a user designates a measurement position on the image for a phase image the user wishes to observe, a checking position designation accepting unit (25) transfers the hologram data for the designated measurement position to an external server, and the server performs phase recovery and image reconstruction to create a partial phase image. When receiving the image data making up the partial phase image, the display processing unit (27) presents the partial phase image on the display unit (202). The huge amount of hologram data for the entire observation target area requires much time to transfer and process the data, whereas the amount of hologram data at one measurement position is significantly small. Therefore, this device allows an operator to check a partial phase image to determine the suitability of the measurement and the presence or absence of defects in a sample before performing reconstruction of the phase image based on the hologram data of the entire observation target area.

Description

Cell observation device

The present invention relates to a cell observation apparatus for observing the state of a cell. More specifically, a hologram in which interference fringes between an object wave and a reference wave are recorded by a holographic microscope, and phase information and intensity information are obtained based on the hologram data. The present invention relates to a cell observation apparatus that creates a phase image, an intensity image, and the like after calculation.

In the field of regenerative medicine, research using pluripotent stem cells such as iPS cells and ES cells has been actively conducted in recent years. In general, since a cell is transparent and difficult to observe with a normal optical microscope, a phase contrast microscope has been widely used for cell observation. However, since it is necessary to perform focusing when taking a microscopic image in a phase contrast microscope, it takes a long time to measure when a microscopic image of each small region obtained by finely dividing a wide observation target region is acquired. There is a problem of too much. In order to solve this, in recent years, digital holographic microscopes using digital holography technology have been developed and put into practical use (see

Patent Documents

1 and 2, Non-Patent Document 1, etc.).

Digital holographic microscopes acquire interference fringes (holograms) formed on the detection surface of an image sensor or the like by the object light reflected or transmitted from the light source and the reference light directly reaching from the same light source. Then, phase information and amplitude (intensity) information are acquired by performing a light wave back-propagation calculation process based on the hologram, and an intensity image and a phase image are created as a reconstructed image. Such a digital holographic microscope has an advantage that phase information at an arbitrary distance can be obtained at the stage of arithmetic processing after obtaining a hologram, so that it is not necessary to perform focusing one by one during photographing.

Digital holographic microscopes include in-line, off-axis, and phase shift types. In each of these methods, the configuration of an optical system for acquiring a hologram is mainly different. In the off-axis type, the light emitted from the laser light source is usually divided into reference light and irradiation light that irradiates the object, and the object light and reference light that have passed through the object are incident on the irradiation light differently from each other. The light is incident on the image sensor at an angle. On the other hand, in the inline type, the light emitted from the laser light source is irradiated on the object without being divided, and the object light that has passed through the object and the reference light that has passed through the vicinity of the object without passing through the object are combined. Incidently incident on the image sensor. Further, in the phase shift type, a hologram whose phase is different in a plurality of stages is obtained by changing the optical path length of the reference light divided using the phase shift interferometer in a plurality of stages.

Regardless of which method is used, the digital holographic microscope needs to calculate the phase information and amplitude information of the light wave based on the acquired hologram data, and to perform processing for imaging it with a computer. For example, Patent Document 1, Non-Patent Document 2, and the like have proposed a phase recovery method by iterative calculation of light wave propagation based on a hologram for a plurality of wavelength lights acquired by an inline digital holographic microscope. Is quite complex and computationally intensive.

In a cell observation apparatus for observing living cells in culture, it is necessary to create a high-resolution image so that each cell can be observed in detail over a wide range of the entire culture plate or the entire well formed on the plate. is there. In order to obtain such a phase image and intensity image with a cell observation apparatus using a digital holographic microscope, holograms are acquired for each of a number of small regions obtained by finely dividing the observation target region (for example, the entire culture plate). It is necessary to obtain a two-dimensional distribution of phase information and amplitude information for each small region by performing arithmetic processing based on the above, and to reconstruct an image of the observation target region by synthesizing such two-dimensional distributions of a large number of small regions.

As described above, since the amount of calculation of phase information and the like and image reconstruction processing are originally large, when such a calculation is performed for each of a large number of small regions, the amount of calculation is considerably large. Therefore, even if a computer with a certain degree of performance is used, it takes time until the phase image for the observation target region is reproduced after the measurement is started.

According to the study by the present inventors, for example, the entire surface of the culture plate is divided into about 900 small areas, and holograms for a plurality of wavelengths are obtained with high resolution (for example, 4000 × 3000 pixels / sheet) for each small area. It takes 1 hour or more from the start of measurement until the phase image reconstruction process is completed. That is, the phase image and the intensity image of the sample can be observed only after 1 hour or more has elapsed since the measurement start time.

If there is a defect such as contamination of foreign matter in the culture plate to be observed, or if there is a measurement failure or omission, the measurement itself is wasted. This is after a considerable amount of time has elapsed since the time. Therefore, the measurement time is wasted, and there is a problem that the efficiency of cell observation work and analysis work is lowered.

In response to such a problem, in the existing apparatus, when a hologram image of the entire observation target area is obtained in the holographic microscope, it is immediately displayed on the computer screen for controlling the microscope, and the operator can confirm it. I am doing so. Unlike the phase image and intensity image obtained by image reconstruction, the hologram image provides almost no information about individual cells, but the outline of wells and large foreign objects in the culture plate can be sufficiently visually confirmed. . Therefore, if the operator looks at the displayed hologram image and confirms an abnormal state such as, for example, a failure in installing the culture plate in the apparatus or contamination of a large foreign object, the processing is stopped at that point and the situation is stopped. Confirm. As a result, with respect to some defects, it is possible to avoid performing a wasteful process by finding the defect before confirming the reconstructed phase image, that is, immediately after the end of imaging.

However, as described above, unlike the phase image and the intensity image, the hologram image is not an image in which individual cells can be clearly observed, and it is difficult to discriminate even if a small foreign substance or the like is mixed. Therefore, for example, even if the culture conditions of the cells are inappropriate and differentiation has progressed abnormally or cells that are different from the target are mixed, it is necessary to discard the sample. Therefore, it is not known until the phase image and the intensity image are confirmed, and wasteful time and manpower are consumed. For these reasons, there is a strong demand for an apparatus that can detect the above-described sample and imaging problems before the execution of a complicated and time-consuming process or even after the start of the process.

International Patent Publication No. 2016/084420 JP-A-10-268740

The present invention has been made to solve the above-described problems, and the object of the present invention is to perform phase processing on a sample containing cells by performing arithmetic processing such as phase recovery based on a hologram acquired by a holographic microscope. In a cell observation device that reconstructs an image or intensity image, it is possible to avoid wasting time and manpower by allowing the user to quickly and accurately grasp the occurrence of a sample failure or measurement failure. It is to provide a cell observation device.

The present invention made to solve the above problems is a cell observation device using a holographic microscope,
a) a light source,
b) a detection unit that acquires a hologram that is an interference fringe between the object wave and the reference wave when the sample including cells is irradiated with light emitted from the light source unit;
c) a moving unit that moves one or both of the light source unit and the detection unit and the sample so that the measurement position on the sample moves;
d) Measurement for controlling the light source unit, the detection unit, and the moving unit so as to repeat acquisition of the hologram at each measurement position in a predetermined observation target region while moving the measurement position on the sample by the moving unit. A control unit;
e) calculating phase information and / or intensity information based on hologram data obtained at each measurement position in a predetermined observation target region under the control of the measurement control unit, and corresponding to the observation target region A first reconstructed image creating unit for creating a phase image showing a two-dimensional distribution of phase information and / or an intensity image showing a two-dimensional distribution of intensity information;
f) Phase information based on hologram data obtained at one or more specific measurement positions or ranges designated by the user prior to or in parallel with the processing by the first reconstructed image creation unit And / or a second reconstructed image for calculating intensity information and creating a partial phase image showing a two-dimensional distribution of phase information corresponding to the measurement position or range and / or a partial intensity image showing a two-dimensional distribution of intensity information The creation department;
g) a display processing unit for displaying the partial phase image and / or the partial intensity image created by the second reconstructed image creating unit on the display unit;
It is characterized by having.

In the cell observation apparatus according to the present invention, typically, the sample is a cell culture container, and the maximum area where hologram data can be acquired by the holographic microscope is the whole culture container or a part of the culture container. It can be assumed that Examples of the culture container include a cell culture plate in which one or a plurality of wells are formed, a petri dish, and a culture flask for mass culture. Therefore, the cell observation apparatus according to the present invention is a suitable apparatus for non-invasive observation of living cells being cultured in such a culture container.

In the cell observation device according to the present invention, for example, the moving unit moves the light source unit and the detection unit integrally with respect to the culture plate whose position is fixed, so that the light emitted from the light source unit on the culture plate is The irradiated measurement position is changed. The measurement control unit irradiates light (generally coherent light) to one measurement position in the culture plate while moving the light source unit and the detection unit integrally in a stepwise manner by the moving unit, thereby detecting the detection unit. The operation of acquiring data (hologram data) indicating the two-dimensional distribution of light intensity by the hologram formed on the detection surface is repeated.

The first reconstructed image creating unit calculates phase information and / or intensity information based on hologram data obtained at different measurement positions, and sets the observation target region based on the obtained phase information and / or intensity information. A corresponding phase image and / or intensity image is created. However, since measurements are generally performed at a large number of measurement positions, the amount of hologram data is enormous, and it takes time to create a phase image and an intensity image. On the other hand, the second reconstructed image creation unit has one measurement or one specific measurement position designated by the user prior to or in parallel with the processing by the first reconstructed image creation unit. Phase information and / or intensity information is calculated based only on hologram data obtained within a position or within a predetermined range spanning a plurality of measurement positions, and a partial phase image and / or partial intensity image is calculated based on the obtained information. create. The range in which this partial phase image and / or partial intensity image is created is limited to a much smaller area than the entire observation target area. That is, the partial phase image or the partial intensity image is an image corresponding to a very small part of the entire observation target region. The display processing unit displays the partial phase image and / or the partial intensity image on the screen of the display unit.

The amount of hologram data that is the target of processing in the second reconstructed image creation unit is much smaller than that of hologram data that is the target of processing in the first reconstructed image creation unit. Therefore, the time required for the second reconstructed image creation unit to read the hologram data is much shorter than the time required for the first reconstructed image creation unit to read the hologram data, and the second reconstructed image creation The time required for the arithmetic processing and image reconstruction in the unit is much shorter than that in the first reconstructed image creating unit. Therefore, the partial phase image and the partial intensity image are displayed in a relatively short time from the time when the measurement is completed.

As described above, the partial phase image and the partial intensity image are images corresponding to a very small part of the entire observation target area, but are displayed when the user designates an appropriate or particularly important measurement position or range. From the partial phase image and the partial intensity image, it is possible to confirm the suitability of measurement (imaging), the presence or absence of cell abnormalities, the presence of foreign matter, and the like before the phase image of the entire observation target region is created. As a result, when the operator determines that the reconstruction process of the phase image or the like of the entire observation target region is unnecessary, the process is not performed or even if the process is started By canceling, it is possible to avoid wasting time and labor.

In the present invention, the holographic microscope system such as in-line type, off-axis type, and phase shift type is not limited. However, the optical system has a simple configuration and the distance between the sample and the light source unit is changed in a plurality of stages. An in-line configuration using holograms at a plurality of wavelengths is desirable in that the drive mechanism is simplified because it is not necessary.

In the present invention, preferably,
A hologram image creating unit that creates a hologram image based on the hologram data obtained at each measurement position in the predetermined observation target region and displays the hologram image on the screen of the display unit;
Instruction for the user to instruct the measurement position or range corresponding to the partial phase image and / or partial intensity image created by the second reconstructed image creation unit on the hologram image displayed by the hologram image creation unit An input section;
It is good to set it as the structure further provided.

Hologram images can be displayed immediately after the measurement, unlike phase images. Further, even when measurement is being performed, a partial hologram image corresponding to the measurement position can be displayed every time measurement for one measurement position is completed. Since this hologram image is a two-dimensional distribution of light intensity corresponding to the hologram, the living cells are often not visualized sufficiently, but the edges of wells with large steps are sufficiently visible. . Therefore, the operator instructs the instruction input unit, for example, which partial phase image or partial intensity image of which well is to be viewed on the displayed hologram image during measurement execution or immediately after the measurement is completed. Thereby, the operator can confirm the partial phase image and partial intensity image of a desired measurement position and range in a short waiting time.

In the present invention, only one of the partial phase image and the partial intensity image may be displayed. However, in the partial phase image, cells are easy to see but foreign matters such as dust tend to be hard to see. On the other hand, in the partial intensity image, foreign substances tend to be visible, but cells tend not to be visible. Therefore, it is more preferable to display both the partial phase image and the partial intensity image in the same range.

That is, in the present invention, preferably,
The second reconstructed image creation unit creates a partial phase image and a partial intensity image corresponding to a specific measurement position or range,
The display processing unit may be configured to display the partial phase image and the partial intensity image on the same screen.

In addition, if hologram data obtained by a holographic microscope is used, phase information and intensity information at different focal positions can be obtained by arithmetic processing. Therefore, in the present invention,
The second reconstructed image creating unit creates a partial phase image and / or a partial intensity image at a plurality of focal positions,
The display processing unit may include a focus instruction input unit for a user to indicate a focal position, and may display a partial phase image and / or a partial intensity image at the focal position instructed by the instruction input unit.

As an embodiment of the above configuration, the focus instruction input unit includes a slider displayed on the screen and an operation unit that moves a knob of the slider, and the focus position changes according to the position of the knob. It can be set as the thing to do.

According to this configuration, it is possible to easily compare partial phase images and partial intensity images with different focal positions, so that the operator can select a partial phase image with which the cell state can be observed most clearly, for example. It is possible to judge the quality of the product and the suitability of the measurement. Thereby, a more appropriate judgment can be made.

As another embodiment of the above configuration,
The sample is a culture plate in which a plurality of wells are formed,
When the phase image and / or the intensity image is displayed on the display unit from the display processing unit, the display processing unit further includes a focus instruction input unit for a user to specify a focus position for each well,
The first reconstructed image creating unit may be configured to create a phase image and / or an intensity image at the focal position for each well instructed by the focus instruction input unit.

When a plurality of wells are formed on the culture plate, the height of the bottom surface of the well and the height of the medium in the well are different from one well to another, and therefore the focusing position is often different from one well to another. On the other hand, according to the above configuration, the focal position of each well can be determined at the stage of creating the partial phase image and partial intensity image before creating the phase image and intensity image of the entire culture plate. As a result, only the phase image and intensity image at the focus position of each well need be created at the stage of creating the phase image and intensity image of the entire culture plate. Can be reduced.

In the present invention, the first reconstructed image creating unit and the second reconstructed image creating unit realize their functions with the same computer as the measurement control unit and the display processing unit or with a hardware circuit housed in the same housing. However, each function may be realized by a separate computer or a completely separate hardware circuit.

Specifically, for example, the functions of the first reconstructed image creating unit and the second reconstructed image creating unit, which require complicated calculations, are personal computers having the function of a measurement control unit that controls the holographic microscope main body responsible for measurement. It may be realized by a server (high performance computer) connected to the computer via a communication network. On the other hand, the function of the display processing unit may be provided in the same personal computer as the measurement control unit, and display may be performed based on data received from the server. In addition, since many of the processes in the first reconstructed image creation unit and the second reconstructed image creation unit are basically the same, they can be substantially the same constituent elements.

According to the cell observation apparatus according to the present invention, without waiting for a long time required to create a phase image and an intensity image of the entire observation range by image reconstruction processing based on hologram data acquired by measurement, work is performed. The person can confirm the suitability of the measurement and the defect of the sample quickly and accurately. This avoids wasting time by avoiding the execution of image reconstruction processing when such troubles are confirmed, or by stopping it immediately even during execution. can do. As a result, cell observation work can be performed efficiently.

BRIEF DESCRIPTION OF THE DRAWINGS The whole block diagram of the cell observation apparatus which is one Example of this invention. The schematic block diagram of the terminal for a measurement in the cell observation apparatus of a present Example. The conceptual diagram for demonstrating the image reconstruction process in the cell observation apparatus of a present Example. Explanatory drawing of the characteristic processing operation in the cell observation apparatus of a present Example. The figure which shows an example of the screen displayed with the terminal for a measurement after completion | finish of imaging | photography in the cell observation apparatus of a present Example. The figure which shows an example of the phase image and intensity | strength image of the designated measurement position which are displayed with the terminal for a measurement in the cell observation apparatus of a present Example. The figure which shows an example of the display screen in the case of displaying a slice image with the terminal for browsing in the cell observation apparatus of a present Example.

Hereinafter, an embodiment of a cell observation device according to the present invention will be described with reference to the accompanying drawings.
FIG. 1 is an overall configuration diagram of the cell observation apparatus of this embodiment, and FIG. 2 is a schematic configuration diagram of a measurement terminal in the cell observation apparatus of this embodiment.

The cell observation apparatus of this embodiment includes a measurement terminal 1, a browsing terminal 3, and a server 5 connected via a communication network 4 such as the Internet or an intranet. In FIG. 1, one measuring terminal 1 and one browsing terminal 3 are shown, but an appropriate number of each can be provided.

The server 5 is a high-performance computer. As functional blocks embodied by dedicated software installed in the computer, a data transmission / reception unit 51, a hologram data storage unit 52, a first phase recovery calculation unit 53, a second A phase recovery calculation unit 54, a first image reconstruction unit 55, a second image reconstruction unit 56, an image data storage unit 57, and the like are provided. In addition, although the 1st phase recovery calculation part 53 and the 2nd phase recovery calculation part 54 are provided separately, and the 1st image reconstruction part 55 and the 2nd image reconstruction part 56 are provided separately, as mentioned later, The calculations executed by the first phase recovery calculation unit 53 and the second phase recovery calculation unit 54 are substantially the same, and are executed by the first image reconstruction unit 55 and the second image reconstruction unit 56. The processing is substantially the same with some exceptions. Thus, substantially they can be the same component.

The measurement terminal 1 includes a microscope observation unit 10 and a control / processing unit 20. In the cell observation apparatus of this embodiment, the microscopic observation unit 10 is an inline digital holographic microscope, and includes a light source unit 11 including a laser diode and the image sensor 12 as shown in FIG. Between the image sensor 12, a culture plate 13 including cells 14 that are observation objects is arranged. The light source unit 11 and the image sensor 12 are integrally movable in two axial directions of the X axis and the Y axis that are orthogonal to each other by a moving unit 15 including a driving source such as a motor.

In FIG. 2, only one light source unit 11 and one image sensor 12, i.e., one set, are shown in order to avoid complication of the drawing. Is provided with four sets sandwiching one culture plate 13 and, as will be described later, holograms for different measurement positions of one culture plate 13 by the four sets of light source units 11 and the image sensor 12. It is possible to obtain in parallel.

The entity of the control / processing unit 20 controls the operation of the microscopic observation unit 10 and sends the data acquired by the microscopic observation unit 10 to the server 5. Further, the control / processing unit 20 receives the data processed by the server 5 and displays it. A personal computer (PC). As functional blocks embodied by dedicated software installed in this PC, the imaging control unit 21, hologram data storage unit 22, data transmission / reception unit 23, hologram image creation unit 24, confirmation position designation reception unit 25, image data storage A unit 26, a display processing unit 27, a focus designation receiving unit 28, and the like. The control / processing unit 20 is connected to an input unit 201 that is a pointing device such as a keyboard and a mouse, and a display unit 202.

As with the control / processing unit 20 in the measurement terminal 1, the browsing terminal 3 is a general PC. Then, the dedicated software installed in the PC can receive data from the server 5 and display an appropriate image formed based on the data.

Next, the operation when acquiring hologram data in the cell analyzer of this embodiment will be described with reference to FIG. FIG. 3 is a conceptual diagram for explaining image reconstruction processing in the cell observation apparatus of the present embodiment.

FIG. 3 (a) is a schematic top view of the culture plate 13 used in the cell observation apparatus of this example. The culture plate 13 is formed with six wells 13a having a circular shape when viewed from above, and cells are cultured in the wells 13a. Here, the entire culture plate 13, that is, the entire rectangular range including the six wells 13 a is the observation target region. The microscopic observation unit 10 includes four sets of a light source unit 11 and an image sensor 12, and each set of the light source unit 11 and the image sensor 12 has a total of 4 etc. as shown in FIG. It is responsible for collecting the hologram data of the four divided areas 81 divided. That is, the four sets of the light source unit 11 and the image sensor 12 share the collection of hologram data over the entire culture plate 13.

The range in which one set of the light source unit 11 and the image sensor 12 can be photographed at a time includes the one well 13a in the four-divided range 81 as shown in FIGS. 3B and 3C. This is a range corresponding to one imaging unit 83 obtained by dividing the square range 82 into 10 equal parts in the X-axis direction and 12 equal parts in the Y-axis direction. One quadrant 81 includes a predetermined number of, for example, 15 × 12 = 180 imaging units 83 designated in advance by the operator. The four light source units 11 and the four image sensors 12 are respectively arranged in the vicinity of four vertices of a rectangle having the same size as the four-divided range 81 in the XY plane including the light source unit 11 and the image sensor 12. The holograms for four different imaging units 83 on the culture plate 13 are simultaneously acquired.

When collecting hologram data for the culture plate 13, the operator first sets the culture plate 13 on which the cells 14 to be observed are cultured at a predetermined position of the microscopic observation unit 10, and identifies the culture plate 13. Information such as a number and measurement date / time is input from the input unit 201 to instruct measurement execution. Upon receiving this measurement instruction, the imaging control unit 21 controls each part of the microscopic observation unit 10 to perform imaging.

That is, one light source unit 11 irradiates a predetermined area (one imaging unit 83) of the culture plate 13 with coherent light having a small angle spread of about 10 °. The coherent light (object light 17) that has passed through the culture plate 13 and the cells 14 reaches the image sensor 12 while interfering with the light (reference light 16) that has passed through the region adjacent to the cells 14 on the culture plate 13. The object light 17 is light whose phase has changed when passing through the cell 14. On the other hand, the reference light 16 is light which does not pass through the cell 14 and thus does not undergo phase change caused by the cell 14. Therefore, on the detection surface (image surface) of the image sensor 12, an interference image, that is, a hologram, is formed between the object light 17 whose phase is changed by the cell 14 and the reference light 16 whose phase is not changed. 2D light intensity distribution data (hologram data) corresponding to is output from the image sensor 12.

As described above, coherent light is emitted from the four light source units 11 toward the culture plate 13 almost simultaneously, and the four image sensors 12 acquire hologram data of regions corresponding to different imaging units 83 on the culture plate 13. Is done. Each time measurement at one measurement position is completed, the light source unit 11 and the image sensor 12 are moved by the moving unit 15 in the X-axis direction and the Y-axis direction by a distance corresponding to one imaging unit 83 in the XY plane. It is sequentially moved in steps. As a result, measurement is performed with 180 imaging units 83 included in the four-divided range 81, and measurement of the entire culture plate 13 is performed with the four sets of light source units 11 and the entire image sensor 12. The hologram data obtained by the four image sensors 12 of the microscopic observation unit 10 in this way is stored in the hologram data storage unit 22 together with attribute information such as measurement date and time.

After the measurement (shooting) for all the imaging units 83 is completed, when a predetermined operation described later is performed by the operator, the data transmission / reception unit 23 uses the hologram data stored in the hologram data storage unit 22 as the measurement date and time, etc. The attribute information is sequentially transferred to the server 5 via the communication network 4. It should be noted that raw, that is, unprocessed hologram data may be sent from each measurement terminal 1 to the server 5, but processing that corrects an error factor specific to each measurement terminal 1 as necessary. The processed hologram data may be sent to the server 5.

In the server 5, the data transmitter / receiver 51 receives the hologram data sent from the measurement terminal 1, and identifies identification information for specifying the measurement terminal 1, identification information of the culture plate input at the time of imaging, measurement date and time, and the like. Hologram data is stored in the hologram data storage unit 52 together with the attribute information. After collecting the hologram data, the first phase recovery calculation unit 53 reads out the hologram data for each imaging unit from the hologram data storage unit 52 and restores the phase information by performing propagation calculation processing of the light wave, and intensity (amplitude) information. Ask for. Since the spatial distribution of the phase information and the intensity information is obtained for each imaging unit 83, when the phase information and the intensity information of all the imaging units 83 are obtained, the first image reconstruction unit 55 performs the phase information and the intensity information. Based on the above, a phase image and an intensity image of the entire observation target region are formed.

More specifically, the first image reconstruction unit 55 reconstructs the phase image of each imaging unit 83 based on the spatial distribution of the phase information calculated for each imaging unit 83, and connects the phase images in the narrow range. By performing a tiling process (see FIG. 3D), a phase image of the observation target region, that is, the entire culture plate 13 is formed. Of course, in the tiling process, an appropriate correction process may be performed so that the phase images at the boundaries of the imaging units 83 are smoothly connected. In calculating the phase information and reconstructing the image, an algorithm disclosed in known documents such as

Patent Documents

1 and 2 may be used.

In addition, the reconstructed image obtained by normal processing is the highest resolution image obtained in principle from the acquired hologram data. Not only that, the resolution can be improved by binning processing based on the highest resolution phase image. You may make it produce the phase image of the resolution | decomposability (magnification) of the multiple steps | steps which dropped. Then, the image data constituting the phase image and the intensity image created in this way are stored in the image data storage unit 57.

Although depending on the performance of the server 5, the amount of hologram data obtained from one culture plate 13 is enormous, so that the processing in the first phase recovery calculation unit 53 and the first image reconstruction unit 55 takes a considerable amount of time. Cost. Therefore, even if all the measurements are completed on the measurement terminal 1 side, the phase image and the intensity image of the entire observation target region cannot be browsed immediately. Therefore, in the cell observation apparatus of the present embodiment, the characteristic processing described below is executed so that the operator can quickly grasp the measurement failure or the contamination of the foreign matter after the measurement is completed.

FIG. 4 is an explanatory diagram of this characteristic processing operation, FIG. 5 is a diagram showing an example of a screen displayed on the measurement terminal after the end of photographing, and FIG. 6 is a phase image of the designated measurement position displayed on the measurement terminal and It is a figure which shows an example of an intensity | strength image.

When the operator performs a predetermined operation with the input unit 201 before the measurement is executed, the display processing unit 27 displays a captured image display screen 60 as shown in FIG. On the photographed image display screen 60, an image display area 61 and a plate information display area 62 are arranged, and a “stop” button 63 is arranged at the lower right. In the plate information display area 62, attribute information such as the name (plate name) and identification number (plate ID) of the culture plate 13 during or after measurement is displayed. However, the screen shown in FIG. 5 is the one after the measurement is completed, and no substantial image is displayed in the image display area 61 before the measurement is started.

When measurement is started and hologram data is obtained for the region corresponding to each imaging unit 83 on the culture plate 13 as described above, the hologram image creation unit 24 is based on the obtained data for each imaging unit 83. A hologram image showing a two-dimensional distribution of light intensity is created. The hologram image created at this time is a thumbnail image with the lowest resolution. The display processing unit 27 pastes and displays the created hologram thumbnail image at a position corresponding to each imaging unit in the image display area 61. That is, every time hologram data of one image pickup unit is newly obtained, a thumbnail image of the hologram based on the data is added to the image in the image display area 61 and displayed. Then, when the measurement for all the imaging units is completed, a hologram image of the entire observation target region as shown in FIG. 5 is displayed. When the operator clicks the “execute image creation” button 65 in the captured image display screen 60 after the measurement is completed, the collected hologram data is transferred to the server 5 and the image reconstruction process for the full screen is performed as described above. Executed.

In addition, although it is difficult to observe the state of cells in a hologram image, many of the phenomena caused by malfunctions of the apparatus such as damage to the light source unit 11 and the image sensor 12 and measurement troubles can be grasped on the image. It is. Therefore, the operator monitors the hologram image displayed in the image display area 61 during the measurement, and clicks the “stop” button 63 when it can be determined that there is some problem. Then, the imaging control unit 21 receives this operation and stops measurement. As described above, when there is some problem in the measurement, it is possible to avoid spending time for the remaining useless measurement by quickly stopping the measurement.

When it is desired to confirm the phase image and the intensity image immediately after the measurement is completed, the operator uses an pointing device such as a mouse that is a part of the input unit 201 to display the image display area 61 in the captured image display screen 60 shown in FIG. The cursor is moved onto a desired one of a number of thumbnail images constituting the hologram image displayed on the screen. The confirmation position designation accepting unit 25 accepts this operation, and displays a rectangular mark 66 indicating the selected imaging unit superimposed on the image. When the operator performs a double-click operation with the pointing device, the confirmation position designation receiving unit 25 recognizes one imaging unit selected at that time as a partial phase image creation target, and accordingly Then, the display processing unit 27 displays the designated position image display screen 70 as shown in FIG. 6 on the screen of the display unit 202. An image display area 71 is provided on the designated position image display screen 70.

In parallel, the confirmation position designation receiving unit 25 reads out the hologram data obtained for one designated imaging unit from the hologram data storage unit 22, and the data transmission / reception unit 23 sends the read data through the communication network 4 to the server 5. Forward to. Note that the hologram data transfer operation for one image pickup unit may be performed in parallel with the transfer operation of the hologram data of the entire imaging target region to the server 5 as described above, or preferentially. You may go. In the latter case, if the hologram data transfer operation for the entire imaging target area has already been started, the hologram data may be transferred for one imaging unit instructed to be temporarily interrupted.

In the server 5, the data transmission / reception unit 51 receives the hologram data for one imaging unit sent from the measurement terminal 1 as described above, and temporarily stores the data in the hologram data storage unit 52. Since the amount of data transferred at this time is much smaller than the amount of hologram data in the entire observation target area, the transfer time is also significantly shorter (see FIG. 4).

The second phase recovery calculation unit 54 reads out hologram data of one imaging unit stored in the hologram data storage unit 22 and performs light wave propagation calculation processing to restore phase information and obtain intensity information. Subsequently, the second image reconstruction unit 56 forms a partial phase image and a partial intensity image for one imaging unit based on the calculated phase information and intensity information. In addition, when calculating phase information and intensity information from hologram data, information on an arbitrary focal position can be calculated. The focal position set at this time is a default value designated from the measurement terminal 1. (For example, the focus position set at the most recent past time point for the culture plate and well having the same identification number). The image data constituting the partial phase image and the partial intensity image is stored in the image data storage unit 57 and sent from the data transmission / reception unit 51 to the measurement terminal 1. Since computation processing and image reconstruction processing need only be performed for one imaging unit, tiling processing etc. are unnecessary, so the processing time until the partial phase image and partial intensity image are formed is also the phase of the entire observation target region. This is much shorter than the time required to form the image and the intensity image (see FIG. 4).

As described above, the time required for transferring hologram data for one imaging unit selected and instructed by the operator and forming a partial phase image and a partial intensity image based on the data is short. Therefore, as described above in the measurement terminal 1, a part of the measurement terminal 1 is transferred from the server 5 to the measurement terminal 1 within a relatively short time from the time when the operator double-clicks the thumbnail image corresponding to one imaging unit. Image data constituting the phase image and the partial intensity image is sent. The sent image data is temporarily stored in the image data storage unit 26.

The display processing unit 27 creates a partial phase image and a partial intensity image based on the image data, and displays the two images side by side in the image display area 71 in the designated position image display screen 70. Note that the type of image displayed in the image display area 71 can be selected by putting a check mark in the display image selection check box 72, and only one of the partial phase image and the partial intensity image is displayed in the image display area 71. Can also be displayed. In the example of FIG. 6, the focal position of the displayed partial phase image and partial intensity image is 5450 μm.

The focus position set initially is not necessarily the focus position in cell observation, and it is necessary to change the focus position when observing a foreign object having a height different from that of the cell. Therefore, when observing partial phase images and partial intensity images at different focal positions, the operator sets conditions for determining a plurality of stages of focal positions in the slice condition setting area 73 in the designated position image display screen 70. That is, when the range of the focus position and the slice width (the step width of the focus position) are respectively input as numerical values, the number of slices (the number of steps of the focus position) is automatically calculated and displayed. These numerical values can also be left as default. When the operator clicks the “create slice image” button 74 after setting the slice conditions, the focus designation receiving unit 28 receives this operation and instructs the server 5 on the set slice conditions.

In the server 5, the second phase recovery calculation unit 54 and the second image reconstruction unit 56 perform phase information at a plurality of focal positions according to the designated slice condition based on the hologram data for the selected one imaging unit. And intensity information are calculated, and a plurality of partial phase images and partial intensity images having different focal positions are formed. The image data constituting the partial phase image and the partial intensity image at the plurality of focal positions are stored in the image data storage unit 57 and sent from the data transmission / reception unit 51 to the measurement terminal 1. The sent image data is temporarily stored in the image data storage unit 26. As described above, when the image data constituting the partial phase image and the partial intensity image at different focal positions are prepared, the operator in the focal position selection operation area 75 in the designated position image display screen 70 becomes effective. That is, the slider knob in the focal position selection operation area 75 can be moved by the pointing device, and the numerical value of the focal position can be directly input into the text box.

When the operator moves the knob of the slider in the focal position selection operation area 75 to the left or right using the pointing device, the focus designation receiving unit 28 obtains the focal position corresponding to the position of the knob on the slider, and the display processing unit 27 displays the image. The partial phase image and the partial intensity image displayed in the display area 71 are updated to the image at the focal position. In the example of FIG. 6, since the number of slices is 11, the partial phase image and the partial intensity image at 11 different focal positions can be confirmed by operating the slider. The operator can find a focal position at which the cell to be observed can be most clearly observed by comparing partial phase images or partial intensity images at different focal positions.

If partial phase images and partial intensity images at appropriate focal positions are obtained in this way, the operator confirms these images and the cells are abnormal, or a lot of fine foreign matters are mixed, etc. It is determined whether or not there is a problem with the camera, and further whether or not there is a problem with shooting. If it is determined that there is no need to create a phase image or intensity image of the entire observation target region, execution of a reconstruction process for the full-screen phase image and intensity image based on the hologram data is avoided. Alternatively, even when the transfer of all hologram data from the measurement terminal 1 to the server 5 is started by the operation of the “execute image creation” button 65 in FIG. 5, the data is obtained by performing a predetermined operation. The transfer can be stopped (see FIG. 4). Since such a determination can be made within a relatively short time from the end of photographing, it is not necessary to wait for a long time until a phase image is created by image reconstruction processing of the full screen.

Also, as described above, after the appropriate focus position is determined by comparing the partial phase image and partial intensity image at different focus positions, the operator clicks the “Set this value as the focus position of well” button 76. When operated, the focus designation accepting unit 28 accepts this operation and informs the server 5 of the information on the designated focus position together with the identification number of the well in which the imaging unit such as the partial phase image currently displayed exists. In the server 5, the instructed focal position is stored in the hologram data storage unit 52 in association with the well identification number, and the first phase recovery calculation unit 53 performs phase information and intensity of the imaging unit corresponding to the well with the identification number. When calculating the information, phase information and intensity information at the focal position associated with the identification number are calculated. That is, it is possible to determine the focal position for each well when creating the phase image and the intensity image of the entire imaging target region while checking the partial phase image and the partial intensity image on the measurement terminal 1.

Of course, the focal position that can be determined based on the partial phase image and the partial intensity image for one imaging unit is only for one well in which the imaging unit exists. Therefore, when it is desired to individually set the focal positions for all of the six wells, one imaging unit is selected for each well on the hologram image displayed in the image display area 61 of the captured image display screen 60. It is necessary to repeat the procedure of displaying the partial phase image and the partial intensity image for the imaging unit on the designated position image display screen 70, and further setting the slice conditions to create the slice image. Of course, the focal position set for one well can be used for other wells.

Thus, in the cell observation apparatus of the present embodiment, before creating and displaying the phase image and the intensity image of the entire observation target region, an appropriate focal position is set for each well, and the phase image of only the focal position is set. And intensity images can be created and displayed. Thereby, it is possible to save the time required to create an image at an unnecessary focal position.

In the above description, the case where a partial phase image or the like is confirmed on the measurement terminal 1 immediately after measurement is performed on the measurement terminal 1 is described. However, an appropriate hologram can be used on the browsing terminal 3 that does not include the microscopic observation unit 10. The display of the image and the confirmation of the partial phase image for one imaging unit selected on the hologram image can be performed. FIG. 7 is a diagram illustrating an example of a display screen when a hologram image and a partial phase image are displayed on the browsing terminal 3.

As shown in FIG. 7, the slice image confirmation display screen 90 is provided with a hologram image display area 91 and a partial phase image display area 92. In the hologram image display area 91, a hologram image based on the hologram data stored in the hologram data storage unit 52 of the server 5 after shooting is displayed. The hologram image to be displayed at this time can be appropriately selected by designating, for example, the identification number of the culture plate or the measurement date / time. When the operator wants to see a plurality of partial phase images having different focal positions, the operator designates one imaging unit on the hologram image, and sets conditions for determining a plurality of stages of focal positions in the slice condition setting area 93. Then, a “slice image preview” button 94 is clicked.

This instruction and setting contents are sent from the browsing terminal 3 to the server 5, and the second phase recovery calculation unit 54 and the second image reconstruction unit 56 of the server 5 store the hologram data of the designated imaging unit as described above. The phase information at a plurality of focal positions designated based on is calculated, and a partial phase image is reconstructed for each focal position. And the image data which comprises the some partial phase image from which a focus position differs are transmitted to the terminal 3 for browsing. The browsing terminal 3 displays the partial phase image in the partial phase image display area 92 based on the received image data. Further, the displayed partial phase image is changed to an image at a different focal position in accordance with the operation of the slider in the focal position selection operation area 95. Thereby, the operator can also confirm the partial phase image of the culture plate image | photographed in the past.

In the above embodiment, the partial phase image in the imaging unit is reconstructed from the hologram data of one imaging unit designated by the operator. The partial phase image in the range may be reconstructed from the hologram data included in the specified range. Thereby, since the amount of data transferred to the server 5 is further reduced, the data transfer time and processing time can be further shortened.

Further, after displaying the partial phase image and the intensity phase image for one imaging unit, the range designated by the operator in the image may be enlarged and displayed. In addition, when the operator appropriately specifies a part of one imaging unit on the hologram image, the specified range is obtained after reconstructing the partial phase image and partial intensity image for that imaging unit. Only these images may be cut out and displayed.

In addition, it is possible to specify not only one but also a plurality of imaging units on the hologram image, or to arbitrarily specify a range having a size larger than one imaging unit, and the specified plurality of imaging units or ranges. The partial phase image and the partial intensity image corresponding to may be reconstructed and displayed. However, since the amount of data to be processed increases as the number of specified imaging units increases or the specified range increases, the time is increased until a partial phase image or the like can be displayed. For this reason, it is preferable to predetermine the upper limit of the number of imaging units that can be specified and the width of the range based on the time constraint.

In the above embodiment, the microscopic observation unit 10 is an in-line type digital holographic microscope. However, the microscopic observation unit 10 is not limited to the inline type as long as it acquires a hologram for each measurement position in the observation target region. Alternatively, an off-axis type or phase shift type digital holographic microscope may be used.

Moreover, in the said Example, although arithmetic processing, such as phase recovery, was implemented in the server 5 connected with the measuring terminal 1 via the communication network 4, as a structure which implements all the processes with a stand-alone type apparatus, Of course it is good.

Furthermore, each of the above-described embodiments and the above-described modified examples is an example of the present invention, and further appropriate changes, modifications, and additions within the scope of the present invention are included in the scope of the claims of the present application. It is natural.

DESCRIPTION OF SYMBOLS 1 ... Measuring terminal 10 ... Microscopic observation part 11 ... Light source part 12 ... Image sensor 13 ... Culture plate 13a ... Well 14 ... Cell 15 ... Moving part 16 ... Reference light 17 ... Object light 20 ... Control and processing part 21 ... Imaging control Unit 22 ... Hologram data storage unit 23 ... Data transmission / reception unit 24 ... Hologram image creation unit 25 ... Confirmation position designation reception unit 26 ... Image data storage unit 27 ... Display processing unit 28 ... Focus designation reception unit 201 ... Input unit 202 ... Display unit 3 ... Browsing terminal 5 ... Server 4 ... Communication network 51 ... Data transmission / reception unit 52 ... Hologram data storage unit 53 ... First phase recovery calculation unit 54 ... Second phase recovery calculation unit 55 ... First image reconstruction unit 56 ... First Two-image reconstruction unit 57 ... Image data storage unit 60 ... Captured image display screen 61 ... Image display area 62 ... Plate information display area 63 ... "Stop" button 65 ... "Create image "Execute" button 66 ... Mark 70 ... Designated position image display screen 71 ... Image display area 72 ... Display image selection check box 73 ... Slice condition setting area 74 ... "Create slice image" button 75 ... Focus position selection operation area 76 ... "This "Set value to well focal position" button 90 ... Slice image confirmation display screen 91 ... Hologram image display area 92 ... Partial phase image display area 93 ... Slice condition setting area 94 ... "Slice image preview" button 95 ... Focus position selection operation region

Claims

A cell observation device using a holographic microscope,
a) a light source,
b) a detection unit that acquires a hologram that is an interference fringe between the object wave and the reference wave when the sample including cells is irradiated with light emitted from the light source unit;
c) a moving unit that moves one or both of the light source unit and the detection unit and the sample so that the measurement position on the sample moves;
d) Measurement for controlling the light source unit, the detection unit, and the moving unit so as to repeat acquisition of the hologram at each measurement position in a predetermined observation target region while moving the measurement position on the sample by the moving unit. A control unit;
e) calculating phase information and / or intensity information based on hologram data obtained at each measurement position in a predetermined observation target region under the control of the measurement control unit, and corresponding to the observation target region A first reconstructed image creating unit for creating a phase image showing a two-dimensional distribution of phase information and / or an intensity image showing a two-dimensional distribution of intensity information;
f) Phase information based on hologram data obtained at one or more specific measurement positions or ranges designated by the user prior to or in parallel with the processing by the first reconstructed image creation unit And / or a second reconstructed image for calculating intensity information and creating a partial phase image showing a two-dimensional distribution of phase information corresponding to the measurement position or range and / or a partial intensity image showing a two-dimensional distribution of intensity information The creation department;
g) a display processing unit for displaying the partial phase image and / or the partial intensity image created by the second reconstructed image creating unit on the display unit;
A cell observation apparatus comprising:
The cell observation apparatus according to claim 1,
A hologram image creating unit that creates a hologram image based on the hologram data obtained at each measurement position in the predetermined observation target region and displays the hologram image on the screen of the display unit;
Instruction for the user to instruct the measurement position or range corresponding to the partial phase image and / or partial intensity image created by the second reconstructed image creation unit on the hologram image displayed by the hologram image creation unit An input section;
A cell observation device further comprising:
The cell observation apparatus according to claim 1,
The second reconstructed image creation unit creates a partial phase image and a partial intensity image corresponding to a specific measurement position or range,
The cell observation apparatus, wherein the display processing unit displays the partial phase image and the partial intensity image on the same screen.
The cell observation apparatus according to claim 1,
The second reconstructed image creating unit creates a partial phase image and / or a partial intensity image at a plurality of focal positions,
The display processing unit includes a focus instruction input unit for a user to indicate a focal position, and displays a partial phase image and / or a partial intensity image at the focal position instructed by the instruction input unit. apparatus.
The cell observation device according to claim 4,
The focus instruction input unit includes a slider displayed on a screen and an operation unit that moves a knob of the slider, and the focus position changes according to the position of the knob. .
The cell observation device according to claim 4,
The sample is a culture plate in which a plurality of wells are formed,
When the phase image and / or the intensity image is displayed on the display unit from the display processing unit, the display processing unit further includes a focus instruction input unit for a user to specify a focus position for each well,
The cell observation device, wherein the first reconstructed image creating unit creates a phase image and / or an intensity image at a focal position for each well instructed by the focus instruction input unit.