JPWO2019180833A1 - Cell observation device - Google Patents

Abstract

The phase information calculation unit (23) and the image reconstruction unit (24) perform arithmetic processing and the like based on the hologram data obtained by the microscopic observation unit (10), which is a holographic microscope, so that transparent cells can be easily observed. Create an IHM phase image. The region identification processing unit (26) uses a discriminator constructed by learning teacher data in which the undifferentiated cell region and the undifferentiated deviant cell region are characteristic regions, and in the created IHM phase image, the undifferentiated cell region and Distinguish from undifferentiated deviant cell regions. Based on this area identification result, the area identification image creation unit (28) creates an area identification image in which each area is divided by a predetermined color, and the display processing unit (29) creates an IHM corresponding to the same observation range. The phase image and the area identification image are displayed side by side on the display unit (40). As a result, even an operator with little experience or skill can efficiently confirm the state of cells during culturing.

Description

The present invention relates to a cell observation device for observing the state of cells, and more specifically, in the process of culturing pluripotent stem cells (ES cells and iPS cells), the state of cells is non-invasively determined and there is a problem in culturing. The present invention relates to a cell observation device suitable for confirming the presence or absence.

In the field of regenerative medicine, research using pluripotent stem cells such as iPS cells and ES cells has been actively conducted in recent years. In research and development of regenerative medicine using such pluripotent stem cells, it is necessary to culture a large amount of undifferentiated cells in a state of maintaining pluripotency. Therefore, it is necessary to select an appropriate culture environment and stably control the environment, and it is necessary to confirm the state of cells during culture at a high frequency. For example, when a cell in a cell colony deviates from the undifferentiated state, in this case, all the cells in the cell colony have the ability to differentiate, so that finally all the cells in the colony are undifferentiated. It transitions to the deviation state. Therefore, the worker daily checks whether or not cells that have deviated from the undifferentiated state (cells that have already differentiated or cells that are likely to differentiate, hereinafter referred to as "undifferentiated deviated cells") are generated in the cultured cells. If undifferentiated deviant cells are found, they need to be removed promptly. In addition, foreign substances such as dust may be mixed into the medium in the process of culturing, and it is necessary to remove such foreign substances as quickly as possible.

Whether or not the pluripotent stem cells maintain the undifferentiated state can be reliably determined by staining with an undifferentiated marker. However, since the stained cells die, undifferentiated marker staining cannot be performed to determine pluripotent stem cells for regenerative medicine. Therefore, in the current field of cell culture for regenerative medicine, it is determined whether or not the worker is an undifferentiated cell based on the morphological observation of the cell using a phase-contrast microscope. A phase-contrast microscope is used because cells are generally transparent and difficult to observe with a normal optical microscope.

Further, as disclosed in Non-Patent Document 1, recently, an apparatus for acquiring an observation image of a cell by using a holographic technique has also been put into practical use. As disclosed in Patent Documents 1 to 4, this device clearly observes cells by performing data processing such as phase retrieval and image reconstruction on hologram data obtained by a digital holographic microscope. An easy-to-use phase image (hereinafter referred to as "IHM phase image" because an in-line holographic microscopy (IHM) is used) is created. With a digital holographic microscope, phase information at an arbitrary distance can be calculated at the stage of arithmetic processing after acquiring hologram data, so it is not necessary to perform focusing each time during shooting, and the measurement time can be shortened. There is an advantage.

However, although cells can be observed clearly to some extent in a phase-contrast microscopic image or an IHM phase image, a considerable skill is required for an operator to visually accurately determine undifferentiated cells and the like. Therefore, the number of workers who can take charge of such a judgment is quite limited. In addition, it is inevitable that the judgment will vary because it is based on human judgment. In addition, it is difficult for even a skilled worker to process a large number of images because it takes time to make an accurate determination. Therefore, although these conventional methods are acceptable at the research level at universities and research institutes, they are not suitable for industrial mass production of pluripotent stem cells.

International Patent Publication No. 2017/203718 Pamphlet International Patent Publication No. 2017/204013 Pamphlet International Patent Publication No. 2016/08424 Pamphlet Japanese Unexamined Patent Publication No. 10-268740

"Cell culture analyzer CS-1", [online], Shimadzu Corporation, [Search on February 14, 2018], Internet <URL: https://www.an.shimadzu.co.jp/bio /cell/cs1/index.htm ＞

The present invention has been made to solve the above problems, and an object of the present invention is to determine the state and quality of cells based on observation images obtained for cells, and to improperly mix foreign substances. It is an object of the present invention to provide a cell observation device capable of performing work such as confirming whether or not there is a state of illness even by an operator having little experience or skill, and capable of efficiently performing such work.

The cell observation device according to the present invention made to solve the above problems is
a) An observation image acquisition unit that acquires an observation image of cells,
b) A region identification unit that identifies a characteristic region indicating two or more cell states for all or a part of the observation image acquired by the observation image acquisition unit.
c) An image in which two or more feature areas identified by the area identification unit are divided into visually identifiable modes to create an area identification image, and the image is displayed on the screen of the display unit. Display processing unit and
It is characterized by having.

In the present invention, the observed image is typically a phase-contrast microscopic image obtained by a phase-contrast microscope, or a phase-contrast image, an intensity image, or a pseudo-phase image (position by a phase-contrast microscope) obtained from hologram data obtained by a holographic microscope. An image corresponding to a phase-contrast microscopic image) and the like.

When such an observation image is obtained by the observation image acquisition unit, the region identification unit has two or more features showing different cell states for the entire observation image or, for example, a partial observation image in a range specified by the user. Identify the area. The algorithm for region identification in the region identification unit is not particularly limited, but one that utilizes supervised machine learning as a part thereof is useful. That is, as one aspect of the present invention, the region identification unit uses a trained model created by machine learning using an image in which two or more feature regions in an observation image are divided in a mode that can be identified as teacher data. It can be used to identify the feature region for all or part of the observation image.

Supervised machine learning includes Support Vector Machine (SVM), Random Forest, AdaBoost, Simple Bayes, k-neighborhood method, and Deep Learning including neural networks. Etc., an appropriate method can be used. Further, instead of learning the image itself, the feature amount extracted from the image by texture analysis or the like may be learned. Further, depending on the type of cell and the mode of the characteristic region, it may be possible to identify by a simpler method without using machine learning, for example, a method of determining a pixel value according to a threshold value.

The above characteristic regions differ depending on the type of cells to be observed, the purpose of observation, etc., but for example, when the cells to be observed are pluripotent stem cells including human iPS cells, the cell states different from each other are undifferentiated cells and undifferentiated cells. Deviation cells and the like. Therefore, in the present invention, for example, in the region where one of the characteristic regions is a cell or a cell colony and the other one of the characteristic regions is a cell or a cell colony, either undifferentiated cells or undifferentiated deviant cells It can be an area that is. Further, as different cell states, cells differentiated into cells of a target organ (for example, myocardium for heart transplantation) and other cells (for example, nerve cells) may be used.

The image display processing unit creates a region identification image in which the two or more feature regions identified as described above are divided and drawn in a visually identifiable manner. Then, the area identification image is displayed on the screen of the display unit. Here, the "visually identifiable aspect" is typically a filling process using different display colors, that is, color coding of the feature area. Of course, the feature area may be divided by different shades on the gray scale, or the feature area may be divided by different graphic patterns of the filled portion.

In the region identification image displayed in this way, for example, the undifferentiated cell region, the undifferentiated deviant cell region, the foreign body region, and the like are shown in different colors and the like. As a result, even an inexperienced worker who cannot distinguish the state of cells from the observation image can determine whether or not undifferentiated deviant cells exist in the cell colony, and if so, undifferentiated cells and undifferentiated cells. It is possible to grasp at a glance where the boundary with the differentiated cells is, or whether or not foreign substances are mixed in the medium.

In the present invention, preferably, the area identification unit generates information for identifying the feature area on a pixel-by-pixel basis, and the image display processing unit is a pixel unit or a unit of a small area in which a plurality of adjacent pixels are grouped together. It is preferable that the different feature areas are divided and drawn in a visually distinguishable manner.

For example, in an observation image with a small magnification, one cell may be as large as one pixel of the image, but one cell is determined to be an undifferentiated deviation cell and one pixel corresponding to the cell. Is often difficult to understand even if it is displayed in a color different from the surrounding pixels on the area identification image. In such a case, if it is not a pixel unit but a unit of a small area in which a plurality of adjacent pixels are grouped together, and any one of the plurality of pixels included in one small area is identified as a certain feature area. By showing the small area with the color associated with the feature area, it becomes easy to visually understand that the feature area exists. As a result, it is possible to prevent the operator from overlooking a very small feature area when checking the image.

Further, in the present invention, the image display processing unit may be configured to display information indicating the cell state to be identified by the region identification unit and the distinguishable mode for each characteristic region together with the region identification image.

Specifically, when each feature region in the region identification image is indicated by a different display color, information such as "red: undifferentiated deviation, green: undifferentiated" is included in a part of the image or in the vicinity of the image. It is good to display in. As a result, the worker can grasp the cell state in the image at a glance.

Further, in the present invention, the image display processing unit may be configured to enlarge or reduce the area identification image displayed on the screen of the display unit according to the operation of the user. Further, the image display processing unit may be configured to display an enlarged image of the range when a predetermined range is selected on the area identification image.
As a result, the operator can accurately grasp the cell state in a very small range of, for example, one cell.

Further, in the above configuration, the image display processing unit may display the enlarged image and the area identification image before the enlargement on the same screen.
As a result, the operator can grasp the distribution state of the cell state over a wide range on the sample and a part of the fine cell state in the sample.

Further, in the above configuration, the image display processing unit may display information indicating the cell state to be identified by the region identification unit and the distinguishable mode for each characteristic region together with the enlarged image.
As a result, the operator can accurately grasp the cell state in the enlarged image just by looking at the enlarged image.

Further, preferably in the present invention, the observation image acquisition unit includes a holographic microscope and an image reconstruction processing unit that creates a phase image and / or an intensity image based on the hologram data obtained by the holographic microscope. Including
The image display processing unit may be configured to display the area identification image and the phase image and / or intensity image on the same screen.

The holographic microscope may be an in-line type, an off-axis type, a phase shift type, or the like regardless of the method.

According to this configuration, the information at an arbitrary focal position can be obtained by calculation at the stage of calculating the phase information and the intensity information based on the hologram data, which is troublesome at the stage of photographing the sample containing cells with a holographic microscope. No need to focus. Therefore, the sample can be photographed smoothly in a short time, which is suitable for observing living cells. In addition, it is possible to compare a phase image in which cells can be observed relatively clearly and an intensity image in which foreign substances can be observed relatively clearly and a region identification image on the same screen as compared with a general optical microscopic image. Therefore, it is possible to check whether or not the feature area is properly identified by the automatic identification process while comparing the images.

Further, in the above configuration, when a predetermined range is selected on the area identification image, the image display processing unit corresponds to the enlarged image of the range and the enlarged image in the phase image and / or the intensity image. It is advisable to display the image of the specified range on the same screen.

Furthermore, in the above configuration, when the enlargement / reduction operation is performed on any of the enlarged image and the enlarged image of the phase image and / or the intensity image, the image display processing unit responds to the operation. It is advisable to link all the images to enlarge or reduce them.
As a result, the region identification image in the same range can always be compared with the phase image or the intensity image, so that it is possible to accurately confirm whether or not the region identification is appropriate. In addition, since it is not necessary for the operator to individually enlarge / reduce each image, the observation work can be performed efficiently.

Further, in the above configuration, the image display processing unit may be configured to display a composite image in which the region identification image and the phase image or the intensity image are superimposed.

In that case, the image display processing unit may be configured to adjust the transparency of at least one of the superimposed images according to the operation by the user.

According to such a configuration, the operator can easily and visually grasp the relationship between the pattern or boundary on the phase image or the intensity image and the feature area. Of course, it is convenient to allow the user to appropriately switch between a display in which a plurality of images are arranged side by side and a display in which the images are superimposed.

According to the present invention, in the field of culturing pluripotent stem cells such as iPS cells and ES cells, an operator can determine whether the cells being cultured maintain an undifferentiated state or are in an undifferentiated state. It is possible to reduce the labor of determining whether or not undifferentiated deviating cells are present in the cell colony, or whether or not foreign substances are mixed in the medium. Further, even when the skill and skill of the worker in charge of such work is insufficient, it is possible to reduce the variation in judgment, reduce judgment mistakes and oversights, and improve the judgment accuracy. In addition, a large amount of observation images can be efficiently processed, and work efficiency can be improved. As a result, quality control of cells during culture becomes easy, and productivity in cell culture can be improved.

The schematic block diagram of the cell observation apparatus which is an Example of this invention. The flowchart which shows the flow of the area identification processing of the observation image in the cell observation apparatus of this Example. The schematic diagram which shows an example of the area identification processing in the cell observation apparatus of this Example. The figure which shows one aspect of the display of the area identification image in the cell observation apparatus of this Example. The figure which shows the other aspect of the display of the area identification image in the cell observation apparatus of this Example.

Hereinafter, an embodiment of the cell observation apparatus according to the present invention will be described with reference to the accompanying drawings.
FIG. 1 is a schematic configuration diagram of the cell observation device of this embodiment.

The cell observation device of this embodiment includes a microscopic observation unit 10, a control / processing unit 20, an input unit 30 and a display unit 40, which are user interfaces.
The microscopic observation unit 10 is an in-line holographic microscope, includes a light source unit 11 including a laser diode and an image sensor 12, and a cell colony (or a single cell) 14 between the light source unit 11 and the image sensor 12. The containing culture plate 13 is arranged.

The control / processing unit 20 controls the operation of the microscopic observation unit 10 and processes the data acquired by the microscopic observation unit 10, and includes the imaging control unit 21, the hologram data storage unit 22, and the phase information calculation. Unit 23, image reconstruction unit 24, reconstruction image data storage unit 25, area identification processing unit 26, area identification result data storage unit 27, area identification image creation unit 28, display processing unit 29, and so on. Is provided as a functional block. Details of the area identification processing unit 26 will be described later.

Usually, the entity of the control / processing unit 20 is a personal computer in which predetermined software is installed, a higher-performance workstation, or a computer system including a high-performance computer connected to such a computer via a communication line. is there. That is, the function of each block included in the control / processing unit 20 is stored in the computer or computer system, which is executed by executing software installed in a computer system including a single computer or a plurality of computers. It can be embodied by processing using various types of data.

The cell observation device of this example can observe various cells, but here, an observation image of the cells is acquired for the purpose of observing the cells necessary for culturing human iPS cells, and the observation image is obtained. A process for identifying an undifferentiated cell region and an undifferentiated deviating cell region in a cell colony and outputting the identification result will be described.

FIG. 2 is a flowchart showing the flow of the region identification processing of the observation image in the cell observation apparatus of this embodiment, and FIG. 3 is a schematic diagram showing an example of the region identification processing.
The operator sets the culture plate 13 containing the cell colony 14 at a predetermined position and performs a predetermined operation on the input unit 30. In response to this operation, the imaging control unit 21 controls the microscopic observation unit 10 to acquire hologram data as follows (step S1).

That is, the light source unit 11 irradiates a predetermined region of the culture plate 13 with coherent light having a spread of a minute angle of about 10 °. The coherent light (object light 16) transmitted through the culture plate 13 and the cell colony 14 reaches the image sensor 12 while interfering with the light (reference light 15) transmitted through the region close to the cell colony 14 on the culture plate 13. .. The object light 16 is light whose phase has changed when it passes through the cell colony 14, while the reference light 15 is light that does not pass through the cell colony 14 and therefore does not undergo the phase change caused by the colony 14. Therefore, on the detection surface (image surface) of the image sensor 12, an image is formed by interference fringes between the object light 16 whose phase has changed due to the cell colony 14 and the reference light 15 whose phase has not changed.

The irradiation region (observation region) of the coherent light emitted from the light source unit 11 is smaller than the size of the culture plate 13. Therefore, the culture plate 13 is irradiated with the coherent light emitted from the light source unit 11 while the light source unit 11 and the image sensor 12 are integrally moved by a predetermined distance in the X-axis direction and the Y-axis direction by a movement mechanism (not shown). , The interference image of a part of the culture plate 13 is repeatedly photographed. As a result, hologram data (two-dimensional light intensity distribution data of the hologram formed on the detection surface of the image sensor 12) over a wide two-dimensional region on the culture plate 13 can be acquired.

As described above, the hologram data obtained by the microscopic observation unit 10 is sequentially sent to the control / processing unit 20 and stored in the hologram data storage unit 22. In the control / processing unit 20, the phase information calculation unit 23 reads the hologram data from the hologram data storage unit 22 and executes a predetermined arithmetic process for phase recovery to calculate the phase information of the entire observation area (photographing area). To do. Then, the image reconstruction unit 24 creates an IHM phase image based on the calculated phase information, and stores the image data in the reconstruction image data storage unit 25 (step S2). When calculating such phase information and creating an IHM phase image, a well-known algorithm disclosed in Patent Documents 3 and 4 and the like may be used. Generally, in an IHM phase image, it becomes easy to see the outline (boundary) of a cell and the pattern inside the cell, which are transparent and difficult to see with an optical microscope.

In addition to the phase information, the intensity information, the pseudo-phase information, and the like are also calculated based on the hologram data, and the image reconstruction unit 24 creates a reproduced image (IHM intensity image, IHM pseudo-phase image) based on these. You can also do it. The IHM intensity image may be used in place of the IHM phase image. Further, the IHM pseudo-phase image is an image obtained by combining the IHM phase image and the IHM intensity image, and corresponds to a phase-contrast microscope image obtained by a phase-contrast microscope. Further, when creating an IHM phase image or the like based on hologram data, it is possible to create an IHM phase image or the like at a plurality of focal positions.

The region identification processing unit 26 performs a process for identifying two characteristic regions, an undifferentiated cell region and an undifferentiated deviant cell region, in the created IHM phase image (step S3). Here, this area identification process will be described.

The area identification processing unit 26 includes a classifier constructed in advance by learning using a large amount of teacher data, and uses this classifier to discriminate a plurality of feature areas on an input image in pixel units and identify them. It outputs the result. What kind of feature area is identified depends on the teacher data used during learning. Therefore, here, a large number of IHM phase images prepared in advance and the result of discriminating between the undifferentiated cell region and the undifferentiated deviant cell region (that is, the correct image) by a skilled worker are trained. Perform learning as data.

The correct image is a label image in which an undifferentiated cell region, an undifferentiated deviated cell region, and a background region (a region in which no cell exists) on a given IHM phase image are labeled on a pixel-by-pixel basis. Roughly speaking, the label image is "1" for pixels determined to be undifferentiated cell regions, "2" for pixels determined to be undifferentiated deviation cell regions, and pixels determined to be other background regions. Is an image to which a predetermined pixel value such as "0" is assigned to, in other words, information in which the position information of each pixel is associated with the pixel value of "0", "1" or "2". is there. If different display colors are assigned to different labels in this label image and drawn, an image in which the undifferentiated cell region, the undifferentiated deviant cell region, and the background region are color-coded and displayed will be obtained.

A large number of pairs of IHM phase images and correct answer images are prepared as teacher data, and a machine learning learning model (that is, a classifier) is constructed so that the output image is as close as possible to the correct answer image using this IHM phase image as an input image. To do. Various methods are known as such supervised machine learning algorithms, such as support vector machines, random forests, AdaBoost, naive Bayes, k-nearest neighbors, and deep learning including neural networks. The method may be used. Further, even when inputting an image, an algorithm may be used in which the feature amount extracted from the image by texture analysis or the like is learned instead of learning the image itself.

The function for constructing the classifier (learned machine learning model) as described above may be built into the control / processing unit 20, but in general, the calculation of such learning processing is considerably burdensome for the computer and has high performance. It takes time if it is not a computer. Therefore, a classifier capable of highly accurate area identification is constructed by performing learning processing using another high-performance computer, and is stored in a memory that is a part of the area identification processing unit 26. .. Then, the area identification processing unit 26 generally performs only the identification processing using the classifier.

In step S3, the area identification process may be performed on the entire IHM phase image created in step S2, but the area identification process is performed only on a part of the range of the IHM phase image designated by the operator. May be carried out.

The region identification result of the region identification processing unit 26 using the above-mentioned classifier is based on the original IHM image in pixel units for each characteristic region (here, undifferentiated cell region, undifferentiated deviation cell region, and other background regions). ) Is a label image labeled. This identification result is stored in the area identification result data storage unit 27. The area identification image creation unit 28 receives such data and creates an area identification image by coloring the pixels included in each of the labeled feature areas with the display color assigned in advance to each label (step S4). ).

The display color assigned to each feature area is predetermined by default, but the operator can arbitrarily specify it from the input unit 30. Specifically, for example, if the undifferentiated cell region is displayed as "green", the undifferentiated deviated cell region is displayed as "red", and the background region is displayed as "black", the boundary between the background and the cell colony (that is, the cell) is displayed. The outline of the colony) and the boundary between undifferentiated cells and undifferentiated deviation cells in the cell colony are clear.

FIG. 3 is a schematic diagram showing an example of a region identification image for an IHM phase image. In this figure, since the color cannot be expressed, each feature area is divided by the difference in shade on the gray scale. In this way, not only the division of each feature area by color display, but also various modes as long as the boundary of each feature area is visually easy to see, such as the difference in shade on the gray scale and the difference in the fill pattern. Can be taken.

The display processing unit 29 creates a display screen in which the area identification image created in step S5 and the IHM phase image are arranged side by side, and displays this on the screen of the display unit 40 (step S5). FIG. 4 is a diagram showing an aspect of such a display screen.

In FIG. 4, an area identification image display frame 51 and an IHM phase image display frame 52 are provided in the identification result display screen 50, and the respective images are displayed in the display frames 51 and 52. Both images are images of exactly the same range on the culture plate 13. For example, on one of the images, the operator uses the input unit (specifically, a pointing device such as a mouse) 30 to enlarge the image. When an operation such as reduction, movement, or rotation is performed, the display processing unit 29 that recognizes the operation reads the data necessary for display from the reconstructed image data storage unit 25 and the area identification result data storage unit 27, and reads the area identification image. The images displayed on the display frame 51 and the IHM phase image display frame 52 are updated. As a result, the operator can always confirm the IHM phase image and the region identification image in the same observation range.

Further, instead of arranging the region identification image and the IHM phase image side by side, it is possible to make one of the images semi-transparent and display them in an overlapping manner. FIG. 5 is a diagram showing an aspect of such a display screen. In FIG. 5, the superimposed image display frame 53 and the slider 54, which is an operator for designating the transparency of the semitransparent image, are provided in the identification result display screen 50, and the superimposed image display frame 53 contains the superimposed image display frame 53. An image in which the region identification image in the same observation range and the IHM phase image are superimposed is displayed. The operator compares the region identification image with the IHM phase image using either the display screen shown in FIG. 4 or FIG. 5, and confirms, for example, the correctness of the distinction between the undifferentiated cell region and the undifferentiated deviant cell region. Can be done.

Further, instead of displaying the area identification image and the IHM phase image side by side or overlapping, only the area identification image may be displayed. The format in which the area identification image is displayed may be selected by the user using GUI widgets such as radio buttons and check boxes arranged on the same screen.

Further, in the region identification image display frame 51 or in the vicinity outside the frame, information indicating the correspondence between the display color and the cell state indicated by the display color (that is, undifferentiated cell state, undifferentiated deviant cell state) is provided. It may be indicated by letters. As a result, the operator can grasp the range of undifferentiated deviant cells at a glance in the displayed region identification image.

As described above, the feature area is identified in pixel units, and the identification result is also recorded in pixel units. Therefore, the area identification image can also be color-coded in pixel units, but it is adjacent instead of pixel units. Each feature area may be color-coded in units of small areas in which a plurality of pixels are grouped. In this case, for example, if any one of the plurality of pixels included in one small region is identified as an undifferentiated deviant cell region, the small region is colored with the color associated with the undifferentiated deviant cell region. It is good to show. This makes it possible to visually clearly indicate the presence of one or a small number of undifferentiated deviant cells even when displaying a low-magnification image, such as the entire culture plate. As a result, it is possible to prevent the undifferentiated deviant cell region from being overlooked when the operator confirms the image.

In the above embodiment, the IHM phase image and the like are divided into three characteristic regions including the background region, but the number of identification regions such as the foreign matter region may be appropriately increased.

Further, in the cell observation device shown in FIG. 1, all the processing is performed by the control / processing unit 20, but in general, a huge amount of calculation of phase information based on hologram data and imaging of the calculation result are performed. Needs to be calculated. In addition, the load of region identification processing by machine learning is heavy. Therefore, a personal computer connected to the microscopic observation unit 10 is used as a terminal device, and a computer system in which this terminal device and a server, which is a high-performance computer, are connected via a communication network such as the Internet or an intranet is used. Complicated calculations and processing such as are performed by a high-performance computer, and the roles are divided so that the control of the microscopic observation unit 10 and the display processing using the processed data are executed by a relatively low-performance personal computer. Good.

Further, in the cell observation apparatus of the above embodiment, an in-line type holographic microscope was used as the microscopic observation unit 10, but if it is a microscope that can obtain a hologram, an off-axis (off-axis) type, a phase shift type, etc. It goes without saying that it can be replaced with a holographic microscope of the above method.

Further, the above-mentioned embodiment is merely an example of the present invention, and it is clear that the present invention is included in the claims even if it is further modified, modified or added as appropriate within the scope of the present invention.

10 ... Microscopic observation unit 11 ... Light source unit 12 ... Image sensor 13 ... Culture plate 14 ... Cell colony 15 ... Reference light 16 ... Object light 20 ... Control / processing unit 21 ... Imaging control unit 22 ... Hologram data storage unit 23 ... Phase information Calculation unit 24 ... Image reconstruction unit 25 ... Reconstructed image data storage unit 26 ... Area identification processing unit 27 ... Area identification image creation unit 28 ... Area identification result data storage unit 29 ... Display processing unit 30 ... Input unit 40 ... Display unit

The cell observation device according to the present invention made to solve the above problems is
a) An observation image acquisition unit that acquires an IHM phase image of cells as an observation image using an in-line holographic microscope (IHM) .
b) For all or a part of the observation image acquired by the observation image acquisition unit, a machine learning model is used to output an image in which two or more feature regions in the observation image are divided in response to the input of the observation image. A region identification unit that identifies characteristic regions that indicate two or more cellular states,
c) An image in which two or more feature areas identified by the area identification unit are divided into visually distinguishable modes to create an area identification image, and the image is displayed on the screen of the display unit. Display processing unit and
It is characterized by having.

Observation image in the present invention is a phase image image obtained from the hologram data obtained in-line holographic microscope.

Supervised machine learning includes Support Vector Machine (SVM), Random Forest, AdaBoost, Simple Bayes, k-neighborhood method, and Deep Learning including neural networks. Etc., an appropriate method can be used. Further, instead of learning the image itself, the feature amount extracted from the image by texture analysis or the like may be learned .

Further, preferably in the present invention, the observation image acquisition unit includes an in- line holographic microscope and an image reconstruction processing unit that creates a phase image and / or an intensity image based on the hologram data obtained by the holographic microscope. , Including
The image display processing unit may be configured to display the area identification image and the phase image and / or intensity image on the same screen.

Claims (14)

a) An observation image acquisition unit that acquires an observation image of cells,
b) A region identification unit that identifies a characteristic region indicating two or more cell states for all or a part of the observation image acquired by the observation image acquisition unit.
c) An image in which two or more feature areas identified by the area identification unit are divided into visually distinguishable modes to create an area identification image, and the image is displayed on the screen of the display unit. Display processing unit and
A cell observation device comprising. The cell observation device according to claim 1.
The region identification unit uses a trained model created by machine learning using an image in which two or more feature regions in the observation image are divided in a distinguishable manner as teacher data, and uses the entire observation image. A cell observation device comprising identifying the characteristic region for or a part thereof. The cell observation device according to claim 1.
One of the characteristic regions is a region that is a cell or a cell colony, and the other one of the characteristic regions is a region that is either an undifferentiated cell or an undifferentiated deviant cell in the region that is a cell or a cell colony. A cell observation device characterized by. The cell observation device according to claim 1.
The image display processing unit is a cell observation device that enlarges or reduces the area identification image displayed on the screen of the display unit according to the operation of the user. The cell observation device according to claim 1.
The area identification unit generates information for identifying the feature area on a pixel-by-pixel basis.
The image display processing unit is a cell observation device characterized in that different feature regions are divided and drawn in a visually identifiable manner in a pixel unit or a unit of a small region in which a plurality of adjacent pixels are grouped together. The cell observation device according to claim 1.
The image display processing unit is a cell observation device, characterized in that information indicating a cell state to be identified by the region identification unit and an identifiable mode for each feature region is displayed together with the region identification image. The cell observation device according to claim 1.
The image display processing unit is a cell observation device characterized in that when a predetermined range is selected on the region identification image, an enlarged image of the range is displayed. The cell observation device according to claim 7.
The image display processing unit is a cell observation device characterized in that the enlarged image and the area identification image before the enlargement are displayed on the same screen. The cell observation device according to claim 7 or 8.
The image display processing unit is a cell observation device characterized in that information indicating a cell state to be identified by the region identification unit and an identifiable mode for each feature region is displayed together with the enlarged image. The cell observation device according to claim 1.
The observation image acquisition unit includes a holographic microscope and an image reconstruction processing unit that creates a phase image and / or an intensity image based on the hologram data obtained by the holographic microscope.
The image display processing unit is a cell observation device characterized in that the region identification image and the phase image and / or intensity image are displayed on the same screen. The cell observation device according to claim 10.
When a predetermined range is selected on the area identification image, the image display processing unit includes an enlarged image of the range and an image of a range corresponding to the enlarged image in the phase image and / or the intensity image. A cell observation device characterized by displaying images on the same screen. The cell observation device according to claim 11.
When an enlargement / reduction operation is performed on any of the enlarged image and the enlarged image of the phase image and / or the intensity image, the image display processing unit links all the images in accordance with the operation. A cell observation device characterized by enlarging / reducing. The cell observation device according to claim 10.
The image display processing unit is a cell observation device characterized by displaying a composite image in which the region identification image and the phase image or intensity image are superimposed. The cell observation device according to claim 13.
The image display processing unit is a cell observation device characterized in that the transparency of at least one of the superimposed images is adjusted according to an operation by a user.

Images