JP2006317406A - Cell imaging device, method, and program, and cell observation device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To minimize damage to a cell by illumination light while culturing a plurality of viable cells for a long period, and to image cell images with an appropriate exposure at a plurality of times in a plurality of visual fields, in imaging cells such as the viable cells containing fluorescence protein having many constraints. <P>SOLUTION: An exposure detection setting section 309 sets an exposure condition at a next imaging time, based on the luminance value of the imaged cell images imaged every visual field in past times, and an imaging section 201 images cells according to each exposure condition set every visual field at the next imaging time. Thus, many cells in a plurality of visual fields can be imaged with the appropriate exposure every visual field. For this purpose, radiation of dedicated and excessive illumination light is not required for exposure measurement, only radiation for essential cell imaging is required, and damage to the cells can be minimized. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a cell image capturing apparatus, a cell image capturing method, a cell image capturing program, and a cell observation apparatus that acquire a cell image appropriate for exposure while culturing living cells for a long period of time.

  Conventionally, various attempts have been proposed in order to obtain an image with proper exposure under imaging conditions with a microscope. As the simplest configuration, there is one that detects the amount of illumination light using a luminance detection sensor and corrects the exposure condition using the detected illumination light amount information. For example, Patent Document 1 discloses a configuration in which an illumination light amount is detected by a luminance detection sensor while switching an optical path, and control is performed so that a shutter speed is optimized.

  Patent Document 2 discloses a configuration in which an optimal exposure condition is obtained from luminance information of an actually captured image, exposure control is performed, and the captured image is corrected according to the exposure condition.

  Patent Document 3 discloses a technique for obtaining an appropriate light amount by performing at least two light amount measurements and at least two exposures in order to cope with the fading of the preparation by exposure correction as a problem peculiar to fluorescence observation. ing.

  Further, Patent Document 4 discloses a configuration in which the amount of light emitted from a strobe, which is a light source, and steady light are monitored by separate sensors, and the exposure is properly maintained by turning off the strobe light source when the accumulated amount of light reaches a predetermined level. is doing.

JP-A-5-341194 JP 2003-163833 A JP-A-6-110104 Japanese Patent No. 3372330

  In general, when fluorescent imaging is performed on a living cell into which a fluorescent protein has been introduced, the luminance of the fluorescent image changes momentarily according to the expression state of the fluorescent protein and the active state of the cell. When the brightness of the cells, which was low at the beginning of the culture, becomes higher with the expression of the fluorescent protein, or when the expression level of the fluorescent protein decreases and the brightness decreases, the brightness increases and decreases repeatedly. The change pattern varies depending on the imaging target. Therefore, it is desirable that the exposure at the time of imaging live cells is adjusted according to the progress of observation.

  Here, it is possible to adjust the exposure before starting observation and use fixed exposure conditions during observation, but it is extremely difficult to set the initial exposure conditions, and changes in luminance are assumed. When exhibiting different behaviors, there is a high possibility that effective image information cannot be obtained due to luminance saturation due to overexposure or blackout due to underexposure. In the long-term culture observation experiment of cells, since one experiment period is long, redoing due to imaging failure in the middle has a large time loss and should be avoided as much as possible. In the case of a general imaging target, feedback control that can sufficiently irradiate the target with light is possible and there are few restrictions. Irradiation causes damage to cells and causes a decrease in activity. Therefore, in order to avoid excessive illumination light irradiation, the number of imaging needs to be minimized.

  In this regard, the one in Patent Document 1 that performs exposure control by measuring the amount of illumination light is intended to always irradiate a certain amount of illumination light on the premise that the state of the observation target is constant. However, since the living cells into which the fluorescent protein is introduced change the luminance every moment according to the expression of the fluorescent protein as described above, there is no guarantee that the exposure will be optimal even when a certain amount of illumination light is irradiated.

  In long-term culture observation of living cells, a certain field of view is taken at 1 frame / several minutes, for example, 1 frame / 15 minutes. Therefore, multiple fields of view that make one round in several minutes are set as multiple observation points. In general, each field of view is imaged at intervals of several minutes while switching the plurality of fields of view. For this purpose, moving the slide glass (sample) for culturing cells so that the relative position with the image sensor changes by the stage transport mechanism, so that live cells are imaged in multiple fields of view at multiple time points. Yes. In this regard, Patent Document 2 is configured on the premise that imaging and exposure control are performed in a short closed loop of about the frame rate, and in a configuration in which exposure control is performed in such a short closed loop, Such multi-field switching cannot be supported. In addition, since irradiation with illumination light and imaging are repeated continuously, the cells are damaged by the illumination light and the activity is reduced.

  Moreover, although the thing of patent document 3 uses the physical model regarding the fading of a preparation, it is very difficult to represent the brightness | luminance change of the living cell which introduce | transduced fluorescent protein with such a physical model.

  Further, the technique disclosed in Patent Document 4 requires monitoring the light amount, accumulating calculation, and feeding back to the light source during a short period of strobe light emission. In this case, it is difficult in terms of speed to use a normal high-resolution image sensor for monitoring the amount of light, and an optical sensor different from the image sensor is required as described in Patent Document 4. Such a configuration complicates the device configuration and also causes a loss of light quantity if the optical path is divided. In addition, changing the amount of illumination light applied to the sample (cell) is not preferable because it affects the cell activity. However, the method disclosed in Patent Document 4 controls the timing at which the strobe light source is turned off. (Irradiation conditions) will fluctuate.

  The present invention has been made in view of the above, and relates to imaging of cells with many restrictions such as living cells into which fluorescent proteins have been introduced, while culturing a plurality of living cells for a long period of time and damaging the cells with illumination light. An object of the present invention is to provide a cell image capturing device, a cell image capturing method, a cell image capturing program, and a cell observation device capable of capturing a cell image with appropriate exposure in a plurality of fields of view at a plurality of times. To do.

  In order to solve the above-described problems and achieve the object, the cell imaging apparatus according to claim 1 includes an imaging unit that images cells in a plurality of visual fields at a plurality of time points according to an exposure condition set for each visual field. Exposure condition setting means for setting the exposure condition at the time of next imaging based on the luminance value of the imaged cell image of the field of view for each field of view.

  According to a second aspect of the present invention, in the above-described invention, the exposure condition setting unit predicts the luminance value of the cell at the next imaging based on the luminance value of the imaged cell image of the visual field for each visual field. And a predicting unit configured to set the exposure condition for the next imaging of the visual field based on the luminance value predicted by the predicting unit.

  According to a third aspect of the present invention, there is provided the cellular image capturing apparatus according to the above invention, wherein the predicting unit predicts a luminance value after recording image data of a captured cell image in a recording device.

  According to a fourth aspect of the present invention, there is provided the cellular image capturing apparatus according to the above invention, wherein the predicting unit predicts a maximum luminance value of the cell at the next imaging.

  According to a fifth aspect of the present invention, in the above-described invention, the cell image capturing apparatus is characterized in that the prediction means predicts a luminance value by polynomial approximation.

  According to a sixth aspect of the present invention, in the above-described invention, the predicting unit preliminarily uses the predicted value of the luminance value of the cell image at the next imaging as a reference table with respect to the luminance value of the cell image captured in the past. The brightness value is predicted with reference to the reference table.

  According to a seventh aspect of the present invention, there is provided the cell imaging apparatus according to the above invention, wherein the exposure condition setting unit sets the exposure condition so that an irradiation condition for the cell at the time of imaging is constant.

  According to an eighth aspect of the present invention, in the above invention, the exposure condition setting unit sets the exposure condition by setting an exposure time.

  In the cell image imaging device according to claim 9, in the above invention, the exposure condition setting means sets the exposure condition by making an exposure time inversely proportional to a luminance value predicted by the prediction means. Features.

  According to a tenth aspect of the present invention, in the above-described invention, the exposure condition setting unit is a reduction provided on an optical path between a sample for culturing cells to be imaged and an imaging element of the imaging unit. The exposure condition is set by setting the degree of dimming by the optical mechanism.

  The cell imaging device according to claim 11 is characterized in that, in the above invention, the dimming mechanism is a neutral density filter.

  The cell image capturing apparatus according to claim 12 is characterized in that, in the above invention, the exposure condition setting means sets the exposure condition by making the transmittance of the neutral density filter inversely proportional to a maximum luminance value. .

  According to a thirteenth aspect of the present invention, there is provided the cell image capturing apparatus according to the above invention, further comprising recording means for adding the exposure condition at the time of capturing to the image data of the captured cell image and recording.

  The cell image pickup device according to claim 14 is characterized in that, in the above-described invention, the cell image pickup device further includes a culture means for culturing the cells, and the image pickup means images the cells in culture stored in the culture means. .

  The cell observation device according to claim 15 is the exposure at the time of imaging the cell image imaging device according to any one of claims 1 to 14 and the luminance of the image data of the cell image captured by the cell image imaging device. Brightness correction means for normalizing according to conditions, cell recognition means for recognizing cells from the image data of the normalized cell image, and cell parameters indicating the characteristics of the cells recognized by the cell recognition means are normalized. A cell parameter measuring means for measuring based on the image data of the cell image, and a correspondence between cells recognized from the image data of each normalized cell image for each visual field captured at different time points. And a cell tracking means based on the above.

  The cell image capturing program according to claim 16 is a cell image capturing program for capturing a cell with a cell image capturing device, and the cell image capturing program includes a plurality of time points according to exposure conditions set for each field of view. An imaging step of imaging a cell in a field of view, and an exposure condition setting step of setting the exposure condition at the next imaging based on a luminance value of a cell image already captured in the field of view for each field of view. And

  The cell image imaging program according to claim 17 is the above invention, wherein the exposure condition setting step predicts the luminance value of the cell at the next imaging based on the luminance value of the imaged cell image of the visual field for each visual field. Including a prediction step, and setting the exposure condition for the next imaging of the visual field based on the luminance value predicted in the prediction step.

  The cell image imaging method according to claim 18 is a cell image imaging method using a culture means for culturing cells and an imaging means for imaging cells contained in the culture means, wherein the cells are captured by the culture means. A culture cell imaging step of acquiring a cell image by imaging the cells in culture according to the exposure conditions set for each field of view at a plurality of fields of view at a plurality of time points while culturing, and for each field of view, An exposure condition setting step for setting the exposure condition at the time of next imaging based on a luminance value of a captured cell image.

  The cell image capturing method according to claim 19 is the above-described invention, wherein the exposure condition setting step predicts the luminance value of the cell at the next imaging based on the luminance value of the imaged cell image of the visual field for each visual field. Including a prediction step, and setting the exposure condition for the next imaging of the visual field based on the luminance value predicted in the prediction step.

  According to the cell imaging device, the cell imaging method, the cell imaging program, and the cell observation device according to the present invention, the exposure at the next imaging based on the brightness value of the captured cell image captured in the past for each field of view. Since the conditions are set and the cells are imaged according to the exposure conditions set for each field of view at the next imaging, a large number of cells across multiple fields of view can be imaged with appropriate exposure for each field of view. For this reason, it is not necessary to irradiate extra illumination light dedicated for exposure measurement, and it is sufficient to irradiate for original cell imaging. There is an effect that it is possible to provide a device suitable for imaging each field of view while culturing cells for a long time.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. A cell observation device including a cell imaging device according to an embodiment of the present invention cultivates a plurality of living cells introduced with a fluorescent protein for a long period of time, and displays an image of a cell while correcting exposure in a plurality of fields of view at a plurality of times. It captures images, recognizes individual cell regions, and tracks cell position changes over time, and independently measures cell parameters indicating the characteristics of individual cells.

  FIG. 1 is a schematic block diagram showing a configuration example of the cell observation device according to the present embodiment. The cell observation device according to the present embodiment generally includes a culture unit 101 for culturing cells, an imaging unit 201 for imaging cells accommodated in the culture unit 101, and the overall processing of the cell observation device. And a control unit 301 for controlling the operation, a recording unit 302 for centrally and permanently recording various data such as image data captured by the imaging unit 201 and processed data, and input of various types of information. In addition to a receiving input unit 303 and a display unit 304 that displays various information such as image information and presents it to the operator, a preprocessing unit 305, a cell recognition unit 306, a parameter measurement unit 307, a cell tracking unit 308, an exposure A detection setting unit 309, an imaging number counting unit 310, an occupied area calculation unit 311, a focus detection unit 312, and a luminance correction unit 313 are provided.

  These units 302 to 313 are connected to the control unit 301 and controlled by the control unit 301. In FIG. 1, control connections from the control unit 301 to the culture unit 101 and the imaging unit 201 are not particularly illustrated. In addition, the control unit 301, the preprocessing unit 305, the cell recognition unit 306, the parameter measurement unit 307, the cell tracking unit 308, the exposure detection setting unit 309, the imaging number counting unit 310, the occupied area calculation unit 311, the focus detection unit 312, Each process performed by the brightness correction unit 313 is performed while a CPU mounted on the cell observation device writes necessary data in a storage device such as a RAM as appropriate based on a processing program stored in a memory such as a ROM. .

  First, the culture unit 101 will be described. The slide glass 102 holds a plurality of living cells C into which fluorescent proteins that are expressed without being localized are introduced in advance, and is installed in the culture unit 101. The culture unit 101 has the same configuration as the culture vessel disclosed by, for example, Japanese Patent Application Laid-Open No. 2004-113175. In this case, the fluorescent protein may be anything as long as it does not localize, and a common jellyfish-derived fluorescent protein or the like can be used. For example, pEGFP-N1 manufactured by BD Biosciences Clontech can be used.

  FIG. 2 is a horizontal sectional view showing a configuration example of the culture unit 101, and FIG. 3 is a longitudinal front view showing a configuration example of the culture unit 101. As shown in FIGS. 2 and 3, the culture unit 101 as a culture means includes a front and back through-hole 103 that can accommodate the slide glass 102 therein, and is made of a material having excellent heat conduction, such as a stainless or aluminum housing. A body 104, an observation window 105 formed of two optically smooth glass plates that block the front and back through-holes 103 of the housing 104, and a culture solution supply pipe 106 that supplies the culture solution A into the housing 104. A culture medium discharge pipe 107 that discharges the culture medium A that is no longer needed from the inside of the casing 104, and two rectifying plates 108 provided at the entrance of the culture medium A to the casing 104.

  In order to grow the living cells C in a healthy manner, it is desirable to always supply the fresh culture solution A over the entire area on the slide glass 102. However, when the flow of the culture solution A is abrupt, there is a possibility that the living cells C that have been deposited on the slide glass 102 will be detached. For this reason, in this embodiment, a rectifying plate 108 is installed in the vicinity of each of the pipes 106 and 107 so that the flow of the culture solution A can be uniformly dispersed and recovered.

  FIG. 4 is a perspective view illustrating a configuration example of the rectifying plate 108. As shown in FIG. 4, the current plate 108 is a porous member in which a plurality of through holes 108 a are formed in the thickness direction. The inlet-side rectifying plate 108 distributes and distributes the culture solution A flowing in from the culture solution supply pipe 106 in the plurality of through holes 108 a, and the outlet-side rectifying plate 108 flows out at once through the culture solution discharge pipe 107. The culture solution A is distributed and distributed in the plurality of through holes 108a. Thereby, the concentrated flow can be converted into a dispersed flow, and the culture solution A can be flowed at a constant flow rate and flow rate in the vicinity of the slide glass 102 where the living cells C are arranged.

  A temperature control unit 109 is attached to the culture unit 101, and a hot water flow path 110 through which the hot water W is circulated is formed around the culture unit 101. By circulating the warm water W through the warm water channel 110, the heat of the warm water is transmitted to the culture solution A via the housing 104. At this time, temperature information from a temperature sensor (not shown) is transmitted to the control unit 301 at predetermined time intervals, and the control unit 301 maintains hot water so that the temperature in the culture unit 101 is maintained in a range of 37 ± 0.5 ° C. Control W temperature and flow rate.

Further, pH information of the culture solution A is transmitted to the control unit 301 at predetermined time intervals by a pH sensor (not shown), and the control unit 301 maintains the CO in the culture solution A so that the pH of the culture solution is maintained in a predetermined range. ₂ Control the concentration.

  The unused culture solution is stored in a culture solution storage unit (not shown), and is kept at about 4 ° C. by a cold storage mechanism (not shown) in order to suppress deterioration over time. The chilled culture solution is heated to about 37 ° C. by a culture solution heating mechanism (not shown) and then supplied into the housing 104 through the culture solution supply pipe 106.

  The culture solution discharged through the culture solution discharge pipe 107 is stored in a waste solution storage unit (not shown). A part of the discharged culture solution may be mixed with fresh culture solution and supplied into the housing 104. In this case, the impact on the cells associated with the replacement of the culture solution is reduced, and the culture can be performed for a longer period. It becomes a suitable configuration.

  FIG. 5 is a cross-sectional view illustrating a heat insulation configuration example of a boundary portion between the culture unit 101 side and the imaging unit 201 side. The heat generated by the culture unit 101 is not transmitted to the imaging unit 201 side by providing a heat insulating unit 111 as a heat insulating unit. Various installation locations of the heat insulating unit 111 that insulates between the culture unit 101 and the imaging unit 201 are conceivable. In the present embodiment, between the housing 104 of the culture unit 101 and the imaging element constituting the imaging unit 201. The heat insulation part 111 is installed in the. The heat insulating portion 111 is a sheet shape using a highly heat-insulating and elastic member such as rubber, silicon, polyurethane, and the like, and is provided with a through-hole 112 having approximately the same diameter as the objective lens 202. The culture unit 101 and the objective lens 202 are optically connected through the through hole 112 and can freely exchange light rays. On the other hand, most of the heat generated by the culture unit 101 is blocked by the heat insulating unit 111. In general, the optical system is adjusted on the assumption that the optical system is used at around 25 ° C., and the performance assumed to be heated by the heat from the culture unit 101 cannot be exhibited. In particular, a solid-state imaging device such as a CCD provided in the imaging unit 201 increases in noise and deteriorates S / N as the temperature rises. Therefore, in order to capture weak fluorescence, it is kept as low as possible (but does not condense). There is a need.

  As the culture means, it is more preferable to use a culture medium such as the culture unit 101 that can exchange the culture solution, but it is also possible to observe cells using a general well plate. However, when a general well plate is used, the culture solution cannot be replaced while maintaining the environmental state. Therefore, compared to the case where the culture unit 101 is used, the culture solution deteriorates due to cell metabolism. The culture period is limited to a short time due to the influence.

  With the above configuration, the temperature inside the culture unit 101 and the pH of the culture solution A are kept substantially constant. Healer cells are used as an example of living cells serving as measurement samples. HeLa cells are derived from cervical cancer and are widely used in drug discovery toxicity tests and the like. The type of fluorescent protein to be introduced may be changed according to the content of the assay.

  Next, the imaging unit 201 will be described. The imaging unit 201 includes an excitation light illumination unit 203, a dichroic mirror 204, an objective optical system 205, an imaging optical system 206, a fluorescence imaging unit 207, an infrared light illumination unit 208, a dichroic mirror 209, The imaging optical system 210 and an infrared light imaging unit 211 are included. That is, the imaging unit 201 of the present embodiment is configured to have a fluorescence imaging system and an infrared light imaging system.

  First, the light emitted from the excitation light illuminating unit 203 is reflected by the dichroic mirror 204 and irradiated onto the slide glass 102 through the objective optical system 205 including the objective lens 202 and the observation window 105. Using the irradiated light as excitation light, fluorescence is emitted from the fluorescent protein introduced into the living cells C on the slide glass 102, and both reflected light and fluorescence of the excitation light are emitted from the observation window 105. The emitted light passes through the objective optical system 205 again and reaches the dichroic mirror 204, but only the fluorescence is transmitted, and the reflected light of the excitation light is blocked. The fluorescence that has passed through the dichroic mirror 204 is enlarged and projected by the imaging optical system 206 onto a solid-state imaging device such as a CCD or CMOS provided in the fluorescence imaging unit 207 as a cell light imaging means.

  The formed fluorescence image of the measurement sample is converted into image data by a solid-state imaging device included in the fluorescence imaging unit 207, and is temporarily or permanently recorded in the recording unit 302 under the control of the control unit 301. FIG. 6 is an explanatory diagram showing an example of a cell image in culture that has been subjected to fluorescence imaging.

  Further, the light emitted from the infrared light illumination unit 208 is irradiated on the slide glass 102 through one observation window 105, and the transmitted light is emitted from the other observation window 105. The emitted light passes through the objective optical system 205 and reaches the dichroic mirror 209, but all infrared light is reflected. The reflected infrared light is enlarged and projected by the imaging optical system 210 onto a solid-state imaging device such as a CCD or CMOS provided in the infrared light imaging unit 211 as an infrared light imaging means. An infrared light image of the formed measurement sample is converted into image data by a solid-state imaging device such as a CCD or CMOS provided in the infrared light imaging unit 211, and temporarily or in the recording unit 302 under the control of the control unit 301. Record permanently.

  In general, fluorescent proteins cannot be introduced uniformly into all cells, and even if introduced, they do not always express immediately, so there is a need for a means of stably observing the entire cell over time, In the present embodiment, this is the infrared light image. When displaying an infrared light image, even in a measurement sample in which little or no fluorescent protein is expressed in the initial state, the observation range can be confirmed and adjusted while visually recognizing the cell image.

  Infrared light has lower phototoxicity to living cells than visible light, so that the activity of cells can be maintained for a longer period of time compared to imaging using visible light. In addition, since the entire visible light can be used as excitation light for fluorescence imaging, restrictions on available fluorescent proteins are eased.

  Thus, according to the imaging unit 201 of the present embodiment, the live cell C is imaged by imaging the live cell C on the slide glass 102 using the fluorescence imaging unit 207 and the infrared light imaging unit 211. Cell image data that is image data of an image can be acquired. In the present embodiment, imaging by the fluorescence imaging unit 207 is automatically performed at preset time intervals under the control of the control unit 301. Then, when it is desired to observe the state of the cells, the user can observe the living cells C using the infrared light imaging unit 211 at a desired time as necessary. In addition, if the imaging by the infrared light imaging unit 211 is performed not only at a desired time by the user but also at a timing synchronized with the imaging by the fluorescence imaging unit 207 under the control of the control unit 301, the infrared light image and the fluorescence In addition to facilitating association with images, it is also easy to associate living cells C included in both images. As a result, when observing the living cells C while culturing them for a long time, it is possible to efficiently observe the living cells C while suppressing a decrease in the activity of the cells. A function of displaying the time when a fluorescent image or an infrared light image is captured on the display unit 304 may be added.

  Since the configuration of the present embodiment includes the fluorescence imaging unit 207 and the infrared light imaging unit 211, it is possible to perform imaging of the fluorescent image and imaging of the infrared light image in parallel, and perform imaging while switching between the two. Compared with the case where it does, the time which imaging requires is reduced significantly, and the drive part for switching is also unnecessary.

  If a ring stop is added to the infrared illumination unit 208 and a phase plate is inserted in the optical path from the dichroic mirror 209 to the imaging optical system 210, phase difference observation can be performed instead of transmission observation. Phase contrast observation provides an image with higher contrast than transmission observation.

  If a polarizer and a DIC (Differential Interference Contrast) element are inserted into the infrared illumination unit 208, and a DIC slider and an analyzer are inserted in the optical path from the dichroic mirror 209 to the imaging optical system 210, differential interference observation is used instead of transmission observation. It can be performed. Differential interference observation provides an image with higher contrast than transmission observation.

  The stage conveyance mechanism 113 changes the relative position between the slide glass 102 and the solid-state imaging device included in the fluorescence imaging unit 207 or the infrared imaging unit 211, and performs imaging and recording as many times as necessary for each field of view (imaging range). Is repeated to acquire image data. In the case of imaging while switching a plurality of fields of view, the stage position in each field of view is recorded, and the stage position is reproduced by the stage transport mechanism 113 prior to the second and subsequent imaging of each field of view. FIG. 7 is an explanatory diagram showing a state of imaging with a plurality of fields of view (views 1 to N). The position of each visual field is arbitrary, and is not particularly limited to a lattice shape. Moreover, there may be overlap between the fields of view. FIG. 8 is an explanatory diagram illustrating an example of the imaging timing of each of the visual fields 1 to N. Each of the visual fields 1 to N performs imaging in a predetermined order so that the imaging interval is substantially constant (for example, every 15 minutes). When imaging each of the visual fields 1 to N, the exposure conditions set by the exposure detection setting unit 309 are followed. The processing content of the exposure detection setting unit 131 will be described later.

  After each image is captured for each field of view, the focus detection unit 312 detects whether or not the focusing at the time of image data imaging is appropriate. If the focusing at the time of imaging is inappropriate, the imaging of the in-focus inadequate part is performed again immediately or at the time when imaging of another arbitrary observation part is completed. At this time, the focusing condition may be changed.

  Further, although the culture solution A circulates in the culture unit 101, the circulation may be temporarily stopped in accordance with the timing at which imaging is performed. Thereby, the fluctuation | variation of the background at the time of the imaging derived from the distribution | circulation of a culture solution can be avoided.

  Further, the number of times of imaging in a predetermined field of view is counted by an imaging number counting unit 310 as an imaging time point recognition unit, and after imaging a predetermined number of times (frames) of a predetermined field of view, an image is displayed on the display unit 304 as a notification unit The operator may be asked to confirm the contents. If the operator determines that there is no problem in the contents, the processing is continued. If it is determined that there is a problem, an instruction from the operator is accepted for resetting the imaging conditions. Alternatively, the processing may simply be stopped. If there is no response from the operator after a certain period of time, the continuation or cancellation of the process is selected according to a predetermined instruction.

  In the present embodiment, the number of times of imaging by the fluorescence imaging unit 207 (or the infrared light imaging unit 211) is counted, and the operator is asked to confirm the image after a predetermined number of times of imaging. In addition to using as a reference, as an imaging time point recognition means, a means for measuring the passage of a predetermined time from the start of observation (for example, acquiring information on the time at which cell image data was captured, and the time exceeds a predetermined time In such a case, it may be possible to confirm the image by providing (recognizing that cell image data at a predetermined time point has been acquired).

  If the culture period is long, the cells in culture may grow and die out of the expected range, the image brightness value may be excessive, etc., and an image suitable for cell observation cannot be acquired at a certain point in time. Cases are also expected. Therefore, as described above, by letting the operator recognize that the predetermined time has passed or by canceling the processing, the operator can check the image acquired so far and the image at that time. Then, subsequent cell observation can be appropriately performed.

  Next, a processing procedure of image data acquired by the fluorescence imaging unit 207 among the captured images will be described. FIG. 9 is a schematic flowchart illustrating an example of image data processing executed by the imaging unit 201 and the like under the control of the control unit 301.

  First, during the imaging of the cell image by the imaging unit 201 (step S1), in parallel with the imaging operation for each visual field, the brightness value of the cell image captured for each visual field by the exposure detection setting unit 309 as the exposure condition setting unit Based on the above, the exposure condition for the next imaging is changed and set (step S2). Thereby, when each field of view is imaged in step S1, the exposure condition is changed so as to follow the exposure condition set in step S2. Prior to various processing of the captured cell image data, the brightness correction unit 313 as brightness correction means performs brightness correction to normalize the cell image data (step S3).

  Next, pre-processing is performed by the pre-processing unit 305 (step S4), and cells are recognized by the cell recognition unit 306 serving as cell recognition means (step S5). Next, cell parameters indicating the characteristics of the recognized cells are measured based on the cell image data normalized by the parameter measuring unit 307 as a cell parameter measuring means (step S6). Further, the cell tracking unit 308 as a cell tracking means determines the identity of cells at different time points recognized from the cell image data of images taken at different time points based on the cell parameters (step S7). Alternatively, the tracking result is further corrected (step S8). Then, the obtained tracking result is displayed on the display unit 304 (step S9), and the above processing steps are similarly repeated until the observation is completed (step S10: Yes).

  Note that the processing of steps S1 to S3 (or steps S1 to S4) is performed in advance at a plurality of times, and the processing after step S4 (or after step S5) for the image data acquired at each time is later performed. You may make it carry out collectively. As described above, when image data captured at a plurality of times is acquired in advance and image data processing is performed later, the image data and image data processing are compared with the case where image data processing is performed in parallel. The device configuration is simplified, and the responsiveness and stability can be improved using an inexpensive computer.

The contents of each processing step will be described individually. First, the process of the exposure detection setting unit 309 will be described. A field of view that has been imaged at time t _j is selected, and a maximum luminance value in the image is obtained from a cell image of the field of view. Similarly, the maximum luminance value in each image is obtained from cell images captured at past times t _j−1 , t _j−2 and t _j−3 in the same visual field. The maximum luminance value in the cell image of the field of view captured at time t _{j + 1} corresponding to the next imaging is predicted by polynomial approximation using the obtained maximum luminance values. FIG. 10 is a characteristic diagram showing an example of how the luminance value of the cell image changes over time. The past time t _j-3 ~t _j polynomial approximation based on respective maximum luminance values captured in the maximum luminance value in its field of view of the cell image captured at time t _{j + 1} corresponding to the next time imaging As a result of prediction, it is shown by a broken line in FIG. Here, the time t _jx represents the imaging time before the visual field of interest x times, and the time t _{j + x} represents the imaging time of the visual field of interest after x _rotations .

Here, it supplements about polynomial approximation. _{Assuming that} the maximum luminance value at time t _j is expressed as I _j or the like, the maximum luminance value I _{j + 1} at time t _{j + 1} can be predicted as in equation (1) by cubic approximation. However, the maximum luminance value such as I _j is a value obtained by normalizing actually measured values under respective exposure conditions.

In addition, although the example of the 3rd order polynomial approximation is shown here as an example, the polynomial approximation by another order may be sufficient and a nonlinear approximation may be performed. The degree of polynomial approximation may be dynamically changed according to the progress of observation. Further, approximation by least square estimation may be performed. In addition, with respect to the luminance value of the cell image captured in the past, the predicted value of the luminance value of the cell image at the next imaging is stored in advance as a reference table, The luminance value can also be predicted. For example, the correspondence between the maximum luminance value before time t _j and the predicted value of the maximum luminance value at time t _{j + 1} is prepared in advance as a reference table, and the maximum at time t _{j + 1} is referred to by referring to the reference table. It is also possible to obtain a predicted value of the luminance value.

Here, when the predicted value I _{j + 1} of the maximum luminance value at the next imaging is smaller than the predetermined lower limit value I _JL or larger than the predetermined upper limit value I _JU , with the current exposure condition, The exposure of the cell image obtained at the time of the next imaging is expected to be inappropriate, and the exposure condition needs to be set (changed).

  The exposure condition can be set by setting the amount of irradiation light, a dimming mechanism (for example, insertion / extraction of ND (Neutral Density) filter (neutral density filter)), aperture, and exposure time. However, among these, it is necessary to consider that changing the setting of the amount of irradiation light applied to the sample (slide glass 102) affects the activity of cells in culture. In addition, changing the aperture setting affects the depth of field at the time of imaging, and the linearity of brightness is lost, making it difficult to normalize and correct observation data before and after changing exposure conditions. There is. That is, it becomes difficult to guarantee temporal consistency of observation data. Therefore, in the long-term culture observation of living cells, it is more preferable to set the exposure condition by inserting or removing the ND filter on the optical path between the sample (slide glass 102) and the image sensor, or by changing the exposure time.

Here, as a procedure for setting (changing) the exposure condition, for example, a processing procedure when setting the exposure time will be described. In the case of this method, the optimum exposure time T _{j + 1} at the next imaging is calculated according to the equation (2).

[Equation 2]
T _{j + 1} = α _j · (I _JU / I _{J + 1} ) · T _j (I _{j + 1} > I _JU )
T _{j + 1} = T _j (I _JL ≦ I _{j + 1} ≦ I _JU )
T _{j + 1} = β _j · (I _JL / I _{J + 1} ) · T _j (I _{j + 1} <I _JL )
………………………………………………………… (2)

Here, T _j represents the current exposure time, and α _j and β _j represent predetermined coefficients. In order to actually set the exposure time, the shutter speed of the camera may be changed. For example, the charge accumulation time in the image sensor may be set (changed), or the exposure time may be set (changed) using a mechanism such as a mechanical shutter.

Such processing is performed for each field of view, and an optimal exposure time T _{j + 1} is calculated and set as an exposure condition. At the next imaging, the irradiation condition is always constant, and imaging is performed using a shutter speed corresponding to the optimum exposure time T _{j + 1} . As shown in Equation (2), the exposure time is set to be shorter as I _{j + 1} becomes larger. As a result, it is possible to capture an appropriately exposed cell image without luminance saturation or black crushing while minimizing damage to living cells during culture. When the image data of the captured cell image is recorded in the recording unit 302, the exposure condition at the time of imaging is also recorded.

  The above-described exposure detection setting process is performed at a time point between any imaging for each of the fields of view 1 to N. Here, immediately after the start of observation, sufficient observation data for predicting the maximum luminance value has not been obtained yet, so the initial exposure condition set by the operator is used as it is, and the prediction of the maximum luminance value is Switch to set (change) exposure conditions for prediction.

In addition, as a procedure for setting (changing) the exposure condition, for example, a processing procedure in the case of ND filter insertion / extraction will be described. In the case of this method, the optimum transmittance T ′ _{j + 1} of the ND filter at the next imaging is calculated according to the equation (3).

[Equation 3]
T ′ _{j + 1} = α ′ _j · (I _JU / I _{J + 1} ) · T ′ _j (I _{j + 1} > I _JU )
T ′ _{j + 1} = T ′ _j (I _JL ≦ I _{j + 1} ≦ I _JU )
T ′ _{j + 1} = β ′ _j · (I _JL / I _{J + 1} ) · T ′ _j (I _{j + 1} <I _JL )
………………………………………………………… (3)

Here, _T'j transmittance of the current ND filter, _{α'j, β'j} represents a predetermined coefficient. In the present embodiment, as indicated by a broken line in FIG. 1, the ND filter 212 having the optimum transmittance obtained by the equation (3) is used for the sample (that is, the slide glass 102) and the imaging device of the fluorescence imaging unit 207. Provide on the optical path between.

Such processing is performed for each field of view, and an optimal transmittance T ′ _{j + 1} is calculated and set as an exposure condition. At the next imaging, the irradiation condition is always constant, and imaging is performed using the ND filter 212 having the optimum transmittance T ′ _{j + 1} . A plurality of types of ND filters 212 may be prepared in advance and switched by a turret, or the transmittance may be continuously changed using liquid crystal. As shown in Expression (3), the optimum transmittance of the ND filter is set so as to decrease as I _{j + 1} increases. As a result, it is possible to capture an appropriately exposed cell image without luminance saturation or black crushing while minimizing damage to living cells during culture. When the image data of the captured cell image is recorded in the recording unit 302, the exposure condition at the time of imaging is also recorded.

  Note that the exposure condition may be changed by either changing the exposure time or inserting / removing the ND filter, or both of them may be used. Further, the above-described processing by the exposure detection setting unit 313 may be performed on either the fluorescent image or the infrared light image.

FIG. 11 is a schematic flowchart illustrating an example of an imaging process performed by the imaging unit 201 in which such exposure detection setting processing is performed in parallel. First, the variable n related to the visual field is set to 1 and the variable j related to the imaging timing time is initialized to 0 (step S101). Next, the exposure condition relating to the target visual field n is read (step S102). Here, the initial exposure conditions set by the operator or the like are read out until the exposure conditions associated with the prediction of the luminance value are set, and then changed when the exposure condition change setting is made. The exposure condition is read out. Then, wait until the imaging timing t _j of the attention field n (step S103: No), When turned capturing timing (step S103: Yes), the exposure condition changed as necessary in accordance with the set exposure conditions Then, the cells in the visual field n are imaged (step S104). After the imaging, the image data of the captured cell image is recorded in the recording unit 302 together with the exposure conditions at that time (step S105). Thereafter, n is incremented by +1 and the field of interest n is moved to the next (step S106). At this time, when n reaches the total number of visual fields N (step S107: Yes) and imaging is not completed (step S108: No), n is set to 1 and the visual field of interest is returned to the first visual field, and the time for imaging timing is related. The variable j is incremented by +1 (step S109), and the same processing is repeated from step S102.

FIG. 12 is a schematic flowchart showing an example of exposure detection setting processing by the exposure detection setting unit 309 executed in parallel with the imaging processing. First, a variable n related to the field of interest to be subjected to exposure detection setting processing is set to 1 (step S201). Next, the process waits until time t _j reaches a predetermined threshold value t _T as the degree of progress of imaging by the imaging unit 201 (step S202). As described above, since the observation data sufficient for predicting the maximum luminance value is not obtained immediately after the start of observation as described above, the exposure detection setting process is performed until the number of times of imaging specified by the threshold t _T is over. Is a process for waiting and using the initial exposure conditions. In the present embodiment, the threshold value t _T is set to be equivalent to, for example, four times, as shown in FIG.

If the threshold is reached t _T (step S202: Yes), determines whether reached exposure detection timing related interest field n (step S203), whether the image of time t _j is already captured (step S204). The detection timing is a time point between any imaging for each field of view n. Whether an image has been captured means a state in which image data of the cell image is recorded in the recording unit 302. The image data is used for the next process of predicting the maximum luminance value. Since the image data is also the original image data, the process waits until the storage process is completed. Reaching detection timing (step S203: Yes), if the image is already captured (step S204: Yes), obtains the maximum luminance value of each cell image at time t _j ~t _j-3 of the noted field n, The maximum luminance value I _{j + 1} of the cell of the field of interest n at the time t _{j + 1} at the next imaging is predicted by polynomial approximation (step S205).

Then, when the predicted maximum luminance value I _{j + 1} falls within the predetermined lower limit value I _JL to upper limit value I _JU (step S206: Yes), the exposure condition is not changed and set. When the determined maximum luminance value I _{j + 1} does not fall within the range of the predetermined lower limit value I _JL to the upper limit value I _JU (step S206: No), the imaging at the time t _{j + 1} of the field of interest n The exposure condition for use is changed and set (step S207). Thereafter, n is incremented by +1 and the field of view n is moved to the next (step S208). At this time, if n reaches the total number of visual fields N (step S209: Yes) and the detection setting process is not completed (step S210: No), n is set to 1 and the visual field of interest is returned to the first visual field (step S211). The same processing is repeated from step S202.

  Next, processing by the brightness correction unit 313 will be described. The luminance correction unit 313 normalizes the luminance of the cell image image data captured as described above. That is, each pixel data is multiplied by a coefficient according to the exposure condition recorded together with the image data of the cell image. This coefficient is determined to be larger as the exposure time is shorter and smaller as the exposure time is longer.

  Thereafter, using the normalized image data of the cell image, the pre-processing unit 305 and the subsequent processes are performed as follows. First, in step S4, the preprocessing unit 305 applies an edge-preserving low-pass filter to the image data. The edge-preserving low-pass filter has a smoothing effect other than the edge part while suppressing deterioration of the spatial frequency high-frequency component at the edge part, and is capable of removing noise while preserving cell outline information. It is suitable for this method.

  A bilateral filter (see Tomasi & Manduchi, “Bilateral Filtering for Gray and Color Images”, Proceedings of the 1998 IEEE International Conference on Computer Vision, Bombay, India) is known as a filter that satisfies these requirements. This is also used in the method.

  Next, a sharpening filter for edge enhancement is further applied to the image data after applying the edge preserving low-pass filter. The sharpening filter is a filter that obtains the sum by performing weighting as shown in FIG. 13, for example, on the pixel of interest and its neighboring eight pixels, and sharpening processing can be realized by repeatedly executing this for each pixel.

  In step S5, the pre-processed image data is analyzed by the cell recognition unit 306 according to the following procedure, and the area occupied by each cell is recognized. According to this procedure, not only when the cells are scattered without being adjacent to each other, but also when the cells are adjacent to each other and dense, the area occupied by each cell can be recognized. The present invention can also be applied when the edge of the cell region is not clear.

  First, an image is divided into regions where high luminance pixels are concentrated. In general, in a fluorescent image, cells have a high-luminance pixel cluster appearance, so dividing an area into areas (lumps) where high-luminance pixels are concentrated corresponds to dividing the image into areas for each cell. .

  As a process that satisfies such requirements, watershed area division is known. As the cell recognition processing procedure of the first embodiment, this watershed region segmentation method is used (Vincent & Soille, “Watersheds in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations”, IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL.13, NO.6, JUNE 1991). In the original paper, watershed area division is to divide an image into areas where low-luminance pixels are concentrated, but here, the luminance is inverted and applied to the division of high-luminance areas. Individual regions in the obtained region division result are cell regions.

  Here, integration processing may be performed in which a plurality of cell regions are integrated into a new cell region according to the characteristics of adjacent cell regions. Since the result of the watershed area division process generally tends to be divided into small areas, the quality of the recognition result can be improved by performing the integration process.

A first method of region integration will be described with reference to FIG. FIG. 14 is a schematic flowchart showing a first method example of region integration. First, the point where the luminance is maximum in each cell region, that is, the vertex of the luminance is obtained (step S511). Next, any two adjacent cell regions are selected (step S512), and a road distance D _UW along a line segment connecting the vertices is obtained (step S513). Equation (4) is used to calculate the road distance D _UW .

Here, I (P) is the luminance value of the pixel P in the image after application of the edge preserving low-pass filter, and / I (P _S ) is the luminance value of two vertices in the image after application of the edge preserving low-pass filter. The average, Σ, represents that the sum is obtained for all the pixels of the line segment connecting the vertices.

In step S513, the path distance D _UW between vertices is obtained for all combinations of adjacent cell regions, and in step S514, the path distance D _UW is compared with a predetermined threshold value V _UW . As a result of the comparison, if it is equal to or less than the predetermined threshold value V _UW (step S514: Yes), the cell regions are integrated into one region (step S515). This process is repeated in the same manner until all the combinations are completed (step S516: Yes).

A second method of region integration will be described with reference to FIG. FIG. 15 is a schematic flowchart showing a second method example of region integration. First, an edge extraction filter, for example, a Sobel filter is applied to the output result of the edge preserving low-pass filter to obtain an edge image (step S521). Considering the boundary between any adjacent cell regions, any two adjacent cell regions are selected (step S522), and the edge strength D _UE defined by equation (5) is obtained (step S523).

  Here, E (P) represents the luminance value of the pixel P in the edge image, and Σ represents that the sum is obtained for all the pixels included in the boundary between the cell regions.

In step S523, after determining the edge strength D _UE for all combinations of adjacent cell area, in step S524, and compares this edge strength D _UE with a predetermined threshold value V _UE. As a result of the comparison, if it is equal to or less than the predetermined threshold value V _UE (step S524: Yes), the cell regions are integrated into one region (step S525). This process is repeated in the same manner until all the combinations are completed (step S526: Yes).

  These first and second region integration methods may be used separately or sequentially in any order. Further, the validity of each cell region may be verified using luminance information. For this purpose, a pixel having the maximum luminance value is obtained for each divided cell region. If the luminance value is smaller than a predetermined threshold value Vtmin, it is determined that the region is not a cell region, and the pixel to which it belongs is included. Excluded from subsequent processing. As a result, cells with insufficient introduction or expression of the fluorescent protein and background regions that are not cells can be excluded.

  Furthermore, the luminance of each pixel in the cell region may be compared with a predetermined threshold value Vpmin, and pixels having a luminance lower than the threshold value Vpmin may be excluded from the cell region. The pixels excluded in this way are not used for the subsequent processing. Thereby, a low-luminance part with a low S / N can be excluded from the cell region, and the boundary shape of the cell region can be recognized more accurately. The obtained cell region and a set of pixels belonging to each cell region are recorded in the recording unit 302.

In addition, it is also possible to recognize a cell area | region using the infrared-light image which the infrared light imaging part 211 imaged. When the infrared light image is a phase difference image, the luminance value of the area where the cells exist is observed as a luminance value different from the background. Therefore, a difference from the representative background luminance value P _BG is obtained for each pixel in the image, only pixels whose difference is larger than a predetermined threshold value V _PG are extracted, and a general labeling process is performed to determine adjacent pixels. If integrated, the cell region can be recognized.

  Returning to the processing of FIG. 9, in step S <b> 6, the parameter measurement unit 307 measures the cell parameter for each cell region recognized by the cell recognition unit 306, and records the measurement result in the recording unit 302. FIG. 16 is an explanatory diagram illustrating an example of measurement results of cell parameters recorded in the recording unit 302. Where M is the number of recognized cell regions. In the present embodiment, cell parameters are, for example, the position of the center of gravity, area, circularity, sum of luminance, average luminance, and standard deviation of luminance as measurement item targets, and are associated with the image data and cell region in the recording unit 302. Record. Depending on the content of the assay, general measurement items such as perimeter, ferret diameter, length, width, and maximum brightness may be added.

  Here, in the present embodiment, the total area of all the cells in the image, that is, the area corresponding to the cell occupation value indicating the degree of the cell region in the image is occupied area calculation section 311 as the occupied area calculation means. When the area occupied by the cell region in the image exceeds a predetermined ratio with respect to the area of the image, the control unit 301 is notified of the event. In this case, the control unit 301 may further notify the operator through the display unit 304 serving as a notification unit, or may change the control state of the culture unit 101 according to a preset setting. Alternatively, you can simply ignore the notification. This function is effective when notifying that the free space in the medium is becoming insufficient in the cell culture process because the free space in the medium may decrease due to cell growth or the like when the culture is prolonged. .

The occupied area calculation unit 311 obtains the area occupied by the cell region in the cell image as the cell occupation value. The cell image may be either a fluorescent image or an infrared light image. When a fluorescent image is a target, the area of the cell region is measured by the parameter measurement unit 307. Therefore, if the areas of all the cell regions in the image are summed, the area occupied by the cell region in the image can be obtained. it can. When an infrared light image is a target, the luminance value of the area where the cells exist is observed as a luminance value different from the background. Therefore, first, a difference from the representative background luminance value P _BG is obtained for each pixel in the image, and only pixels whose difference is larger than a predetermined threshold value V _PG are extracted. Furthermore, if the number of pixels extracted in the image is counted, the area occupied by the cell region in the image can be obtained.

  With the above procedure, the cell parameters can be measured after obtaining the area of each individual cell with respect to a single cell image including a plurality of cell images. By repeatedly capturing a cell image and measuring a parameter every predetermined time interval Δt, it is possible to accumulate cell parameters over time.

  Here, in the process as it is, the cell parameters measured at different times are not associated with each other, and cannot be said to be a state measured over time. Therefore, it is necessary to associate cell regions between cell images taken at different times, and associate cell parameters using the results.

The cell region association is executed in the cell tracking unit 308 as the processing of steps S7 and S8 as follows. Here, the cell region recognized at time t ₁ is represented as R _{t1, m} , and the cell region recognized at time t ₂ is represented as R _{t2, n} . However, time t ₂ is time after time t _{1 in} time series. m and n are identification numbers of cell regions that do not overlap in the same image, and 1 ≦ m ≦ M and 1 ≦ n ≦ N. M and N are cell regions recognized at times t ₁ and t ₂ , respectively. Represents a number.

First, an evaluation function relating to the relationship between two cell regions R _{t1, m} and R _{t2, n} is defined by equation (6). It can be said that the smaller the evaluation value J ₁ calculated by the equation (6), the higher the relationship between the two regions, and the higher the possibility of showing the same cell.

[Equation 6]
_{J 1 = J 1 (R t1} , m, R t2, n) = k d δ d + k a δ a + k c δ c ...... (6)
δ _d : distance between centroids δ _a : area difference δ _c : circularity difference k _d , k _a , k _c : predetermined weighting factors

FIG. 17 is an explanatory diagram showing the results of calculating the evaluation values for possible combinations of m and n. However, in FIG. 17, J ₁ (R _{t1, m} , R _{t2, n} ) is simply expressed as J _{m, n} for ease of description.

Here, the region R _{t2, n ^} at time t ₂ corresponding to the region R _{t1, m} at time t ₁ is determined according to the equation (7). That is, R _{t2, n ^} is a region at time t ₂ that minimizes the evaluation value J ₁ between the region R _{t1, m} .

If the evaluation value J ₁ is the smallest n ^ there are multiple, applying a second evaluation function shown in Equation against them (8), the evaluation value J ₂ determines the combination of smaller. When there are a plurality of combinations that minimize the second evaluation value J ₂ , a message to the operator is displayed on the display unit 304, and a combination that the operator determines to be correct is input from the input unit 303. Based on this, the association is performed.

[Equation 8]
_{J 2 = J 2 (R t1} , m, R t2, n) = k s δ s + k m δ m + k v δ v ...... (8)
[delta] _s: the difference of the sum of luminance [delta] _m: difference in average luminance [delta] _v: the difference k _s of the standard deviation of brightness, k _m, k _v: predetermined weighting factor

However, it is possible to speed up the processing by using only _{one of} the evaluation values J ₁ and J ₂ without obtaining both. Further, the message display to the operator and the input step from the operator may be omitted, and a plurality of correspondences that minimize the evaluation value J ₁ or J ₂ may be recorded.

Region R _t1, region R _t2 of _m and the time t _{2, n ^} is the time t _1, since considered the result of recognition of the same cell at a different time, even instrumented cell parameters of the two different for the same cell It can be regarded as a measured value at time. Therefore, the parameter measurement over time is completed by associating the value of the cell parameter with the cell image, the cell region, the association information of the cell region, and the time information and recording it in the recording unit 302 as a recording unit.

Over time, when a fluorescent protein is newly expressed and emits fluorescence, cells that were outside the observation screen move into the observation screen, multiple overlapping cells are separated, or cells If is was split, the number of cell areas to be recognized in the cell recognition unit 306 is increased, the time t ₁ corresponding to the cell area of the time t ₂ and when the cell area absent, the time t ₂ a plurality of A case occurs where a cell corresponds to one cell at time t ₁ .

In addition, if the fluorescence intensity of the fluorescent protein decreases over time, the cells that were in the observation screen move to the outside of the observation screen, multiple cells overlap, or the cells die, since the number of cell areas to be recognized in the recognition unit 306 is reduced, the time t ₂ corresponding to the cell region of the time t ₁ or when the cell area absent, a plurality of cells at time t ₁ is a time t ₂ 1 The case corresponding to one cell occurs.

  If there is no corresponding cell area, a flag indicating that there is no corresponding area is recorded. When a plurality of cell regions correspond to one cell region, all correspondences are recorded. A message may be displayed to the operator through the display unit 304, and the correspondence relationship may be corrected based on input from the operator. The data expression when recording the correspondence of a plurality of cell regions uses a tree structure in which each time is at a height and each cell region is associated with a node. A graph structure with a higher degree of freedom of expression may be used.

In addition, regarding the association of cell regions, a modified example with the following improvements may be used. In the first modification, when the minimum evaluation value J is larger than a predetermined threshold value V _jmax , the association is regarded as invalid. In this case, it is assumed that the region at time t ₂ corresponding to the region R _{t1, m} cannot be found, and the time-dependent parameter measurement for the region R _{t1, m} is terminated by time t ₁ . This modification is effective for reducing the influence of noise.

In the second modified example, the distance between the centers of gravity of the region R _{t2, n ^} corresponding to the region R _{t1, m} is obtained, and when the distance between the centers of gravity is larger than a predetermined threshold value V _dmax , the association is regarded as invalid. In this case, it is assumed that the region at time t ₂ corresponding to the region R _{t1, m} cannot be found, and the time-dependent parameter measurement for the region R _{t1, m} is terminated by time t ₁ . This modification is effective for reducing errors in the cell region association processing. With the above procedure, cell parameters can be measured over time.

  Finally, returning to the process of FIG. 9, as the process of step S9, the recognized cell region and the measured cell parameter are displayed on the display unit 304 as the cell parameter display means. FIG. 18 is an explanatory diagram illustrating an example of a display of processing results. The display screen 314 included in the display unit 304 has two display areas 314a and 314b, and individual cell areas recognized at the processing target time point are displayed on the display area 314a. Here, a labeling process is applied to the cell region, and a color, brightness, line type, and pattern that can be identified for each region are given and displayed as, for example, label images a to e. An infrared light image or a fluorescent image in the same display range as the label image may be displayed in conjunction with each other, or a plurality of label images, infrared light images, and fluorescent images may be displayed in an overlapping manner. Alternatively, superimpose display may be performed. The measured cell parameters are displayed as a line chart with time on the horizontal axis and parameter values on the vertical axis in the display area 314b. Furthermore, if the display contents of both are synchronized and highlighted in accordance with the mouse operation or the like in the input unit 303 by the operator, the visibility of the display contents is improved. FIG. 19 is an explanatory diagram showing an example in which a line chart of cell parameters corresponding to, for example, the case where the label image c is selected and designated as an instruction to be highlighted is also highlighted. In this case, when the operator selects and emphasizes one of them, the corresponding other is also highlighted in synchronization.

  In this embodiment, the fluorescent protein is introduced into the living cell C and observed, but if a luminescent gene, for example, a luciferase gene is introduced instead of the fluorescent protein, a luminescent image can be taken instead of the fluorescent image. In this case, the excitation light illumination unit 203 and the dichroic mirror 204 are unnecessary, and the configuration can be simplified. The luminescent image is captured by a fluorescence imaging unit 207 as a cell light imaging unit. The light emission image may be processed in the same procedure as the fluorescence image. In this manner, cell image data can be acquired and observed even when the cell emits light other than infrared light, for example, when the cell emits light or emits fluorescence.

  In this embodiment, a fluorescent protein that is expressed without being localized in the cell is used. However, the fluorescent protein is expressed in a localized manner in the cell nucleus, cytoplasm, nuclear membrane, cell membrane, or organelle. It doesn't matter.

  Note that the cell parameters measured by the parameter measurement unit 307 are not limited to those exemplified in the present embodiment, and further, the area, the perimeter, the circumscribed rectangle position, the X-direction ferret diameter, the Y-direction ferret diameter, and the minimum ferret diameter , Maximum ferret diameter, average ferret diameter, convex circumference, roundness (roundness), number of holes, roughness (ratio of convex circumference to circumference), Euler number, length, width, flatness, brightness The total sum, minimum brightness, maximum brightness, average brightness, brightness standard deviation, brightness dispersion, entropy, center of gravity position, second moment, main axis direction, or a plurality thereof may be used.

  Further, the parameter measurement unit 307 can calculate the number of cells, the minimum inter-cell distance, the maximum inter-cell distance, the average inter-cell distance, the standard deviation of the inter-cell distance, and the dispersion of the inter-cell distance for a group consisting of a plurality of arbitrary cells. In addition, any one or more of the minimum value, maximum value, average value, standard deviation, fractional difference, sum, intermediate value of each parameter measured for each cell may be obtained.

  As described above, according to the present embodiment, while culturing a plurality of living cells into which a fluorescent protein has been introduced for a long period of time, exposure is corrected in a plurality of fields of view at a plurality of times, and cell images are captured with appropriate exposure. It is possible to realize an apparatus for measuring individual cell parameters over time while recognizing a cell region and tracking changes in position over time.

  The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention. For example, the processing procedures by the respective units such as the exposure detection setting unit 309 and the luminance correction unit 313 may be realized by executing a cell observation program prepared in advance by a microcomputer such as the control unit 301. This cell observation program can also be distributed via a network such as the Internet. The cell observation program can also be executed by being recorded on a recording medium readable by a microcomputer such as a hard disk, FD, CD-ROM, MO, or DVD and being read from the recording medium by the microcomputer.

It is a schematic block diagram which shows the structural example of the cell observation apparatus containing the cell image imaging device which concerns on embodiment of this invention. It is a horizontal sectional view which shows the structural example of a culture part. It is a vertical front view which shows the structural example of a culture part. It is a perspective view which shows the structural example of a baffle plate. It is sectional drawing which shows the heat insulation structural example of the boundary part of a culture part side and an imaging part side. It is explanatory drawing which shows an example of the cell image in culture by which the fluorescence imaging was carried out. It is explanatory drawing which shows the mode of the imaging by a several visual field (visual fields 1-N). It is explanatory drawing which shows the example of an imaging timing of each visual field 1-N. It is a schematic flowchart which shows the example of image data processing. It is a characteristic view which shows an example of the mode of a time-dependent change of the luminance value of a cell image. It is a schematic flowchart which shows the example of an imaging process by an imaging part. It is a schematic flowchart which shows the example of an exposure detection setting process by an exposure detection setting part. It is a figure which shows the example of weighting by a sharpening filter. It is a schematic flowchart which shows the 1st example of a technique of area | region integration. It is a schematic flowchart which shows the 2nd example of a method of area | region integration. It is explanatory drawing which shows the example of a measurement result of the cell parameter recorded on the recording part. It is explanatory drawing which shows the result of having calculated the evaluation value about the possible combination of m and n. It is explanatory drawing which shows an example of the display of a process result. It is explanatory drawing which shows an example of a highlight display.

Explanation of symbols

201 Image pickup unit 212 ND filter 302 Recording unit 306 Cell recognition unit 307 Parameter measurement unit 308 Cell tracking unit 313 Exposure detection setting unit 314 Brightness correction unit

Claims (19)

Imaging means for imaging cells in a plurality of fields of view at a plurality of time points in accordance with exposure conditions set for each field of view;
Exposure condition setting means for setting the exposure condition at the time of next imaging based on the luminance value of the imaged cell image of the field of view for each field of view;
A cell image imaging apparatus comprising:   The exposure condition setting unit includes a prediction unit that predicts a luminance value of a cell at the next imaging based on a luminance value of a captured cell image of the visual field for each visual field, and sets the luminance value predicted by the prediction unit. The cell image capturing apparatus according to claim 1, wherein the exposure condition for the next imaging of the visual field is set based on the exposure condition.   The cell image capturing apparatus according to claim 2, wherein the predicting unit predicts a luminance value after recording image data of a captured cell image in a recording apparatus.   The cell image capturing apparatus according to claim 2, wherein the predicting unit predicts a maximum luminance value of a cell at the next imaging.   The cell image capturing apparatus according to claim 2, wherein the prediction unit predicts a luminance value by polynomial approximation.   The prediction means stores in advance a predicted value of the luminance value of the cell image at the time of the next imaging as a reference table for the luminance value of the cell image captured in the past, and refers to the reference table to determine the luminance value. Prediction is performed, The cell imaging device according to any one of claims 2 to 4 characterized by things.   The cell image capturing apparatus according to claim 1, wherein the exposure condition setting unit sets the exposure condition so that an irradiation condition for the cell during imaging is constant. .   The cell image capturing apparatus according to claim 1, wherein the exposure condition setting unit sets the exposure condition by setting an exposure time.   9. The cell image capturing apparatus according to claim 8, wherein the exposure condition setting unit sets the exposure condition by making an exposure time inversely proportional to a luminance value predicted by the prediction unit.   The exposure condition setting means sets the degree of dimming by a dimming mechanism provided on an optical path between a sample for culturing cells to be imaged and an image sensor of the imaging means. The cell image capturing apparatus according to claim 1, wherein the setting is performed.   The cell image capturing apparatus according to claim 10, wherein the dimming mechanism is a neutral density filter.   The cell image imaging apparatus according to claim 11, wherein the exposure condition setting means sets the exposure condition by making the transmittance of the neutral density filter inversely proportional to a maximum luminance value.   The cell image capturing apparatus according to any one of claims 1 to 12, further comprising recording means for adding the exposure condition at the time of capturing to the image data of the captured cell image and recording the image data. A culture means for culturing cells;
The cell imaging device according to claim 1, wherein the imaging unit images a cell in culture stored in the culture unit. The cell imaging device according to any one of claims 1 to 14,
Brightness correction means for normalizing the brightness of the image data of the cell image captured by the cell image capturing device according to the exposure condition at the time of imaging;
Cell recognition means for recognizing cells from the image data of the normalized cell image;
Cell parameter measuring means for measuring cell parameters indicating the characteristics of the cells recognized by the cell recognition means based on the image data of the normalized cell image;
Cell tracking means for performing correspondence between cells recognized from the image data of each normalized cell image for each visual field captured at different time points based on the cell parameters;
A cell observation apparatus comprising: A cell imaging program for imaging cells with a cell imaging device,
In the cell imaging device,
An imaging step of imaging cells in multiple fields of view at multiple time points according to the exposure conditions set for each field of view;
An exposure condition setting step for setting the exposure condition at the next imaging based on the luminance value of the imaged cell image of the visual field for each visual field;
A cell image capturing program characterized in that   The exposure condition setting step includes a prediction step of predicting the luminance value of the cell at the next imaging based on the luminance value of the imaged cell image of the visual field for each visual field, and the luminance value predicted in the prediction step The cell image imaging program according to claim 16, wherein the exposure condition for the next imaging of the visual field is set based on the setting. A cell image imaging method using a culture means for culturing cells and an imaging means for imaging cells contained in the culture means,
A cultured cell imaging step of acquiring a cell image by culturing cells in the culturing means while imaging cells in culture according to exposure conditions set for each field of view in a plurality of visual fields at a plurality of time points by the imaging means,
An exposure condition setting step for setting the exposure condition at the next imaging based on the luminance value of the imaged cell image of the visual field for each visual field;
A cell image imaging method comprising:   The exposure condition setting step includes a prediction step of predicting the luminance value of the cell at the next imaging based on the luminance value of the imaged cell image of the visual field for each visual field, and the luminance value predicted in the prediction step 19. The cell image capturing method according to claim 18, wherein the exposure condition for the next imaging of the visual field is set based on the exposure condition.

Images