JP6424911B2 - Method for detecting movement of cardiomyocytes, method for culturing cardiomyocytes, drug evaluation method, image processing program and image processing apparatus - Google Patents

Description

  The present invention relates to a motion detection method, a culture method of cardiomyocytes, a drug evaluation method, an image processing program, and an image processing apparatus, which are optimal for motion analysis of cardiomyocytes differentiated from stem cells.

  In the field of regenerative medicine and drug discovery for myocardial diseases and the like, cardiac muscle which is induced to differentiate from stem cells such as embryonic stem cells (ES cells) and induced pluripotent stem cells (iPS cells) Cells are expected to be used as resources for regenerative medicine and drug discovery. An example of an apparatus for observing the state of cultured cells such as cardiomyocytes includes a culture microscope. In addition, as a method of monitoring cardiomyocytes in a culture vessel, cells or cell populations in a culture vessel containing cardiomyocytes are imaged at regular intervals, and images of cells or cell populations obtained by imaging are compared. In some cases, cardiomyocytes are monitored separately from other cells (see, for example, Patent Document 1).

Unexamined-Japanese-Patent No. 2007-121106

  As described in Patent Document 1 above, there is provided a method of identifying cardiomyocytes in which stem cells autonomously repeat contraction / relaxation pulsating motion cyclically in the process of culturing cardiomyocytes. However, in Patent Document 1 described above, although it is possible to detect the beating area of cardiomyocytes, it is not possible to accurately identify the beating center cells (pacemaker cells) within the beating area.

  The present invention has been made in view of the above problems, and it is an object of the present invention to provide means capable of accurately detecting periodic movement of cells from time-series images acquired in cell observation.

According to a first aspect of the present invention, acquiring a time-series image of cell colony candidates for cardiomyocytes at a predetermined time interval, and one of the cell colonies of the acquired time-series image Extracting the area of the part, detecting the direction of movement of the extracted area, and detecting the stagnation area of movement of the cell colony based on the detected direction of movement. A method of cardiac muscle movement detection is provided.

According to a second aspect of the present invention, there is provided an image processing program that is readable by a computer and causes the computer to function as an image processing apparatus that acquires an image captured by an imaging device and processes the image. A step of acquiring a time-series image of cell colonies to be candidates for cardiomyocytes by a predetermined time by the imaging device; and one of the cell colonies of the acquired time-series image by the image processing device. Moving the cell colony based on the direction of movement detected by the image processing apparatus; and extracting the area of the part, detecting the direction of movement of the extracted area by the image processing apparatus, and the image processing program executed by detecting the residence area, to the computer is provided.

According to a third aspect of the present invention, an image analysis unit is provided which acquires a time-series image in which cell colonies which are candidates for cardiomyocytes are taken at predetermined time intervals by an imaging device and analyzes the image. In the image processing apparatus, the image analysis unit detects a direction of movement of the extracted region, and a partial region extraction unit that extracts a partial region in cell colonies of the acquired time-series image. There is provided an image processing apparatus comprising: a movement direction detection unit; and a central region detection unit that detects a stagnation region of movement of the cell colony based on the detected movement direction.

  According to the present invention, it is possible to accurately detect periodic movement of cardiomyocytes and quantitatively evaluate and judge the cell movement.

It is a flowchart which shows the outline | summary of an image processing program. It is a schematic block diagram of the culture | cultivation observation system shown as an application example of this invention. It is a block diagram of the said culture | cultivation observation system. It is a schematic diagram for demonstrating the mode of beating of a cardiac muscle cell. It is an observation image of the cardiomyocyte colony image | photographed at predetermined time intervals, (a) is a 1st image at the time of relaxation, (b) is a schematic diagram which shows the 2nd image at the time of contraction. It is a schematic diagram for demonstrating the process which performs detection of a motion vector. FIG. 1 is a block diagram showing a schematic configuration of an image processing apparatus. It is a schematic diagram for demonstrating the flow of a pulsation center detection method. It is a schematic diagram for demonstrating the process which calculates an attention point and a small area | region window. It is a schematic diagram which shows the pulsation center area | region in a colony. It is a flowchart which shows the outline | summary of the pulsation center detection method. It is a table | surface which shows the morphological characteristics of the cell and cell colony for specifying the pulsation area | region of a cardiomyocyte.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. As an example of a system to which the image processing apparatus according to the present embodiment is applied, schematic configuration diagrams and a block diagram of a culture observation system are respectively shown in FIG. 2 and FIG. Do. Subsequently, a series of image processing up to identification of pacemaker cells of cardiomyocytes will be described by this culture observation system BS. To identify the pacemaker cells, "cell colonies" of the cultured cells (cardiomyocytes) are extracted based on the morphological characteristics of the cultured cells and cell colonies, and the extracted cell colonies are "moved" from the image. The “area” is identified, and the “beating area” of the cardiomyocytes is further identified from the moving area to identify the “cells (pacemaker cells)” at the beating center (see Table 1 in FIG. 12). Note that this "moving area" includes cells that are not directly related to the beating motion of cardiomyocytes, and so-called noise is also included. Furthermore, an example in which the evaluation of the beating region of cardiomyocytes can be applied to regenerative medicine and drug discovery will be described.

  The culture observation system BS roughly observes the culture chamber 2 provided in the upper part of the housing 1, the shelf-like stocker 3 accommodating and holding a plurality of culture vessels 10, and the sample in the culture vessel 10 Operation unit 5, a transport unit 4 for transporting the culture vessel 10 between the stocker 3 and the observation unit 5, a control unit 6 for overall control of the operation of the system, and a control panel 7 having an image display device. Etc.

The culture room 2 is a room that forms a culture environment, and is a constant temperature room covered with a heat insulating material. Along with the culture chamber 2, a temperature control device 21 for adjusting the temperature in the culture chamber 2, a humidifier 22 for adjusting humidity, a gas supply device 23 for supplying a gas such as CO ₂ gas or N ₂ gas, culture A circulation fan 24 for making the environment of the whole room 2 uniform, an environment sensor 25 for detecting the temperature and humidity of the culture room 2, the concentration of carbon dioxide, and the like are provided. The operation of each device is controlled by the control unit 6, and the culture environment defined by the temperature, humidity, carbon dioxide concentration and the like of the culture chamber 2 is maintained in a state in which the culture conditions set in the operation panel 7 are met.

  The stocker 3 is formed in the shape of a shelf partitioned into front and rear and upper and lower, and an address unique to each shelf is set, for example, when the longitudinal direction is A to C rows and the vertical direction is 1 to 7 , A five rows of shelves are set as "A-5". The culture vessel 10 is appropriately selected from a dish, a well plate, a flask, etc. according to the type of cells to be cultured and the purpose, and for example, a cell sample ("cardiomyocytes" described in detail below) It is injected and maintained with a predetermined medium. A code number is given to each culture vessel 10, and is stored in association with the designated address of the stocker 3.

The transport unit 4 is provided inside the culture chamber 2 and comprises a Z stage 41 movable up and down, a Y stage 42 movable back and forth, an X stage 43 movable left and right, etc. A support arm 45 is provided to lift and support the container 10. The transport unit 4 is configured such that the culture vessel 10 can move between all the shelves of the stocker 3 and the observation unit 5.

  The observation unit 5 includes a macro observation system 54 for macro observation of the sample, a microscope observation system 55 for micro observation of the sample, and a first illumination for illuminating the sample from below the sample table 15 along the optical axis of the macro observation system 54. A portion 51, a second illumination portion 52 which illuminates the sample from above the sample table 15 along the optical axis of the microscopic observation system 55, and a third illumination portion 53 which illuminates the sample from below the imaging device of the observation systems 54, 55 The image processing apparatus 100 includes, for example, an image processing apparatus 100 that acquires a photographed image and performs various image processing. The sample table 15 is formed of a translucent material, and a transparent window 16 is provided in the observation area. In addition, the sample stage 15 includes a fine drive stage movable in the XY direction (in the horizontal plane) and the Z direction (vertical direction) by drive control from the control unit 6, and the culture vessel 10 mounted on the upper surface thereof Is moved in the X, Y, and Z directions, so that the observation region of the culture vessel 10 can be positioned within the observation visual field of the macro observation system 54 or the microscopic observation system 55.

  The first lighting unit 51 is a light source of surface light emission provided on the lower frame 1 b side, and backlights the entire culture vessel 10 from below the sample table 15. The second illumination unit 52 includes a light source 52 a such as an LED (Light Emitting Diode) or a halogen lamp, and an illumination optical system 52 c including an annular diaphragm and a condenser lens, etc. Microscopic observation from above the sample table 15 The sample in the culture vessel 10 is illuminated along the optical axis of the system 55. The third illumination unit 53 includes a plurality of light sources 53a such as LEDs or mercury for emitting light of a wavelength suitable for epi-illumination observation or fluorescence observation, and an optical axis of the microscopic observation system 55 for light emitted from each light source 53a. The illumination optical system 53 c including a beam splitter and a fluorescence filter to be superimposed on each other illuminates the sample in the culture vessel 10 along the optical axis of the microscopic observation system 55 from below the sample table 15. The illumination light amount of each of the light sources 52 a and 53 a is controlled based on the drive signal (current / voltage) from the control unit 6.

  The macro observation system 54 includes an observation optical system 54 a and an imaging device 54 c such as a CCD camera for photographing an image of a sample formed by the observation optical system 54 a. A whole observation image (macro image) from above of the illuminated culture vessel 10 is taken.

  The microscopic observation system 55 includes an observation optical system 55a having an objective lens, a phase ring, and the like, and an imaging device 55c such as a cooled CCD for photographing an image of a sample formed by the observation optical system 55a. The objective lens is configured to be reciprocally movable in the direction (Z direction) along the optical axis by the lens drive mechanism 87, and to change the relative position (optical distance) between the objective lens and the sample surface in the optical axis direction. Makes it possible to adjust the focus. In addition, a plurality of objective lenses are provided and can be set to a plurality of magnifications using a displacement mechanism such as a revolver or a slider. According to the lens setting of the initial selection, at least for low magnification observation. It is switched so as to be able to change magnification between two types of magnifications (for example, for 2 × observation) and for high-magnification observation (for example, 10 × observation). The microscopic observation system 55 includes a phase difference image of the sample illuminated by the second illumination unit 52, a reflection image of the sample illuminated by the third illumination unit 53, a fluorescence image of the sample illuminated by the third illumination unit 53, etc. A microscopic observation image (micro image) obtained by microscopically observing the sample in the culture vessel 10 is taken.

The image processing apparatus 100 is photographed by the imaging device 54c of the macro observation system 54 and the imaging device 55c of the microscopic observation system 55, and processes signals inputted from these imaging devices to process image data of the whole observation image or the microscopic observation image Generate Further, the image processing apparatus 100 performs image analysis on these observed images (image data), generates a time-lapse image, detects a motion vector of cell motion, calculates an image feature quantity, and analyzes a cell pulsation motion. , Etc. The image processing apparatus 100 will be described in detail later.

  The control unit 6 has a CPU 61 for executing processing, a ROM 62 in which control programs and control data of the culture observation system BS are set and stored, a RAM 63 for temporarily storing observation conditions and image data, etc. Control the operation. Therefore, as shown in FIG. 3, the respective constituent devices of the culture chamber 2, the transport device 4, the observation unit 5, and the control panel 7 are connected to the control unit 6. In the RAM 63, environmental conditions of the culture room 2 according to the observation program, an observation schedule, an observation type or an observation position in the observation unit 5, an observation magnification, and the like are set and stored. In addition, the RAM 63 is provided with an image data recording area for recording the image data photographed by the observation unit 5, and the index data including the code number of the culture vessel 10, the photographing date and time, etc. Be done.

  The operation panel 7 is provided with an operation panel 71 provided with an input / output device such as a keyboard, a mouse, and a switch, and a display panel 72 for displaying an operation screen, image data, etc. Selection, input of operation command, etc. are performed. The communication unit 65 is configured in accordance with a wired or wireless communication standard, and data can be transmitted and received to and from a computer externally connected to the communication unit 65.

  In the culture observation system BS configured as outlined above, the CPU 61 controls the operation of each part according to the observation program set in the operation panel 7 and automatically executes imaging of the sample in the culture container 10. When the observation program is started, the CPU 61 reads each condition value of the environmental conditions stored in the RAM 63 and detects the environmental condition of the culture room 2 input from the environmental sensor 25 and detects the difference between the condition value and the actual value. In response to this, the temperature control unit 21, the humidifier 22, the gas supply unit 23, the circulation fan 24 and the like are operated to perform feedback control on the culture environment such as the temperature, humidity, and carbon dioxide concentration of the culture chamber 2.

  Further, the CPU 61 reads the observation conditions stored in the RAM 63, operates the X, Y, and Z stages 41, 42 and 43 of the transport unit 4 based on the observation schedule to observe the culture container 10 to be observed from the stocker 3. The sample is transported to the sample stage 15 and the observation by the observation unit 5 is started. For example, when the observation set in the observation program is macro observation, the culture vessel 10 conveyed from the stocker 3 by the conveyance unit 4 is positioned on the optical axis of the macro observation system 54 and placed on the sample table 15 Then, the light source of the first illumination unit 51 is turned on, and the entire observation image is photographed by the imaging device 54c from above the culture vessel 10 which is back lighted. The signal input from the imaging device 54c to the control unit 6 is processed by the image processing apparatus 100 to generate a whole observation image, and the image data is stored in the image data storage area of the RAM 63 together with index data such as shooting date and time. Be done.

  When the observation set in the observation program is micro observation of the sample at the specific position in the culture vessel 10, the light of the microscopic observation system 55 in the specific position in the culture vessel 10 transported by the transport unit 4 Positioned on the axis and mounted on the sample table 15, the light source of the second illumination unit 52 or the third illumination unit 53 is turned on to make the imaging device 55c capture a microscopic observation image by transmitted illumination, epi-illumination, or fluorescence . A signal captured by the imaging device 55c and input to the control unit 6 is processed by the image processing apparatus 100 to generate a microscopic observation image (phase difference image, fluorescence image, etc.), and the image data is an index such as shooting date and time It is stored in the image data storage area of the RAM 63 together with data and the like.

[Extraction of cell colony of cardiomyocytes]
In the culture observation system BS configured as described above, in the image processing apparatus 100, images of the cultured cells in the culture container 10 are acquired in time series at predetermined time intervals by the imaging devices 54c and 55c (time lapse Shooting). Thus, based on the time-series image in which the cell population (colony) of cardiomyocytes differentiated from the stem cells is photographed, the morphological feature of the cell colony of cardiomyocytes is detected, and the cell colony of cardiomyocytes is extracted. The acquired time-series image is analyzed, and for example, as shown in Table 1 of FIG. 12, the morphological features (outline, thickness, area, feature points (center of gravity, start point, end point, inflection point, branch point) of cell colonies Etc.), shape (including changes in cell internal structure), brightness etc.).
Specifically, as described in Table 1 of FIG. 12, cell colonies are identified based on morphological features of cells.
I. In the case of using the contour of a cell colony, for example, (i) a cell colony in which the average luminance or the dispersion value of the luminance of the contour band of the cell colony (a band having a predetermined width inside along the contour line) is a predetermined value or more Identify Also, (ii) a cell colony is identified from the brightness comparison between the average brightness and the variance of the brightness of the contour band and the variance of the average brightness and the brightness inside the colony.
II. In order to image the entire cell colony with an appropriate exposure amount in consideration of the thickness of the cell colony, for example, (i) the imaging device 54c identifies the cell colony from an image acquired for each different exposure amount. Also, instead of changing the exposure amount, an image in which the sensor gain of the imaging device is adjusted may be used. In addition, it is possible to use the image which adjusted the sensor gain similarly for both exposure similarly also in specification of-movement area | region * pulsation site | part * pacemaker cell mentioned later. Furthermore, instead of the above (i), an image may be acquired by a sensor with a wide dynamic range of the imaging device.
III. When the area of cell colonies is used, (i) those having an area of a predetermined value or more are specified as cell colonies. Alternatively, (ii) those having a predetermined area range or more are identified as cell colonies. IV. When using the shape of a cell colony, (i) the thing of a predetermined | prescribed shape (for example, roundness is more than predetermined value) is specified as a cell colony. Alternatively, (ii) those whose internal structural change is larger than a predetermined structural change are identified as cell colonies (details will be described later). Or (iii
2.) Using a spatial frequency difference between peripheral cells present around cell colonies and cell colonies, a spatial filter (low pass filter) transmitting spatial frequencies possessed by cell colonies is applied to identify cell colonies. The spatial filter may use an optical filter, or may calculate and filter an image.
V. In the case of using brightness information (brightness information) of cell colonies, (i) a cell colony having a brightness distribution in the contour of the cell colony equal to or more than a predetermined brightness distribution is specified as the cell colony. Alternatively, (ii) a cell colony is identified that has a contour band of cell colony having a predetermined brightness.

[Detection of beating motion of cardiomyocytes]
In the culture observation system BS configured as described above, the image processing apparatus 100 acquires a time-series image in which a cell population (colony) of cardiomyocytes is photographed at predetermined intervals by the imaging devices 54c and 55c. It has a function to analyze the acquired time-series image and detect the pulsation movement of cardiomyocytes.

Cardiomyocytes are, for example, embryonic stem cells (ES cells) or induced pluripotent stem cells as a resource for regenerative medicine and drug discovery for heart diseases and the like.
stem cells (iPS cells) are induced to differentiate and autonomously repeat contraction / relaxation pulsating motion periodically at predetermined time intervals. The cardiomyocytes adhere to each other to form a cell population (colony), and are dispersed in the dish (the culture vessel 10) together with the feeder cells.

The image processing apparatus 100 detects a motion vector from the acquired time-series image, sequentially calculates the image feature quantity of the region having motion in which the motion vector is detected, and based on the time-series change of the calculated image feature quantity. It is configured to detect the pulsation of cardiac muscle cells. Now, an image processing method (cell motion detection method) executed by the image processing apparatus 100 will be described from the basic concept. The following description exemplifies a case in which observation is performed based on a phase difference image (microscopic observation image) captured by a phase contrast microscope configured by the second illumination unit 52, the microscopic observation system 55, and the like. In addition, as long as cells can be observed, an observation method other than phase contrast observation may be used. At this time, in the microscopic observation system 55, low-magnification observation and high-magnification observation are possible by switching the observation magnification of the observation optical system 55a.

(Extraction of motion area)
As shown in FIG. 4, since the beating motion of the cardiomyocytes MC is a periodic motion that repeats contraction and relaxation, first, a motion vector is detected from a time-series image obtained by photographing a cell population (colony), The area where the motion vector is detected is extracted as an area with motion ("motion area"). As described in Table 1 of FIG. 12, a motion vector is detected by obtaining “feature points (center of gravity, start point, end point, inflection point, branch point, etc.)” of cell colonies. The details will be described below.

  First, time series images (time t = 1, 2, 3,..., T−1, T,...) In which the colony C of cardiomyocytes MC is photographed are acquired, and a first time t− in this time series image is acquired. 1 (see, for example, FIG. 5 (a)) and a second image (see, for example, FIG. 5) taken at a second time t that is a predetermined time after the first time t-1. 5 (b)) and.

  As shown in FIG. 6, a motion vector (indicated by a symbol "V" in the figure) is, for example, an optical flow (block matching) between a first image at time t-1 and a second image at next time t. It estimates using known methods, such as a method, a gradient method, etc.). This motion vector is detected, for example, in units of pixels in each image, and from the center coordinates (start point) of the target pixel in the first image to the center coordinates (end point) of the corresponding pixel in the second image corresponding to the target pixel Calculated as a vector of More specifically, a motion vector is a vector (also referred to as a "velocity vector") representing the direction and magnitude of motion at each detection point between two images (between the first and second images). The direction of this vector represents the direction of movement, and the length of the vector represents the amount of movement. Here, each pixel in which a motion vector is detected is defined as a motion area, but the view image is divided into blocks of N × M pixels (plural pixels), and the motion vector is detected in units of blocks. This block may be defined as a motion area.

  In the following, the time interval Δt between the first time t-1 and the second time t is matched to the timing of contraction / relaxation, the first time t-1 is relaxation time, and the second time t is The case of capturing a time-series image at the time of contraction will be described as an example. In addition, if it is possible to select at least a first image obtained by photographing the relaxation time and a second image obtained by imaging the contraction time from the acquired time-series images, the sampling interval for acquiring the time-series images is It is not particularly limited.

Here, when the colony C of the cardiomyocytes MC is observed in time series, it is assumed that all the regions in which the motion vector is detected (moving regions) are necessarily beating regions (beating regions) that alternately repeat contraction and relaxation. There is no limit. That is, in the detection of the motion vector, not only only the beating cardiomyocytes are detected as a motion area, but also peripheral cells (including feeder cells) FC which move by being pulled and contracted by contraction and relaxation of the cardiomyocytes. Also, floating cells NC suspended in the medium in the dish and accompanied with movement are detected as movement areas. However, for the present purpose, it is preferable that only cardiomyocytes involved in the beating be accurately detected in distinction from the others. At this time, an internal structure (texture) such as a nucleus or an internal fiber is present in the cardiomyocytes MC, and the internal structure is significantly changed when the cardiac muscle cells repeat contraction / relaxation beating (morphological change). More specifically, the internal structure becomes dense at the time of contraction, and the internal structure becomes the original state (sparse compared to the time of contraction) at the time of relaxation. That is, although the illumination light irradiated to the beating site in the phase contrast observation is divided into the direct light passing through the site and the diffracted light, this diffraction phenomenon occurs at the site where there is a change in the refractive index, so the diffraction The light contains the shape information of the internal structure (phase object) of the beating site, and can be regarded as a remarkable change of the internal structure when the cardiac muscle cell repeats contraction and relaxation beating. On the other hand, foreign substances such as other cells (peripheral cells, floating cells) and dusts may be said to have very little change in internal structure even with movement. Therefore, image feature quantities related to changes in internal structure are calculated in time series with respect to a motion area in which a motion vector is detected, and a beat site is detected from colonies of cardiomyocytes based on time series changes in the image feature quantities. Method (beat detection method) to

(Time-series change of morphological feature)
An image feature (morphological feature) is calculated for a motion area in which a motion vector is detected between a first image and a second image in a time-series image, and a time-series change of the image feature is calculated. Specifically, focusing on the start point and end point of the motion vector for each motion area, the first image corresponds to the small area near the point (near the pixel) corresponding to the start point and the end point on the second image The feature quantities of the image are respectively calculated in the small area near the point (pixel vicinity). Here, for the same object, the small area at the start point means the motion area (or the area including the motion area) at time t-1, and the small area at the end point is the motion area at the time t (or the area including the movement area Means). Therefore, by taking the difference between the image feature quantities of the small area at the start point and the small area at the end point, it is possible to derive a time-series change of the image feature quantity of the motion area between the first image and the second image.

  As the feature amount of the image, various known indexes can be used. For example, the following can be listed. That is, the texture feature quantities indicating the internal structure of the cell colony include (1) sum of differentials, (2) variance of luminance value, and (3) spatial frequency.

(1) Sum of Differentiation The edge intensity is calculated by differentiation of the luminance in the small area (moving area) with respect to the space direction (including difference of luminance values in vertical, horizontal, and diagonal directions, Sobel Filter, etc.).

(2) Dispersion of luminance values By looking at the dispersion of luminance values in a small area (moving area), since there is a dispersion if the value is large, for example, increase or decrease of granules in cells, the number or size of nuclei, Unevenness due to density can be estimated.

(3) Spatial frequency Spectral intensity of spatial frequency by Fourier transform is a multi-dimensional feature that is more detailed texture pattern analyzed unlike the above simple features. By calculating this correlation value, more detailed information can be captured.

  In addition, as described in Table 1 of FIG. 12, in order to detect the movement region of cell colonies, the change rate (change in contraction / expansion) of cell colony area is calculated based on time-series images, The pixel group is identified as a motion area when it indicates the rate of change of. Furthermore, the motion area has a high probability of being present at the edge portion of the cell colony, and it is also possible to specify the motion area by extracting the luminance change area of the edge portion. In addition, in order to consider the thickness of the cell colony, as described above, the exposure dose may be adjusted.

(Extraction of beating area (beating part))
The difference between the texture feature amount in the first image thus obtained and the texture feature amount in the second image is calculated, and this difference is taken as a time-series change amount. As described above, since the internal structure changes alternately in the form of dense → sparse → dense → sparse → ... by repetition of contraction and relaxation at the beating site, the texture feature (image feature) also changes accordingly. . That is, the texture feature increases when the internal structure becomes denser and the texture feature decreases when the internal structure becomes looser. Therefore, the difference of the texture feature amount with respect to the beating site appears as a change amount having a certain size. On the other hand, in the foreign matter such as peripheral cells which are non-pulsating parts or dust, even if there is movement, almost no change in the internal structure is observed, and the change in the texture feature amount is scarce. Therefore, the change amount of the texture feature amount is compared with a predetermined threshold set in advance, and when the change amount of the texture feature amount is equal to or more than the predetermined threshold, the region (motion region) having the texture change is It is determined as

  The predetermined threshold value is, for example, observed in advance a colony of typical cardiomyocytes prior to the detection of a pulse, and is previously referred to as a time-series change of the texture feature value obtained by sampling the beating site present in this colony. It is preferable to calculate. Note that, for example, the predetermined threshold may be arbitrarily changed by the user interface. In this user interface, when “Threshold setting” is selected on the operation panel 71, the “slide lever” is displayed on the display panel 72, and drag operation of the slide lever using a mouse, a keyboard, etc. By moving the lever), it is possible to select the desired threshold.

  The image feature amount is not limited to the above-described texture feature amount, and another index may be adopted. For example, the same effect can be obtained using a simple brightness value (average brightness value). As described above, since repetition of contraction and relaxation at the pulsation site causes the internal structure to change alternately in the form of dense → sparse → dense → sparse → ..., the luminance value also changes accordingly. That is, illumination light is absorbed and darkened when the internal structure becomes dense (the luminance value decreases), and transmitted light increases and becomes bright (the luminance value increases) when the internal structure becomes loose. Therefore, the difference of the texture feature amount with respect to the beating site appears as a change amount having a certain size. On the other hand, in the peripheral cells, dust, etc., almost no change in internal structure is seen, and it can be said that the change in luminance value is also poor. As described above, even when the luminance value is adopted as the image feature quantity, the time-series change of the luminance value appears prominently only in the pulsation site, so that the pulsation site can be detected based on the change in the image feature quantity. it can.

  In addition, as described above, cardiac muscle cells undergo morphological deformation of the internal structure at the timing of pulsation (contraction / relaxation) to change the feature amount (texture feature amount, luminance value, etc.) of the image, so the beat discrimination method It may be added as a condition that the temporal cycle of contraction / relaxation (the reciprocating motion of the motion vector) and the time-series change of the feature of the image are synchronized, by using this method It becomes possible to detect a moving site more accurately.

  Further, as described in Table 1 of FIG. 12, since the beating region has a high probability of being present at the edge portion of the cell colony, a dispersion filter matched to the edge width is applied to obtain the dispersion value of luminance, and this dispersion A value equal to or greater than a predetermined value is identified as a pulsation region. In addition, the pulsation site is identified from within the motion area according to the magnitude of the motion vector. Alternatively, in the motion area, the luminance change / luminance distribution is captured as described above, and the pulsation area is specified. Alternatively, the beat region is identified based on the spectral intensity of the spatial frequency of the internal structure in the movement region of the cell colony.

  Next, a specific application of image analysis performed in the image processing apparatus 100 of the culture observation system BS will be described with reference to FIGS. 1 and 7 together. Here, FIG. 1 is a flowchart showing an outline of processing in the image processing program GP1, and FIG. 7 is a block diagram showing an outline configuration of the image processing apparatus 100.

The image processing apparatus 100 includes an image storage unit 110 that stores an image captured by the imaging devices 54c and 55c, an image analysis unit 120 that acquires and analyzes an image stored in the image storage unit 110, and an image analysis unit 120. And an output unit 130 that outputs the analysis result analyzed by The image analysis unit 120 detects a motion vector between the first image and the second image from the time-series image using a known method such as optical flow (block matching method, gradient method, etc.), and And a feature amount calculator configured to calculate a time-series change of the image feature amount for the motion area in which the motion vector is detected. The output unit 130 outputs an analysis result by the image analysis unit 120, for example, a detection result of a motion vector, information of time-series change in image feature amount, a discrimination result of a cardiomyocyte that beats, etc. Display on the screen, recording on a storage medium, data transmission to the outside via the communication unit 65, and the like are performed.

  The image processing apparatus 100 is configured such that the image processing program GP set and stored in the ROM 62 is read by the CPU 61, and processing based on the image processing program GP is sequentially executed by the CPU 61. In other words, the image processing program GP is software for causing the CPU 61 (computer), which is a hardware resource, to function as an image processing apparatus.

  As described above, in the culture observation system BS, observation of the cardiomyocyte population (colony) in the designated culture container 10 is performed at predetermined time intervals according to the observation conditions set in the observation program. Specifically, the CPU 61 operates the respective stages 41, 42, 43 of the transport unit 4 to transport the culture vessel 10 to be observed from the stocker 3 to the observation unit 5 (in the present embodiment, the optical axis of the microscopic observation system 55) Place on top). The CPU 61 moves the sample table 15 in the in-plane direction (X and Y directions), and moves the objective lens along the optical axis direction by the lens drive mechanism 87 to move the sample in the culture vessel 10 to the objective lens. Perform positioning. Here, an objective lens for high-magnification observation (for example, 10 ×) is used, and thereby, it is possible to more quantitatively judge the feature relating to the internal structure of cardiomyocytes based on the high resolution image.

  Based on the image processing program GP, the image analysis unit 120 is photographed by the imaging device (in the explanation, the imaging device 55c of the microscopic observation system), and the image of the colony of cardiomyocytes stored in the image storage unit 110 is as follows. Image processing.

  First, in step S101, the image analysis unit 120 separates a predetermined time from the first image at time t-1 stored in the image storage unit 110 (for example, the image at the time of relaxation shown in FIG. 5A). The second image at the next time t (for example, the image at the time of contraction shown in FIG. 5B) is acquired.

  In step S102, the motion vector detection unit 121 detects a motion vector between the first image and the second image in pixel units (or block units of a plurality of pixels) using a known method such as an optical flow method. Specifically, as shown in FIG. 6, an arrow indicating a motion vector is allocated to a motion area (pixel or block) in which motion is detected between the first image and the second image.

  In step S103, the feature amount calculation unit 122 calculates a time-series change of the image feature amount for a small area (moving area) near the pixel where the motion vector is detected. Specifically, for each motion vector, the image feature amount is calculated for each of the small region corresponding to the start point of the motion vector in the first image and the small region corresponding to the end point of the motion vector in the second image. Derived as the change amount of Although various indexes can be used as the image feature amount, here, as described above, a texture feature amount, an average luminance value, and the like are exemplified.

  In step S104, the image analysis unit 120 compares the change amount of the image feature amount with a predetermined threshold, and the small region where the change amount of the image feature amount is equal to or more than the predetermined threshold is a pulsation region (beat detection region Detect as).

In step S105, the determination result is output from the output unit 130 and displayed on the display panel 72 or the like. As a specific display method, for example, the pulsation detection area and the other area (non-beat area) are displayed with different hues or luminances, or the non-beat area is filled and displayed, or the non-beat area is displayed. The pulse detection area and the non-beat area are displayed in a distinguishable manner by displaying the image from which the image is removed. In addition, the above determination result output from the output unit 130 is transmitted to a computer or the like externally connected through the communication unit 65, and basic data for displaying a similar image or observing the state of cardiomyocytes It can be used as

[Detection of beating center of cardiomyocyte colony]
By the way, in the colony of cardiomyocytes, pacemaker cells (paced cells) are present at the center of their pulsations, and electrical signals generated from the pacemaker cells are transmitted between cardiomyocytes and between different cells. It is inferred that a pulsating path is formed. Therefore, it is also required to detect not only the beating area but also the beating center from within the colony of cardiomyocytes in order to elucidate the mechanism of beating of cardiomyocytes.

  Therefore, as an applied method of the pulsation detection method exemplified in the above embodiment, the pulsation center detection method will be described below with additional reference to FIG. 8 to FIG. 10 and Table 1 of FIG. Here, FIG. 8 is a schematic view for explaining the flow of a beat center detection method, FIG. 9 is a schematic view for explaining a process of calculating an attention point and a small area window, and FIG. 10 is a beat in a colony. It is a schematic diagram which shows a center area | region.

  First, a motion vector is detected from a time-series image (time t = 1, 2, 3,..., T-1, T,...) Of the colony C of cardiomyocytes MC. Capture time t-1 to time t. The first image at time t-1 and the second image at the next time t are acquired from the time-series image, and the outermost contour of colony C is extracted from the first image at time t-1 among them to perform segmentation And set the internal region I of the colony C. Such outermost contour extraction processing includes, for example, a method of performing binarization after applying a dispersion filter, a dynamic contour method (Snakes or Level Set method), and the like.

  Then, a motion vector between the first image and the second image is detected using a known method such as the optical flow method, and among the detected motion vectors, a motion vector group in the contraction direction (symbol “in FIG. Extract only VG). Here, the motion vector group in the contraction direction refers to a collection of a plurality of vectors oriented so as to converge to a certain point (center) in the image.

  Then, with a certain pixel G (m) in which a motion vector in the contraction direction is detected as a target pixel, a small area window (block of plural pixels) centered on the target point P (0) or this target point P (0) If W (0) is taken into consideration, the motion vector V (P0) at the position P (0) is detected when the attention point P is adopted, and in the case of the small area window W, this block W (0) The average vector V (W0) of is detected. The attention point P (0) is moved according to the direction and the size of the motion vector V (P0) or the average vector V (W0) to set a new attention point P (1).

Similarly, a motion vector V (P1) or an average vector V (W1) is detected for a small area window W (1) centered on the point of interest P (1) or the point of interest P (1). The process is repeated until the end point Pend where movement of the point P (n) disappears. This is performed for all the pixels G (m) in which the motion vector in the contraction direction is detected in the inner area I, and only the end point Pend when there is movement of the attention point P (n) is extracted. As shown in FIG. 10, a collection area of this end point Pend (an area surrounded by a dotted line in the figure) and a pulsation detection area detected by the above-described pulsation detection method (an area shaded in the figure) The beat center area of the colony C can be detected by calculating the overlapping area at and.

Here, in a region derived from a set of end points Pend, there may be cases where it is only a dwelling point due to movement (cells are not beating), but the aggregation area of end points Pend and the aforementioned beat detection It is possible to estimate a detailed beat center (contraction center) by combining the beat detection area derived by the method. In the above description, the beat center area is estimated based on the motion vector in the contraction direction, paying attention to contraction of colony C, but the same is true if the direction of the motion vector is reversed even when colony C is relaxed. An effect is obtained.

  Further, for example, as is apparent from FIGS. 8 and 10, when the direction of the motion vector (direction of change of the position of each part) is detected, the positions (centers of the motion vectors) to which the plurality of motion vectors are Since it becomes the center, it is also possible to obtain the center of the pulsation region directly from the portion where the directions of the plurality of motion vectors overlap (that is, the center of the motion vector). In this case, it is also possible to estimate the beat center without specifying the beat region. When colony C is relaxed, the direction of the vector is opposite to that in FIGS. 8 and 10, and the motion vector is directed in the direction away from the center of pulsation, but the length of the motion vector is similarly extended and the overlapping region Is the heart of the beat.

  Subsequently, an image processing method of a beat center detection method executed in the image processing apparatus 100 will be described with reference to FIG. Here, FIG. 11 is a flowchart showing an outline of processing in the image processing program GP2.

  In step S201, the imaging device 55c captures a time-series image (t = 1, 2, 3,..., T-1, T,...) Of the colony C to be observed in the dish. This time-series image is stored in the image storage unit 110 of the image processing apparatus 100.

  In step S202, the image analysis unit 120 detects a contracted state of the colony C by calculating a motion vector from the time-series image using a known method such as an optical flow method, and the time at the time of this contraction = t. To detect

  In step S203, the image analysis unit 120 acquires the first image at time t-1 and the second image at the next time t from the image storage unit 110 based on the time t set in step S2. Note that the motion vector between the first image and the second image is already detected in the previous step S202, and the motion vector when colony C deforms from the relaxed state to the contracted state is allocated on the first image. .

  In step S204, the image analysis unit 120 applies a dispersion filter to the first image at time t and then performs binarization, the active contour method (Snakes or LebelSet method), or the like. Segmentation is performed to set an internal region I of colony C.

  In step S205, the motion vector detection unit 121 detects a motion vector group VG in the contraction direction from among the plurality of motion vectors detected in the internal region I of the colony C in the first image.

  In step S206, the motion vector detection unit 121 moves the pixel G (m) (pixel number m = 1, 2, 3,... M) in which the motion vector in the contraction direction is detected in the internal region I of the colony C. Extract as a vector detection point.

  In step S207, the image analysis unit 120 sets the initial value 0 to the number of repetitions n, sets the position of a certain pixel G (m) as the initial position P (0) of the attention point P (n), and A rectangular block-like small area window W (0) centered at the point P (n) is set.

  In step S208, the image analysis unit 120 performs arithmetic processing on the motion vector in the contraction direction detected in the small area window W (n) centered on the attention point P (n), and the small area window W (n) Calculate the average vector V (Wn) of

  In step S209, the image analysis unit 120 moves the focus point P (n) according to the average vector V (Wn) to set the next focus point P (n + 1), and sets the focus point P (n + 1) as the center. The small area window W (n + 1) to be set is set.

  In step S210, the image analysis unit 120 determines whether the next target point P (n + 1) has been moved from the original target point P (n), in other words, the next target point P (n + 1) It is determined whether or not the position is different from the original attention point P (n).

  In the case of affirmation determination in step S210 (Yes), that is, in the case where the next target point P (n + 1) has been moved from the original target point P (n), the number n of repetitions in the next step S211. Increment by one and shift to step S208. In the case of a negative determination (No) in step S210, that is, when the next target point P (n + 1) matches the original target point P (n), the movement of the target point P (n) is completed. In step S212, this attention point P (n) is set as the end point Pend.

In step S213, the image analysis unit 120 determines whether the process has been completed for all the pixels G (m) in the internal region I. In the case of a negative determination (No) in step S213, the pixel number m of the target pixel G (m) to be processed is incremented by one in step S214, and the process proceeds to step S207. In the case of an affirmative determination (Yes) in step S213, in step 15, an area where the assembly area of the end point Pend and the pulsation detection area detected in the above-described embodiment (beat detection method) are extracted Do.

  In step S216, the image analysis unit 120 determines the extraction region calculated in step S215 as the beating central region of colony C (that is, pacemaker cells are present in this region).

  As described above, according to the image processing program GP according to the present embodiment, an image processing method (motion detection method: pulsation detection method) configured by executing the image processing program GP, and the image processing apparatus 100. For example, in order to extract an area having motion from a time-series image and to discriminate a pulsation area based on a time-series change of the image feature amount of the motion area, false detection of foreign matter such as dust or floating cells etc. Instead, the beating motion of the cardiomyocytes can be accurately detected, and the beating motion of the cardiomyocytes can be quantitatively evaluated and judged.

  In addition, from the beating area of the colony detected in the beating detection method, the flow of the motion vector group in the contraction direction or the relaxation direction is used to detect the beating center, thereby investigating the mechanism of the beating movement. Can contribute to

(Application to regenerative medicine and drug discovery based on evaluation of beating motion of cardiomyocytes)
As in the above-described embodiment, by identifying the beating central cells (pacemaker cells) of the differentiated cardiomyocytes, the growth state of at least the specified cell colonies of beating central cells is evaluated from the beating cycle. Based on the results of this evaluation, culture conditions of cardiomyocyte cell colonies can be changed to optimize cardiomyocyte culture conditions. Specifically, the change of the culture condition of the cell colony of cardiomyocytes is compared with the beat cycle in a predetermined cycle, and the culture condition is optimized based on the comparison result. Further, as in the above-described embodiment, by specifying the beating central cells (pacemaker cells) of the differentiated cardiomyocytes, the drug is dropped onto the specified cell colony of at least beating central cells at a predetermined timing, and the cardiomyocytes are The effect of the drug can be evaluated from the pulsation cycle. Specifically, the pulsation cycle is compared with a predetermined cycle to evaluate the effect of the cardiomyocyte drug. In addition, in the method for producing a drug, the effect of the drug on cardiomyocytes is evaluated using the above-mentioned method for evaluating a drug, and the evaluation result is used for the blending ratio of the components of the drug.

(Application to pickup of cardiomyocyte colony)
Furthermore, as in the above-described embodiment, by identifying differentiated beating center cells (pacemaker cell cardiomyocytes), it is possible to produce cardiomyocyte colonies by picking up only around the beating center cells (pickup). As a result, the quality of cardiomyocyte colonies can be improved and the efficiency of shipping can be increased. The image processing of cutting out only the periphery of the beating central cell is performed by image processing of contour extraction of cell colonies and the outermost contour extraction processing already described, by cutting out a predetermined width from the extracted contour. The predetermined width is set to an amount sufficiently including the experimentally determined pacemaker cells. Thus, when the cells around the beating central cell are specified, the coordinate system (XYZ) on the stage on which the culture vessel is placed is calculated in the control unit 6 of FIG. Based on this coordinate system, beat center cells to be picked are identified and picked with a surrounding device automatically by a picking device (not shown). The needle tip of the pick-up device's cell pick-up needle is automatically driven by the CPU (control unit) in the picking device, but the coordinates (stage coordinate system) information of the identified cells in the culture vessel (dish) by this CPU Control the driving of the needle tip to the desired cell position by inputting. The picking device picks the identified cells in the culture vessel and transfers it to a new culture vessel. Also, the picking device can electrically drive the picking needle by the operation of the operator. In this case, the cell colony in the culture vessel and the tip of the picking needle are simultaneously displayed on the display panel 72 of FIG. 3 so that the operator can pick up a desired cell while looking at the screen.

  The image processing program GP, the image processing method (beat detection method), and the image processing apparatus 100 according to the present embodiment described above are not limited to the above embodiment, and the technical idea of the present invention Other forms contemplated within the scope are also included within the scope of the present invention.

  In the above-described embodiment, the configuration has been exemplified in which the time-lapse image (image data) captured by the imaging device in the culture observation system BS and stored in the RAM 63 is read to analyze the pulsating movement of cardiomyocytes. The present invention is not limited, and may be configured to analyze in real time the first and second images taken by the imaging device and analyze and display the pulsating movement at the present time, and other observation The pulsating movement may be analyzed by reading a time-series image captured in a system and recorded in a magnetic storage medium or the like or a time-series image transferred through a communication line. Further, the observer sets a predetermined range of the observation image (for example, a specific colony or a specific site in the colony) as an analysis range by a mouse etc., and the image processing apparatus analyzes the beating movement for the set analysis range. May be configured to perform.

  Further, although the above-described embodiment exemplifies a configuration in which a region having motion is extracted using a motion vector between two images in a time-series image, the present invention is not limited to this. The moving region may be extracted using other known methods such as tracking methods used for moving object tracking and the like, image difference processing, and the like.

  In the above embodiment, identification of a colony, identification of a movement region, identification of a pulsation region, identification of a pulsation center are all carried out, but it is not possible to estimate other regions unless all are carried out. However, any one of the identifications may be performed, or the identification of a plurality of types of regions may be performed in combination as appropriate.

Further, in the above-described embodiment, the configuration for calculating the feature (texture feature) based on the internal structure of cardiomyocytes as the image feature is exemplified, but the present invention is not limited to this. You may use the image feature-value based on the external shape of. The image feature amount based on the outer shape is exemplified by the width and length of the region, the area, the perimeter, the complexity of the shape (perimeter ² / area) and the like.

C colony V motion vector MC cardiomyocyte BS culture observation system GP image processing program 5 observation unit 6 control unit 54 macro observation system 54 c imaging device 55 microscopic observation system 55 c imaging device 61 CPU 62 ROM
63 RAM 100 image processing apparatus 120 image analysis unit 121 motion vector detection unit 122 feature amount calculation unit 130 output unit

Claims (17)

Obtaining a time-series image of cell colonies which are candidates for cardiomyocytes at predetermined time intervals;
Extracting a partial region in cell colonies of the acquired time-series image;
Detecting the direction of movement of the extracted area;
Detecting a retention area of movement of the cell colony based on the detected direction of movement;
A method for detecting movement of cardiomyocytes, including:   The method according to claim 1, wherein the partial area is a plurality of pixels around a focused pixel of an image.   The method according to claim 1 or 2, wherein the partial area is a moving area in the cell colony.   The method according to claim 3, wherein a motion vector or an average vector is detected in the partial area.   The partial area to be extracted is moved according to the direction or magnitude of the detected motion vector, and the partial area to be extracted is moved until the motion vector or the average vector is not detected. The cardiac muscle movement detection method according to The method for detecting motion of cardiomyocytes according to any one of claims 1 to 5 , wherein an overlapping region of a pulsation region of the cell colony and the retention region is detected as a central region of the pulsation region.   The method for detecting motion of cardiomyocytes according to claim 6, wherein cells at the center of pulsation are extracted from the central region of the pulsation region. Calculating the direction of movement of the extracted area based on morphological feature quantities of cells of the area extracted from the time-series image;
The method according to any one of claims 1 to 7, further comprising: detecting a beating region of the cell colony based on a time-series change of the morphological feature.   The method for detecting motion of cardiomyocytes according to claim 8, wherein the morphological characteristic amount utilizes information on the internal structure of a cell.   The cardiomyocyte movement according to any one of claims 1 to 9, wherein an edge area of the cell colony is extracted based on the image of the cell colony, and the partial area is extracted from the image of the edge area. Detection method. Detecting the motion of cardiomyocytes by the method of detecting motion of cardiomyocytes according to any one of claims 1 to 10;
Evaluating the growth status of the cell colony of cardiomyocytes whose motion is detected;
Changing the culture conditions of the cell colony based on the result of the evaluation.   The method for culturing cardiomyocytes according to claim 11, wherein the change of the culture condition of the cell colony of the cardiomyocytes is performed based on the comparison result of the exercise cycle by the periodic exercise of the cardiomyocytes being compared with a predetermined cycle. . Detecting the motion of cardiomyocytes by the method of detecting motion of cardiomyocytes according to any one of claims 1 to 10;
Dropping a drug at predetermined timing to a cell colony of cardiomyocytes whose motion is to be detected;
Evaluating the effect of the dropped drug from cyclic motion of cardiomyocytes.   The drug evaluation method according to claim 13, wherein the effect of the drug is evaluated by comparing an exercise cycle by periodic exercise of cardiomyocytes with a predetermined cycle.   The drug evaluation method according to claim 13, wherein an effect of the drug is evaluated, and a component of the drug is adjusted based on a result of the evaluation. An image processing program for causing a computer to function as an image processing apparatus readable by a computer and acquiring an image captured by an imaging device and processing the image,
Acquiring a time-series image of cell colonies which are candidates for cardiomyocytes by a predetermined time separated by the imaging device;
Extracting a partial region of cell colonies of the acquired time-series image by the image processing apparatus;
Detecting the direction of movement of the extracted area by the image processing device;
Detecting, by the image processing device, a stagnant area of movement of the cell colony based on the detected direction of movement;
An image processing program that causes the computer to execute; The image processing apparatus includes an image analysis unit that acquires a time-series image in which cell colonies that are candidates for cardiomyocytes are photographed at predetermined intervals by an imaging device, and analyzes the image.
The image analysis unit
A partial region extraction unit that extracts a partial region of cell colonies in the acquired time-series image;
A motion direction detection unit that detects the direction of motion of the extracted area;
A central area detection unit that detects a stagnation area of the movement of the cell colony based on the detected direction of movement;
An image processing apparatus comprising: