JP2014085949A - Cell behaviour analysis device, cell behaviour analysis method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a cell behavior analysis device or the like capable of facilitating a highly accurate automatic tracking by classifying a detected cell candidate area into a clear area and a non-clear area and executing an automatic tracking for the clear area high in cell detection accuracy based on a calculated feature quantity even under high density condition in which a number of non-clear areas of the cell are available.SOLUTION: A cell behavior analysis device 1 extracts a cell candidate area using a binarized image of a time lapse image which is applied with cell image processing (S1), and executes feature quantity calculation processing for each cell candidate area (S2). The cell behavior analysis device 1 calculates an area size, a luminance value distribution, and a curvature distribution of the cell candidate area, and prepares a high dimensional feature vector based on the calculated feature quantity (S3). The cell behavior analysis device 1 extracts a cell area among the cell candidate areas based on the feature vector (S5). The cell behavior analysis device 1 executes automatic tracking processing for the extracted cell area.

Description

  The present invention relates to a cell behavior analysis apparatus that analyzes the behavior of a cell based on an image obtained by photographing the cell.

  2. Description of the Related Art In recent years, in the field of biology and medicine, there has been a technology for analyzing cell behavior by a computer based on an image obtained by photographing cells included in a predetermined area at regular time intervals (hereinafter referred to as “time-lapse image”). Has been developed. In such a field, there is a demand to automatically track individual cells in order to analyze cell shape, cell moving speed, and the like.

  In Non-Patent Document 1, cell detection is performed for each frame using a phase difference image, and cells are automatically tracked by associating cells detected between frames in which time series are continuous, thereby detecting cells. Techniques for detecting cell death based on the results have been proposed. Non-Patent Document 2 proposes a method of detecting cell division, cell overlap, and the like using the result of automatic tracking of individual cells.

O. Al-Kofahi, R. Radke, S. Goderie, Q. Shen, S. Temple, and B. Roysam, “Automated cell lineage construction: A rapid method to analyze clonaldevelopment established with murine neural progenitor cells,” Cell Cyclevol. 5, no. 3, pp. 327-335, 2006. Takeo Kanade, Zhaozheng Yin, Ryoma Bise, Seungil Huh, Sungeun Eom, Michael Sandbothe and Mei Chen, “Cell ImageAnalysis: Algorithms, System and Applications,” IEEE WACV2011.

By the way, under the condition where the cell density is high, in the time-lapse image at the same time, there are a region where the outline of the cell is relatively clear and a region where the cells are stuck together and the boundary is ambiguous. For this reason, the accuracy of cell detection in a region where the outline of the cell is unclear is extremely poor, and the possibility that the automatic tracking result includes error data increases. Here, the data of the automatic tracking result is error data. For example, the cell correspondence between frames is mistakenly included, and the movement trajectory of a different cell is included in the middle, or the target cell is lost. The movement trajectory is included only halfway. If such error data is included in the population and a statistic is calculated to analyze the behavior of the cell, a correct statistic cannot be obtained and the accuracy of the automatic tracking result decreases.
In particular, in the field of regenerative medicine, mature cell cells for transplantation are cultured, so that a cell behavior analysis apparatus that can automatically track individual cells with high accuracy in non-invasive (non-stained) images under conditions of high cell density. Etc. are desired.

  In the technique described in Non-Patent Document 1, the accuracy of cell detection is extremely low under high density conditions where there are many regions where the outline of the cell is ambiguous, and automatic tracking with high accuracy is difficult. Although the technique described in Non-Patent Document 2 can detect the overlap of cells, it can cope with only about 2 to 3 cells, so that there remains a problem in automatic tracking of individual cells under high-density conditions.

  The present invention has been made in view of the above-described problems. The object of the present invention is to detect a cell candidate detected based on the calculated feature amount even under a high-density condition where there are many unclear areas of cells. It is intended to provide a cell behavior analysis device and the like that can perform automatic tracking with high accuracy by classifying a region into a clear region and a blurred region and performing automatic tracking on a clear region with high cell detection accuracy. .

In order to solve the above-described problem, the first invention is directed to a cell candidate region detection unit that detects a cell candidate region of each cell in the time lapse image for each frame of the time lapse image, and to the detected cell candidate region. A feature amount calculating means for calculating the feature amount, a cell region extracting means for extracting a cell region from the cell candidate region based on the calculated feature amount, and extraction for two consecutive frames in time series A likelihood calculating means for calculating a likelihood of association between the cell regions; an optimal solution specifying means for specifying an optimal solution of the association combination based on the calculated likelihood; and the specified optimal solution And a storage means for storing tracking result data in which position information of the cell region included in is stored in chronological order.
According to the first invention, the cell candidate region is detected for each frame of the time-lapse image, the feature amount is calculated for each detected cell candidate region cell, and the cell candidate region is contoured based on the feature amount. The area is classified into a clear area and an unclear area, and an area with a clear outline is extracted as a cell area. Automatic likelihood tracking is performed by calculating the likelihood that cell regions of frames with continuous time series are associated with each other, and obtaining an optimal solution for the combination of associations based on the likelihood. As a result, an area in which cell detection is functioning correctly is automatically tracked, and an error at the time of cell detection in an area where the outline is ambiguous (overdetection error in which an area where no cell exists is detected, cells exist) The influence of an undetected error in which no area is detected can be suppressed. Therefore, it is possible to automatically track individual cells with high accuracy even under high-density conditions where there are many regions where the cell outline is unclear. In mature culture, the outline of the cell is ambiguous in the low-maturity region, and the outline of the cell becomes clear as the maturity level is high. Separating these regions leads to quality control. .

In addition, it is preferable that the feature amount calculation unit calculates the feature amount using an index related to the luminance distribution of the cell candidate region and an index related to the curvature distribution of the cell candidate region.
Thereby, the shape of each cell candidate region can be calculated as a feature amount, and the cell candidate region can be identified as a region where the outline of the cell is clear and a region where the cell outline is unclear.

Further, the cell region extraction means creates a discriminator for classifying the cell candidate region into a clear region and an unclear region from correct data that is a plurality of predetermined feature vectors, and for the cell candidate region The feature vector is created from the calculated feature quantity, and when the created feature vector is classified into the clear region by the classifier, the cell candidate region related to the feature vector is set as the cell region. Is desirable. The cell candidate area can be classified into a clear area and an unclear area by the machine learning technique.
The cell candidate area can be classified into a clear area and an unclear area by the machine learning technique.

According to a second aspect of the present invention, for each frame of the time-lapse image, a cell candidate region detection step for detecting a cell candidate region of each cell in the time-lapse image, and a feature amount is calculated for the detected cell candidate region. A step of calculating a quantity, a step of extracting a cell area from the candidate cell areas based on the calculated feature quantity, and a correspondence between the cell areas extracted for two consecutive frames in time series A likelihood calculating step of calculating a likelihood of attaching, an optimal solution specifying step of specifying an optimal solution of the association combination based on the calculated likelihood, and a cell region included in the specified optimal solution And a storage step of storing tracking result data in which position information is stored in time series order.
According to the second invention, the cell candidate region is detected for each frame of the time-lapse image, the feature amount is calculated for each detected cell candidate region cell, and the cell candidate region is contoured based on the feature amount. The area is classified into a clear area and an unclear area, and an area with a clear outline is extracted as a cell area. Automatic likelihood tracking is performed by calculating the likelihood that cell regions of frames with continuous time series are associated with each other, and obtaining an optimal solution for the combination of associations based on the likelihood. As a result, an area in which cell detection is functioning correctly is automatically tracked, and an error at the time of cell detection in an area where the outline is ambiguous (overdetection error in which an area where no cell exists is detected, cells exist) The influence of an undetected error in which no area is detected can be suppressed. Therefore, it is possible to automatically track individual cells with high accuracy even under high-density conditions where there are many regions where the cell outline is unclear. In mature culture, the outline of the cell is ambiguous in areas where the maturity is low, and the outline of the cell becomes clear as the maturity becomes high. Separating these areas leads to quality control. .

According to a third aspect of the present invention, in a computer, a cell candidate region detection step of detecting a cell candidate region of each cell in the time lapse image for each frame of the time lapse image, and a feature amount for the detected cell candidate region. A feature amount calculating step for calculating, a cell region extracting step for extracting a cell region from the cell candidate regions based on the calculated feature amount, and the cell region extracted for two consecutive frames in time series A likelihood calculating step for calculating a likelihood of association between each other, an optimal solution specifying step for specifying an optimal solution of the combination set of the association based on the calculated likelihood, and the specified optimal solution A storage step of storing tracking result data in which position information of the cell region is stored in chronological order.
According to the third invention, the cell candidate region is detected for each frame of the time-lapse image, the feature amount is calculated for each detected cell candidate region cell, and the cell candidate region is outlined based on the feature amount. The area is classified into a clear area and an unclear area, and an area with a clear outline is extracted as a cell area. Automatic likelihood tracking is performed by calculating the likelihood that cell regions of frames with continuous time series are associated with each other, and obtaining an optimal solution for the combination of associations based on the likelihood. As a result, an area in which cell detection is functioning correctly is automatically tracked, and an error at the time of cell detection in an area where the outline is ambiguous (overdetection error in which an area where no cell exists is detected, cells exist) The influence of an undetected error in which no area is detected can be suppressed. Therefore, it is possible to automatically track individual cells with high accuracy even under high-density conditions where there are many regions where the cell outline is unclear. In mature culture, the outline of the cell is ambiguous in areas where the maturity is low, and the outline of the cell becomes clear as the maturity becomes high. Separating these areas leads to quality control. .

  According to the present invention, even under high density conditions where there are many unclear areas of cells, the detected cell candidate areas are classified into clear areas and unclear areas based on the calculated feature amount, and the clear areas with high cell detection accuracy are obtained. By performing automatic tracking on the cell behavior analyzing apparatus, it is possible to provide a cell behavior analysis device and the like that can perform automatic tracking with high accuracy.

Diagram explaining maturity in mature culture Diagram showing the hardware configuration of the cell behavior analyzer Flow chart showing the flow of cell candidate area identification processing An image showing an example of a cell collection region with a sharp outline An image showing an example of a cell collection region with unclear outlines The figure which shows an example of curvature distribution and luminance distribution of a cell candidate area with a clear outline The figure which shows an example of the curvature distribution and brightness | luminance distribution of a cell candidate area | region with a blurred outline Flow chart showing the flow of feature amount calculation processing Image showing an example of an image that has undergone cell image processing Diagram showing successful cell region extraction Diagram showing an example of cell region extraction failure Flow chart showing the flow of automatic tracking processing The figure explaining the parameter | index used for likelihood calculation processing The figure explaining the combination set of the correspondence of the cell areas between frames

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a diagram for explaining the maturity in mature culture. As shown in FIG. 1, the region with high maturity has high cell density and high cell detection accuracy because cells tend to exist independently. On the other hand, in a region where the maturity is low, cells tend to exist in an overlapping manner, and the cell detection accuracy is low because the outline is ambiguous and the cells are attached and detected.
In the present embodiment, the cell behavior analysis device performs a cell candidate region identification process described later to identify a region that exists independently from each other and a region in which the cell outline is ambiguous, and further, the cell is independent. An example in which an automatic tracking process is executed only on an existing area will be described.

  FIG. 2 is a hardware configuration diagram of a computer that realizes the cell behavior analysis apparatus 1 according to the embodiment of the present invention. As shown in FIG. 2, the computer includes a control unit 11, a storage unit 12, a media input / output unit 13, a communication control unit 14, an input unit 15, a display unit 16, a peripheral device I / F unit 17, and the like. 18 is connected.

The control unit 11 includes a CPU (Central Processing Unit) and a ROM (Read Only).
Memory), RAM (Random Access Memory) and the like.
The CPU calls and executes a program stored in the storage unit 12, ROM, storage medium, or the like to a work memory area on the RAM, drives and controls each device connected via the bus 18, and the cell behavior analysis device 1 The process to be described later is realized. The ROM is a non-volatile memory and permanently holds a computer boot program, a program such as BIOS, data, and the like. The RAM is a volatile memory, and temporarily holds a loaded program and data, and includes a work area used by the control unit 11 to perform each process.

  The storage unit 12 is an HDD (Hard Disk Drive) or the like, and stores a program executed by the control unit 11, data necessary for program execution, an OS (Operating System), and the like. These program codes are read by the control unit 11 as necessary, transferred to the RAM, and read and executed by the CPU.

The media input / output unit 13 is a media input / output device such as an HD (Hard Disk) drive, a floppy (registered trademark) disk drive, a PD drive, a CD drive, a DVD drive, or an MO drive, and performs data input / output. .
The communication control unit 14 includes a communication control device, a communication port, and the like, and is a communication interface that mediates communication between a computer and a network, and performs communication control between other devices via the network.

The input unit 15 performs data input and includes, for example, a keyboard, a pointing device such as a mouse, and an input device such as a numeric keypad. The input data is output to the control unit 11.
The display unit 16 includes, for example, a display device such as a CRT monitor or a liquid crystal panel, and a logic circuit (a video adapter or the like) for executing display processing in cooperation with the display device, and is input under the control of the control unit 11. The displayed display information is displayed on the display device.
The peripheral device I / F unit (interface) 17 is a port for connecting a peripheral device to the computer, and the computer transmits and receives data to and from the peripheral device via the peripheral device I / F unit 17. The peripheral device I / F unit 17 is configured by USB, IEEE 1394, RS-232C, or the like, and usually includes a plurality of peripheral devices I / F. The connection form with the peripheral device may be wired or wireless.
The bus 18 is a path that mediates transmission / reception of control signals, data signals, and the like between the devices.

  Below, the cell candidate area | region identification process which the cell behavior analysis apparatus 1 performs is demonstrated, referring FIGS. 3-11.

FIG. 3 is a flowchart showing a flow of a cell candidate region identification process executed by the cell behavior analysis apparatus 1.
When a time-lapse image is input, the control unit 11 of the cell behavior analysis apparatus 1 assigns a frame ID for uniquely identifying a frame of the time-lapse image, executes automatic tracking processing, and stores automatic tracking data of cells. The tracking result data to be generated is generated. The control unit 11 assigns frame IDs in chronological order each time a time-lapse image is input, executes automatic tracking processing, and updates tracking result data. The tracking result data is uniquely identified by the cell region ID assigned to the tracking target cell. In the tracking result data, the frame ID and cell information in the frame related to the frame ID are stored in time series. Here, the cell information is data including a cell region ID, a position coordinate group indicating a boundary line of the cell region, a luminance value group corresponding to each pixel in the cell region, and the like.
The tracking result data up to the frame ID “t−1” is generated in advance by the control unit 11 of the cell behavior analysis apparatus 1 and stored in the storage unit 12. The cell candidate region identification process shown in FIG. 3 is a process executed when a time-lapse image is input to the cell behavior analysis apparatus 1 and the frame ID “t” is given.

  As illustrated in FIG. 3, the control unit 11 of the cell behavior analysis device 1 detects a cell candidate region in the time-lapse image with the frame ID “t” (step S <b> 1).

  The control unit 11 of the cell behavior analysis apparatus 1 performs cell image processing on the time-lapse image associated with the frame ID “t”. Cell image processing is image processing in which the luminance of the cell region is high and the luminance of the contour region is low.For example, illuminance unevenness removal and preconditioning processing are used, but a known technique is used. Description is omitted because it is sufficient. Hereinafter, an image subjected to cell image processing is referred to as a precondition image.

  FIG. 4 is an example of a precondition image displaying a cell collection region with a clear outline. FIG. 5 is an example of a precondition image that displays a cell collection region with an unclear outline. The cell behavior analysis apparatus 1 distinguishes between a cell collection region with a clear outline and a cell collection region with a clear outline using a feature vector calculated by a feature amount calculation process described later.

  Subsequently, the control unit 11 of the cell behavior analysis apparatus 1 sets a threshold value in the range of luminance values (0 to 255) of the precondition image. The control unit 11 generates a binarized image in which a pixel that exceeds the threshold is 255 and a pixel that does not exceed the threshold is 0 with respect to the threshold. The control unit 11 extracts and labels the connected components from the generated binary image. Each labeled connected component is set as a cell candidate region, and a boundary between the connected components is set as an outline of the cell candidate region.

  Returning to the description of FIG. The control unit 11 of the cell behavior analysis apparatus 1 performs a feature amount calculation process on the cell candidate region detected in step S1 (step S2). When N cell candidate regions are detected in step S1, N feature amount calculation processes are executed for the N cell candidate regions.

  FIG. 6A shows an example of a binarized image of a cell candidate region with a clear outline. As shown in FIG. 6 (a), a cell candidate region with a sharp outline has a tendency for cells to form one independent region, the brightness of the pixels constituting the cell candidate region is high, and the concave point of the contour is There are features on the image such as few.

  FIG. 7A shows an example of a binarized image of a cell candidate region with an unclear outline. As shown in FIG. 7A, a cell candidate region with a sharp outline tends to be detected in a state where the cell is not broken and is attached to another cell, and the luminance of the pixels constituting the cell candidate region There are features on the image such as low and a large number of concave points on the contour.

FIG. 8 is a flowchart showing the flow of the feature amount calculation process executed by the control unit 11 of the cell behavior analysis apparatus 1 in step S2 of FIG.
The control unit 11 of the cell behavior analysis apparatus 1 calculates the region size of the cell candidate region (step S11). The region size is the number of pixels included in the outline of the cell candidate region.

  Next, the control unit 11 of the cell behavior analysis apparatus 1 calculates a luminance value distribution within the outline of the cell candidate region using the precondition image in which the outline of the cell candidate region is arranged (step S12).

  Specifically, the control unit 11 of the cell behavior analysis device 1 sets a plurality of threshold values in the luminance value range for pixels in the outline of the cell candidate region in the precondition image in which the outline of the cell candidate region is arranged. Set and count the number of pixels within each threshold range. The sum of the number of pixels corresponding to each threshold matches the area size calculated in step S11. The control unit 11 divides the number of pixels for each threshold by the region size. As a result, the sum of the values corresponding to the respective thresholds becomes 1.

  FIG. 6B is an example of a luminance value distribution of a cell candidate region with a clear outline. The horizontal axis represents the luminance value, and the vertical axis represents the normalized number of pixels. In the example shown in FIG. 6B, 20 threshold values are set, and the value obtained by dividing the luminance value by 20 in the range of 0 to 255 is the width of each of the 20 threshold values. FIG. 7B is an example of a luminance value distribution of a cell candidate region with a blurred outline. It is created in the same manner as in FIG. Comparing the luminance value distributions of FIG. 6B and FIG. 7B, it can be seen that the cell candidate region having a clear outline has a higher luminance value as a whole.

  Returning to the description of FIG. Next, the control unit 11 of the cell behavior analysis apparatus 1 calculates the curvature distribution of the contour of the cell candidate region (step S13).

Specifically, the control unit 11 of the cell behavior analysis apparatus 1 traces each point on the outline of the cell candidate region. As a technique for tracing each point, for example, the technique described in Non-Patent Document 3 is used. The control unit 11 can obtain the position information of each point (each pixel) on the contour by tracing the contour of the cell candidate region clockwise.
<Non-Patent Document 3> “bwboundaries”, [online], MathWork, [searched July 13, 2012], Internet
<http://www.mathworks.co.jp/help/en_US/toolbox/images/ref/bwboundaries.html>

Subsequently, the control unit 11 of the cell behavior analysis apparatus 1 calculates the curvature of each point on the outline of the cell candidate region. As a method for calculating the curvature of each point, for example, the angle of two vectors obtained by skipping 5 pixels from both adjacent points constituting the contour with respect to the target point is calculated. Specifically, the technique described in Non-Patent Document 4 is used. The curvature of each point on the contour indicates a concave point when the sign is negative, and is calculated in the range of −1 to +1.
<Non-Patent Document 4> Ryoma Bise, Kang
Li, Sungeun Eom, Takeo Kanade, “Reliably Tracking
Partially Overlapping Neural Stem Cells in DIC Microscopy Image Sequences ”, MICCAI Workshop on Optical Tissue Image analysis in Microscopy
Histopathology and Endoscopy Imperial College London September 24 ^th
2009.

  The control unit 11 of the cell behavior analysis apparatus 1 sets a plurality of threshold values in the curvature range, and counts the number of pixels within each threshold range. The sum of the number of pixels corresponding to each threshold matches the number of pixels on the outline of the cell candidate region. The control unit 11 divides the number of pixels for each threshold by the number of pixels on the contour. As a result, the sum of the values corresponding to the respective thresholds becomes 1.

  FIG. 6C is an example of the curvature distribution of a cell candidate region with a clear outline. The horizontal axis represents the curvature, and the vertical axis represents the normalized number of pixels. In the example shown in FIG. 6C, 20 threshold values are set, and the value obtained by dividing the curvature into 20 equal parts in the range of −1 to +1 is the width of each of the 20 threshold values. FIG. 7C is an example of a curvature distribution of a cell candidate region with a blurred outline. It is created in the same manner as in FIG. Comparing the curvature distributions of FIG. 6C and FIG. 7C, it can be seen that the cell candidate region with unclear outline has more concave points as a whole.

  As described above, the control unit 11 of the cell behavior analysis apparatus 1 performs the feature amount calculation process on the cell candidate region.

  Subsequently, the description returns to FIG. The control unit 11 of the cell behavior analysis apparatus 1 creates a feature vector based on the feature amount of the cell candidate region (step S3).

  The control unit 11 calculates a feature amount that combines the region size calculated in step S11 of FIG. 8, the luminance value distribution calculated in step S12 of FIG. 8, and the curvature distribution calculated in step 13 of FIG. Create a high-dimensional feature vector to represent. In the example shown in FIGS. 6 and 7, a feature vector in a 41-dimensional vector space is created by combining the region size (one dimension), the luminance value distribution (20 dimensions), and the curvature distribution (20 dimensions). The control unit 11 creates a feature vector corresponding to the cell candidate region ID and stores it in the storage unit 12.

  Returning to the description of FIG. The control unit 11 of the cell behavior analysis apparatus 1 confirms whether or not the feature amount calculation processing has been executed for all the cell candidate regions detected in Step S1 (Step S4). If not executed (No in step S4), the control unit 11 repeats from step S2. When it has been executed (Yes in step S4), the control unit 11 extracts a cell region from the cell candidate regions based on the feature vector (step S8).

The control unit 11 uses the correct data stored in advance in the storage unit 12 to identify the cell candidate area as an unclear area and a clear area. The correct answer data is data including a feature vector and a label, and “0” is stored in the label if the feature vector represents the feature of the unclear region, and “1” if the feature vector represents the feature of the clear region. Is stored.
Specifically, the control unit 11 creates a discriminator (classifier) by a machine learning technique using a plurality of correct answer data. Then, the control unit 11 inputs the feature vector of the cell candidate region to the discriminator, and when the feature vector is classified into the class “0”, the feature vector is labeled “0” and “1”. If it is classified into this class, it is labeled “1”.
A known technique can be used for machine learning, and examples thereof include a neural network, a support vector machine, and Adaboost. For example, in the support vector machine, the feature vector of the cell candidate region can be classified into the unclear region and the clear region by configuring a hyperplane for classifying the correct answer data in the feature space including the feature vector.

  The feature vector is 41 dimensions including the region size, the luminance value distribution, and the curvature distribution, but the feature vector may be 40 dimensions. Although the number of divisions of the luminance value distribution and the curvature distribution is 20 here, the number of dimensions of the feature quantity may be changed by dividing the luminance value distribution and the curvature distribution by other values.

  The control unit 11 extracts the cell candidate region of the feature vector labeled “1” as a cell region, labels the extracted cell region with a cell region ID, and stores the cell region ID in the storage unit 12. To do.

  FIG. 9 is an example of a precondition image showing one frame of a time-lapse image. A region where the outline of the cell is relatively clear and a region where the cells stick to each other and the boundary is unclear are mixed. The cell candidate area identification process shown in FIG. 3 was performed on the precondition image shown in FIG. 9 to extract the cell area. FIG. 10 is a binarized image showing a successful example of cell region extraction. The connected portion indicates the extracted cell region. It can be seen that a plurality of independent cell regions are extracted. FIG. 11 is a binarized image showing a failure example of cell region extraction. As for the region where the cell outline is unclear, it can be seen that a plurality of cells are attached to each other and the cell region is extracted.

  As described above, the control unit 11 of the cell behavior analysis apparatus 1 extracts cell candidate regions, calculates the region size, luminance value distribution, and curvature distribution of each cell candidate region, creates a feature vector, and features A cell region is extracted from the cell candidate region based on the vector. As a result, the automatic tracking process described later is executed only for the cell region in which the cells are correctly detected. Therefore, the influence on the automatic tracking process due to an error at the time of cell detection is reduced, and the accuracy of the automatic tracking result is increased.

  Below, the automatic tracking process which the cell behavior analysis apparatus 1 performs is demonstrated, referring FIGS. 12-14.

  FIG. 12 is a flowchart showing the flow of the automatic tracking process executed by the cell behavior analysis apparatus 1. The automatic tracking process shown in FIG. 12 is executed when the cell behavior analysis apparatus 1 executes the cell candidate area identification process shown in FIG. 3 in the frame corresponding to the frame ID “t” and the cell area is extracted. Process.

The control unit 11 of the cell behavior analysis apparatus 1 calculates a likelihood P of association between the cell region in the frame ID “t−1” and the cell region in the frame ID “t” (step 21). The control unit 11 uses the index I _D related to the movement of the cell area, the index I _S related to the shape similarity of the cell area, and the index I _H related to the luminance histogram similarity of the cell area between the cell areas between the frames. P is calculated.

For example, the control unit 11 may calculate the likelihood P by weighting each index, such as P = W _D × I _D + W _S × I _S + W _H × I _H. Here, the weight of _{W D} indicators _{I D,} weight _{W S} indicators _{I S, W H} is the weight of the index _{I H.} In this case, the value of each weight can be appropriately changed by the user.

  FIG. 13 is a diagram for explaining an index used in the likelihood calculation process. The cell region A is extracted in the frame with the frame ID “t−1”. The cell region B is extracted in the frame with the frame ID “t”. That is, in FIG. 13, the cell region A and the cell region B of different frames are shown in an overlapping manner.

Examples of the index _ID related to the movement of the cell region include the distance between the center points. The center point is, for example, the barycentric coordinate of the cell region. Examples of the distance include Euclidean distance and exp (−α × Euclidean distance). Here, exp (•) is an exponential function, and α is a positive constant. The distance between the center points represents the movement of cells between frames. In the example of FIG. 13, for example, the Euclidean distance between the center point a and the center point b is the index _ID .

In addition, the index _ID relating to the movement of the cell region may be, for example, the rate of overlap of the cell regions. In the example of FIG. 13, the overlapping ratio of the cell areas is the ratio of the hatched area to the union of the cell area A and the cell area B. That is, the overlapping ratio of the cell regions is (number of pixels belonging to the cell region A and belonging to the cell region B) / (number of pixels belonging to the cell region A or belonging to the cell region B).

As an index _{I S} on the shape similarity cell region, e.g., SIFT (Scale Invariant Feature Transform) feature amounts, Fourier descriptors (Fourier Descriptors), HOG (Histograms of Oriented
Gradients) and the like. The SIFT feature value is a feature value that is invariant to scale, movement, and rotation. The Fourier descriptor is a feature amount of a shape that uses a Fourier coefficient obtained by converting a contour into a periodic function and expanding the periodic function into a Fourier series to describe the shape of the contour. The HOG feature value is a feature value for extracting a brightness gradient and brightness intensity from a local area of an image.

Examples of the index I _H related to the luminance histogram similarity of the cell region include the distance of the luminance histogram in each cell region. The distance may be, for example, a Bhattacharyya distance.

The various indicators described above are examples, and other indicators may be used in the present invention. Further, it is not necessary to use all of the index I _D regarding the movement of the cell region, the index I _S regarding the shape similarity of the cell region, and the index I _H regarding the luminance histogram similarity of the cell region, and any one or two of them may be used. good. The cell behavior analysis apparatus 1 displays, for example, an index selection screen (not shown) on the display unit 16 so that various indices can be used, and the user can select an appropriate index according to the purpose of the behavior analysis. You may do it.

  Returning to the description of FIG. The control unit 11 of the cell behavior analysis apparatus 1 specifies an optimal solution of the combination of the correspondence between the cell region in the frame ID “t−1” and the cell region in the frame ID “t” (step S22).

  FIG. 14 is a diagram for explaining a combination of cell region associations between frames. FIG. 14A illustrates an example of correspondence between cell regions between frames of the frame ID “t−1” and the frame ID “t”. FIG. 14 (b) illustrates a combination of cell areas corresponding to FIG. 14 (a).

  In FIG. 14A, cell regions A1, A2, and A3 are extracted in the frame associated with the frame ID “t−1”. Further, the cell regions B1, B2, and B3 are extracted in the frame related to the frame ID “t”. That is, in FIG. 14A, the cell regions A1, A2, and A3 of different frames and the cell regions B1, B2, and B3 are shown in an overlapping manner.

  In the embodiment of the present invention, the control unit 11 of the cell behavior analysis apparatus 1 specifies an optimal combination (= optimum solution) of association between cell regions using the likelihood calculated in step S21. . As the optimization processing, for example, it is conceivable to use integer programming. Note that the optimization process is not limited to integer programming.

  For example, the control unit 11 calculates an optimal solution using integer programming from among combinations (= hypotheses) of associations between all cell regions between frames. If there are N cell regions in the frame related to the frame ID “t−1”, N cell regions are also extracted from the frame related to the frame ID “t”. Each combination includes an association of N cell regions. For example, the control unit 11 specifies the combination that maximizes the sum of the likelihoods of association between N cell regions as the optimal solution.

  In the example shown in FIG. 14B, there are six combinations of h1 to h6 in associations between cell regions. The control unit 11 specifies an optimal combination of association from the six patterns h1 to h6 by integer programming. In the example shown in FIG. 14A, the optimum association combination is the combination h6 of (A1⇔B3, A2⇔B2, A3⇔B1).

  In addition, although it has been described that the combination set that maximizes the sum of the N likelihoods is specified as the optimal solution, other evaluation functions may be used.

  Returning to the description of FIG. The control unit 11 of the cell behavior analysis apparatus 1 updates the tracking result data based on the optimal solution specified in step S22 (step S23). The control unit 11 stores cell information such as cell position information (movement trajectory) of the cell region included in the optimal solution in the storage unit 12 and updates the tracking result data.

  Moreover, the control part 11 can calculate indices, such as a cell number and a cell shape, based on tracking result data, and can use it for evaluation of cell quality, evaluation of culture conditions, etc.

  The preferred embodiments of the cell behavior analysis apparatus and the like according to the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to such examples. It will be apparent to those skilled in the art that various changes or modifications can be conceived within the scope of the technical idea disclosed in the present application, and these naturally belong to the technical scope of the present invention. Understood

DESCRIPTION OF SYMBOLS 1 ......... Cell behavior analyzer 11 ......... Control part 12 ......... Storage part 13 ......... Media input / output part 14 ...... Communication control part 15 ......... Input part 16 ......... Display part 17 ... ... Peripheral equipment I / F section

Claims (5)

Cell candidate region detection means for detecting a cell candidate region of each individual cell in the time lapse image for each frame of the time lapse image;
Feature amount calculating means for calculating a feature amount for the detected cell candidate region;
A cell region extraction means for extracting a cell region from the cell candidate region based on the calculated feature amount;
A likelihood calculating means for calculating a likelihood of association between the cell regions extracted for two frames that are continuous in time series;
Optimal solution specifying means for specifying an optimal solution for the combination of associations based on the calculated likelihood;
Storage means for storing tracking result data in which position information of the cell region included in the identified optimal solution is stored in chronological order;
A cell behavior analysis device comprising: The feature amount calculating means includes:
The cell behavior analysis apparatus according to claim 1, wherein the feature amount is calculated using an index related to a luminance distribution of the cell candidate area and an index related to a curvature distribution of the cell candidate area. The cell region extraction means includes
Create a classifier that classifies the cell candidate area into a clear area and an unclear area from correct data that is a plurality of predetermined feature vectors,
Create the feature vector from the feature amount calculated for the cell candidate region,
3. The cell region according to claim 1, wherein, when the created feature vector is classified into the clear region by the classifier, the cell candidate region related to the feature vector is used as the cell region. 4. Cell behavior analyzer. A cell candidate region detection step for detecting a cell candidate region of each individual cell in the time lapse image for each frame of the time lapse image;
A feature amount calculating step for calculating a feature amount for the detected cell candidate region;
A cell region extraction step for extracting a cell region from the cell candidate regions based on the calculated feature amount;
A likelihood calculating step of calculating a likelihood of association between the cell regions extracted for two frames that are continuous in time series;
An optimal solution specifying step of specifying an optimal solution for the combination of associations based on the calculated likelihood;
A storage step of storing tracking result data in which position information of the cell region included in the identified optimal solution is stored in chronological order;
A cell behavior analysis method comprising: On the computer,
A cell candidate region detection step for detecting a cell candidate region of each individual cell in the time lapse image for each frame of the time lapse image;
A feature amount calculating step for calculating a feature amount for the detected cell candidate region;
A cell region extraction step for extracting a cell region from the cell candidate regions based on the calculated feature amount;
A likelihood calculating step of calculating a likelihood of association between the cell regions extracted for two frames that are continuous in time series;
An optimal solution specifying step of specifying an optimal solution of the combination set of the correspondence based on the calculated likelihood;
A storage step of storing tracking result data in which position information of the cell region included in the identified optimal solution is stored in chronological order;
A program for running