JP2014085950A - Cell behavior analysis device, cell behavior analysis method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a cell behavior analysis device or the like capable of facilitating a highly accurate automatic tracking of individual cells even under the high density condition of the cells in a noninvasive image.SOLUTION: A cell behavior analysis device 1 partitions a range of the luminance value of the time lapse image relating to a frame ID "t" applied with cell image processing by a plurality of threshold values, generates a plurality of binarized images, and detects a cell candidate area (S1). The cell behavior analysis device 1 extracts a hypothesis on the correspondence between the cell area relating to a frame ID "t-1" and the cell candidate area relating to the frame ID "t" (S2), and calculates a likelihood of the hypothesis P and each component of the constraint function C. The cell behavior analysis device 1 specifies the optimum hypothesis using the integer programming method (S8), and takes the cell candidate area included in the optimum hypothesis as the cell area relating to the frame ID "t", and updates the tracking result data stored in a storage part 12 (S9).

Description

  The present invention relates to a cell behavior analysis apparatus that analyzes the behavior of a cell based on an image obtained by photographing the cell.

  2. Description of the Related Art In recent years, in the field of biology and medicine, there has been a technology for analyzing cell behavior by a computer based on an image obtained by photographing cells included in a predetermined area at regular time intervals (hereinafter referred to as “time-lapse image”). Has been developed. In such a field, there is a demand to automatically track individual cells in order to analyze cell shape, cell moving speed, and the like.

  In Non-Patent Document 1, cell detection is performed for each frame using a phase difference image, and automatic tracking is performed by associating cells detected between frames in which time series are continuous, and cell detection results are obtained. Based on this, a technique for detecting cell death has been proposed. Non-Patent Document 2 proposes a method of detecting cell division, cell overlap, and the like using the result of automatic tracking of individual cells.

O.Al-Kofahi, R.Radke, S.Goderie, Q.Shen, S.Temple, and B.Roysam, “Automated cell lineage construction: A rapid method to analyze clonaldevelopment established with murine neural progenitor cells,” Cell Cycle vol .5, no.3, pp.327-335, 2006. Takeo Kanade, Zhaozheng Yin, Ryoma Bise, Seungil Huh, SungeunEom, Michael Sandbothe and Mei Chen, “Cell ImageAnalysis: Algorithms, System and Applications,” IEEE WACV2011.

By the way, under the condition where the cell density is high, the cells adhere to each other, and the cell boundary line becomes ambiguous. Therefore, the possibility that the automatic tracking result includes error data increases. Here, the data of the automatic tracking result is error data. For example, the cell correspondence between frames is mistakenly included, and the movement trajectory of a different cell is included in the middle, or the target cell is lost. The movement trajectory is included only halfway. If such error data is included in the population and a statistic is calculated to analyze the behavior of the cell, a correct statistic cannot be obtained and the accuracy of the automatic tracking result decreases.
In particular, in the field of regenerative medicine, mature cell cells for transplantation are cultured, so that a cell behavior analysis apparatus that can automatically track individual cells with high accuracy in non-invasive (non-stained) images under conditions of high cell density. Etc. are desired.

  In the technology described in Non-Patent Document 1, since cell detection and automatic tracking are performed independently, the accuracy of the automatic tracking result greatly affects the accuracy of cell detection. Therefore, under high density conditions, many errors are included in cell detection, and automatic tracking with high accuracy is difficult. Although the technique described in Non-Patent Document 2 can detect the overlap of cells, it can cope with only about 2 to 3 cells, so that there remains a problem in automatic tracking of individual cells under high-density conditions.

  The present invention has been made in view of the above-described problems, and the object of the present invention is a non-invasive image and a cell behavior capable of automatically tracking individual cells with high accuracy even under high-density conditions of cells. It is to provide an analysis device and the like.

In order to achieve the above-mentioned object, the first invention includes a lot of overdetection but almost no detection of cell candidate regions of individual cells in the timelapse image for each frame of the timelapse image. And a hypothesis extracting means for extracting a hypothesis of correspondence between the cell area of the previous frame and the cell candidate area of the detection target frame for the two consecutive frames in time series Likelihood calculating means for calculating the likelihood of the hypothesis extracted by the hypothesis extracting means, and hypothesis specifying means for specifying the optimum hypothesis based on the likelihood calculated by the likelihood calculating means; , A cell region specifying unit that uses the cell candidate region included in the hypothesis specified by the hypothesis specifying unit as the cell region, and a cell region specifying unit that has been specified by the cell region specifying unit. A cellular behavior analysis apparatus characterized by cell area information includes storage means for storing tracking result data stored in chronological order, the including position information of a cell area.
As a result, a cell region is determined using the tracking result data of the previous frame from among the cell candidate regions that are detected so that many overdetections are included but almost no detection is included. By executing the association between the cells of the detection target frame and the detection target frame, it is possible to reduce the influence of the cell detection accuracy on the result of automatic tracking of individual cells. Therefore, it is possible to automatically track individual cells with high accuracy even under high-density conditions where the cell outline is unclear.

In addition, the cell candidate region detection means sets a plurality of threshold values in a range of luminance values of the time-lapse image, generates a binarized image for each threshold for each frame, and generates the generated binarization It is desirable to extract a connected component of an image and set the extracted set of connected components as the cell candidate region.
Thereby, it becomes possible to detect cell candidate regions of individual cells in the time-lapse image so as to include many overdetections but hardly include undetected.

Further, the likelihood calculating means calculates the likelihood using an index related to the movement between the cell region in the previous frame and the cell candidate region in the detection target frame and an index related to the shape of the cell candidate region. Is desirable.
Thereby, the precision of likelihood increases by using the some parameter | index from which a viewpoint differs.

The hypothesis specifying means is a constraint matrix in which the number of the extracted hypotheses is the number of rows, and the sum of the number of the cell regions and the number of the cell candidate regions related to the hypothesis is the number of columns, For each hypothesis, the constraint matrix having the relationship between the hypothesis and the cell region, and the mutual relationship between the cell candidate regions related to the hypothesis, and the answer vector satisfy the constraint condition, It is desirable to identify the optimal hypothesis by solving an integer programming problem that maximizes the product of the transposed vector of the likelihood vector having the likelihood of the hypothesis as an element and the answer vector.
Thereby, an optimal hypothesis can be calculated efficiently.

Moreover, it is desirable to further comprise a cell behavior index calculating means for calculating a cell behavior index based on the tracking result data stored by the storage means.
Thereby, it is possible to calculate an index such as the number of cells and the shape of cells with high accuracy by using the result of automatic tracking of cells, and it can be used for evaluation of cell quality, evaluation of culture conditions, and the like.

According to a second aspect of the present invention, a cell candidate region detection is performed for detecting a cell candidate region of an individual cell in the time lapse image so as to include many overdetections but hardly include undetected for each frame of the time lapse image. A hypothesis extraction step for exhaustively extracting hypotheses of correspondence between the cell region of the previous frame and the cell candidate region of the detection target frame for the two frames that are continuous in time series, and the hypothesis extraction step A likelihood calculating step for calculating the likelihood of the hypothesis extracted by the above, a hypothesis specifying step for specifying the optimum hypothesis based on the likelihood calculated by the likelihood calculating step, and the hypothesis specifying step A cell region specifying step in which the cell candidate region included in the hypothesis specified by the cell region is the cell region, and the cell region specifying step Ri is a cellular behavior analysis method characterized in that it comprises a storage step of cell area information including the position information of the specified the cellular region stores the tracking result data stored in chronological order.
As a result, a cell region is determined using the tracking result data of the previous frame from among the cell candidate regions that are detected so that many overdetections are included but almost no detection is included. By executing the association between the cells of the detection target frame and the detection target frame, it is possible to reduce the influence of the cell detection accuracy on the result of automatic tracking of individual cells. Therefore, it is possible to automatically track individual cells with high accuracy even under high-density conditions where the cell outline is unclear.

According to a third aspect of the present invention, the computer detects a cell candidate region of each cell in the time-lapse image for each frame of the time-lapse image so that many overdetections are included but almost no detection is included. A candidate area detecting means, and a hypothesis extracting means for exhaustively extracting hypotheses of correspondence between the cell area of the previous frame and the cell candidate area of the detection target frame with respect to the two consecutive frames in time series, A likelihood calculating means for calculating the likelihood of the hypothesis extracted by the hypothesis extracting means; a hypothesis specifying means for specifying the optimal hypothesis based on the likelihood calculated by the likelihood calculating means; A cell region specifying unit that uses the cell candidate region included in the hypothesis specified by the hypothesis specifying unit as the cell region; and the cell region specifying unit specified by the cell region specifying unit. Is a program for functioning as a storage means for storing tracking result data cell region information including location information of the area is stored in chronological order.
As a result, a cell region is determined using the tracking result data of the previous frame from among the cell candidate regions that are detected so that many overdetections are included but almost no detection is included. By executing the association between the cells of the detection target frame and the detection target frame, it is possible to reduce the influence of the cell detection accuracy on the result of automatic tracking of individual cells. Therefore, it is possible to automatically track individual cells with high accuracy even under high-density conditions where the cell outline is unclear.

  According to the present invention, it is possible to provide a cell behavior analysis apparatus and the like that can automatically track individual cells with high accuracy even under high-density conditions of cells with non-invasive images.

Diagram showing the hardware configuration of the cell behavior analyzer Flow chart showing the flow of automatic tracking processing Image showing an example of an image that has undergone cell image processing An image obtained by binarizing the luminance value of the image of FIG. An image obtained by binarizing the luminance value of the image of FIG. An image obtained by binarizing the luminance value of the image of FIG. An image obtained by binarizing the luminance value of the image of FIG. The figure which shows an example of the set of the cell area | region of frame ID "t-1", and the cell candidate area | region of frame ID "t". The figure explaining the hypothesis extraction of matching of the cell area | region of frame ID "t-1", and the cell candidate area | region of frame ID "t". The figure explaining the parameter | index used for likelihood calculation processing The figure which shows an example of the likelihood P of a hypothesis, the constraint function C, and the answer vector x The figure which shows distribution of the cell size and cell movement speed in an Example The figure which shows 1 frame of the cell detection result at the time of the culture | cultivation start by the manual analysis of an Example The figure which shows 1 frame of the cell detection result at the time of the culture | cultivation start by the cell behavior analysis apparatus based on this invention of an Example The figure which shows 1 frame of the cell detection result of the cultured cell after 21-day progress by the manual analysis of an Example The figure which shows 1 frame of the cell detection result of the cultured cell after 21-day progress by the cell behavior analysis apparatus based on this invention of an Example.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a hardware configuration diagram of a computer that realizes a cell behavior analysis apparatus 1 according to an embodiment of the present invention. As shown in FIG. 1, the computer includes a control unit 11, a storage unit 12, a media input / output unit 13, a communication control unit 14, an input unit 15, a display unit 16, a peripheral device I / F unit 17, and the like. 18 is connected.

The control unit 11 includes a CPU (Central Processing Unit) and a ROM (Read Only).
Memory), RAM (Random Access Memory) and the like.
The CPU calls and executes a program stored in the storage unit 12, ROM, storage medium, or the like to a work memory area on the RAM, drives and controls each device connected via the bus 18, and the cell behavior analysis device 1 The process to be described later is realized. The ROM is a non-volatile memory and permanently holds a computer boot program, a program such as BIOS, data, and the like. The RAM is a volatile memory, and temporarily holds a loaded program and data, and includes a work area used by the control unit 11 to perform each process.

  The storage unit 12 is an HDD (Hard Disk Drive) or the like, and stores a program executed by the control unit 11, data necessary for program execution, an OS (Operating System), and the like. These program codes are read by the control unit 11 as necessary, transferred to the RAM, and read and executed by the CPU.

The media input / output unit 13 is a media input / output device such as an HD (Hard Disk) drive, a floppy (registered trademark) disk drive, a PD drive, a CD drive, a DVD drive, or an MO drive, and performs data input / output. .
The communication control unit 14 includes a communication control device, a communication port, and the like, and is a communication interface that mediates communication between a computer and a network, and performs communication control between other devices via the network.

The input unit 15 performs data input and includes, for example, a keyboard, a pointing device such as a mouse, and an input device such as a numeric keypad. The input data is output to the control unit 11.
The display unit 16 includes, for example, a display device such as a CRT monitor or a liquid crystal panel, and a logic circuit (a video adapter or the like) for executing display processing in cooperation with the display device, and is input under the control of the control unit 11. The displayed display information is displayed on the display device.
The peripheral device I / F unit (interface) 17 is a port for connecting a peripheral device to the computer, and the computer transmits and receives data to and from the peripheral device via the peripheral device I / F unit 17. The peripheral device I / F unit 17 is configured by USB, IEEE 1394, RS-232C, or the like, and usually includes a plurality of peripheral devices I / F. The connection form with the peripheral device may be wired or wireless.
The bus 18 is a path that mediates transmission / reception of control signals, data signals, and the like between the devices.

  Next, the automatic tracking process executed by the cell behavior analysis apparatus 1 will be described with reference to FIGS.

FIG. 2 is a flowchart showing the flow of the automatic tracking process executed by the cell behavior analysis apparatus 1.
When a time-lapse image is input, the control unit 11 of the cell behavior analysis apparatus 1 assigns a frame ID for uniquely identifying a frame of the time-lapse image, executes automatic tracking processing, and stores automatic tracking data of cells. The tracking result data to be generated is generated. The control unit 11 assigns frame IDs in chronological order each time a time-lapse image is input, executes automatic tracking processing, and updates tracking result data. The tracking result data is uniquely identified by the cell region ID assigned to the tracking target cell. In the tracking result data, the frame ID and cell information in the frame related to the frame ID are stored in time series. Here, the cell information is data including a cell region ID, a region number ID, a position coordinate group indicating the boundary of the cell region, a luminance value group corresponding to each pixel in the cell region, and the like. The area number ID stores “1” when a cell area ID is newly assigned, and is updated to “2” when the same cell area ID as that of the previous frame is assigned.
The tracking result data up to the frame ID “t−1” is generated in advance by the control unit 11 of the cell behavior analysis apparatus 1 and stored in the storage unit 12. The automatic tracking process shown in FIG. 2 is a process executed when a time-lapse image is input to the cell behavior analysis apparatus 1 and the frame ID “t” is given.

As illustrated in FIG. 2, the control unit 11 of the cell behavior analysis device 1 detects a cell candidate region in the time-lapse image with the frame ID “t” (step S <b> 1).
The detection of the cell candidate region will be described with reference to FIGS.

  The control unit 11 of the cell behavior analysis apparatus 1 performs cell image processing on the time-lapse image associated with the frame ID “t”. Cell image processing is image processing in which the luminance of the cell region is high and the luminance of the contour region is low.For example, illuminance unevenness removal and preconditioning processing are used, but a known technique is used. Description is omitted because it is sufficient. FIG. 3 is an example of an image obtained by performing uneven illuminance removal and preconditioning processing on a time-lapse image (in the example of FIG. 3, a photomicrograph of a predetermined region of cultured cells). .

  Subsequently, the control unit 11 of the cell behavior analysis apparatus 1 seems to include many overdetections but hardly include undetected from the images subjected to cell image processing (hereinafter referred to as precondition images). A cell candidate region is detected. Here, overdetection is an error that detects an area that is not a cell, and undetected is an error that an area that is a cell is not detected. In general, when overdetection is increased, non-detection is reduced. That is, by detecting so as to include a lot of overdetection, almost no undetection is included.

Specifically, first, a plurality of threshold values are set in the range of luminance values of the precondition image, and for each threshold value, a pixel exceeding the threshold value is set to 255 (any value that can be distinguished from 0) is set. A binarized image in which pixels that do not exceed 0 are generated.
For example, when the luminance value range of the precondition image is 0 to 255 and 30 threshold values (threshold value 1 to threshold value 30) are set, a value obtained by dividing the luminance value into 30 equal parts in the range of 0 to 255 is obtained. Each width is 30 thresholds.

  FIG. 4 is an image obtained by binarizing the luminance value of the precondition image shown in FIG. FIG. 5 is an image obtained by binarizing the luminance value of the precondition image shown in FIG. FIG. 6 is an image obtained by binarizing the precondition image shown in FIG. FIG. 7 is an image obtained by binarizing the luminance value of the precondition image shown in FIG. As shown in FIGS. 4 to 7, the shape and number of connected components indicating regions that should be cells in the generated binary image are different depending on each threshold level.

For the binarized image generated for each threshold, connected components in the image are extracted and labeled as independent cell candidate regions. A set of cell candidate regions labeled for each threshold is assigned as a cell candidate region related to the frame ID “t”, and a cell candidate region ID that can be uniquely identified is assigned. As a result, the cell candidate region is detected so as to include a lot of overdetection but hardly include undetected.
That is, in the detection of the cell region related to the frame ID “t”, the control unit 11 of the cell behavior analysis apparatus 1 does not determine the cell region at the time of cell detection, but selects the cell candidate region so as to include many overdetections. Based on the detected cell candidate area, the cell area related to the frame ID “t” is determined based on the tracking result data related to the frame ID “t−1”. For this reason, the influence of an error at the time of cell detection in the automatic cell tracking result is reduced.

  As the cell detection method, the method of dividing the luminance value range of the precondition image by a plurality of threshold values and generating a plurality of binarized images to detect the cell candidate region has been described. Other detection methods may be used as long as undetected methods are hardly included. For example, the cell candidate region can be detected by the association using a super pixel which is a known technique.

Returning to the description of FIG. The control unit 11 of the cell behavior analysis apparatus 1 extracts a hypothesis of correspondence between the cell region ID in the tracking result data of the frame ID “t−1” and the cell candidate region ID detected in S1 (step S2). ).
With reference to FIG. 8 and FIG. 9, extraction of the association hypothesis will be described.

FIG. 8 is a diagram illustrating an example of a set of a cell region with a frame ID “t−1” and a cell candidate region with a frame ID “t”. In the example shown in FIG. 8A, the cell region ID = CellA cell region and the cell region ID = CellB cell region specified in the frame according to the frame ID “t−1” are shown. The cell boundary line related to the cell region ID in the frame ID “t−1” is generated based on the tracking result data stored in the storage unit 12. In the example illustrated in FIG. 8B, the cell candidate regions of the cell candidate region ID = A _t 1 to A _t 7 detected in the frame corresponding to the frame ID “t” are illustrated in an overlapping manner.

FIG. 9 is a diagram for explaining hypothesis extraction for associating the cell region ID of the frame ID “t−1” with the cell candidate region ID of the frame ID “t”.
Fig.9 (a) is a graph showing the position shift with respect to the elapsed time of the cell area CellA and the cell area CellB. The horizontal axis of the graph is the frame ID, and the vertical axis is the cell position information. The cell position information is, for example, position coordinates of the center point of the cell area or the cell candidate area. The center point is, for example, the barycentric coordinate of the cell region or the cell candidate region. In the graph, the position coordinates for each frame ID of the cell region CellA and the cell region CellB are plotted based on the tracking result data stored in the storage unit 12 of the cell behavior analysis apparatus 1, and further, in the frame ID “t”. The position coordinates of the cell candidate region ID = A _t 1 to A _t 7 are shown.

FIG. 9B shows a hypothesis for associating the cell region with the frame ID “t−1” and the cell candidate region with the frame ID “t”. For example, hypothesis 1 is a hypothesis in which cell region CellA is associated with cell candidate region A _t 1. Subsequently, the hypothesis that cell region CellA is associated with the cell candidate region _{A t} 2 and Hypothesis 2. Similarly, hypotheses for associating cell region IDs with cell candidate region IDs are exhaustively extracted.

The association between the cell region ID and the cell candidate region ID can be extracted as a hypothesis of a set of all cell region IDs and all cell candidate region IDs, but in terms of the amount of calculation executed by the control unit 11, For example, when the number of pixels belonging to the cell region and belonging to the cell candidate region is 1 or more, or when the distance between the center points of the cell region and the cell candidate region is less than or equal to a threshold value, the cell region and the cell candidate region It is preferable to extract the hypothesis of the correspondence. In the example shown in FIG. 9B, hypotheses 1 to 8 are extracted.
FIG. 9C is a table showing the correspondence between the cell area IDs of hypothesis 1 to hypothesis 8 and the cell candidate area ID.

Returning to the description of FIG. The control unit 11 of the cell behavior analysis apparatus 1 first calculates the likelihood P (1) of hypothesis 1 (the hypothesis in which the cell region CellA is associated with the cell candidate region A _t 1) by likelihood calculation processing (step S4). ). The likelihood P (1) of hypothesis 1 corresponds to the cell candidate area A _t 1 extracted in the frame related to the frame ID “t” from the cell area CellA specified in the frame related to the frame ID “t−1”. Indicates the probability of being attached. The control unit 11 includes an index I _D (CellA−A _t 1) regarding the movement between the cell area CellA of the frame ID “t−1” and the cell candidate area A _t 1 of the frame ID “t”, and the frame ID “t”. The likelihood P (1) is calculated using the index I _S (A _t 1) relating to the shape of the cell candidate region A _t 1.

FIG. 10 is a diagram for explaining an index used for likelihood calculation processing. The cell region CellA is specified in the frame related to the frame ID “t−1”. In addition, the cell candidate region A _t 1 is extracted in the frame related to the frame ID “t”. That is, in FIG. 10, the cell region CellA and the cell candidate region A _t 1 of different frames are overlapped and illustrated.

The index I _D (CellA-A _t 1) relating to the movement of the cell area CellA and the cell candidate area A _t 1 is an index representing the likelihood that the cell area CellA moves to the cell candidate area A _t 1. The overlapping ratio of the area CellA and the cell candidate area A _t 1 is mentioned. In the example shown in FIG. 10, it is the ratio of the hatched portion with respect to the union of the cell region CellA and the cell candidate region A _t 1. In other words, the overlap ratio of the cellular region CellA and cell candidate area _{A t} 1 is (belonging to the cell area CellA, cell candidate area number of pixels belonging to _{A t} 1) belonging to ÷ (cell region CellA, or cell candidate area _{A t} The number of pixels belonging to 1). The cell region CellA and cell candidate area _{A t} 1 of the mobile related index _{I D (CellA-A t 1} ), may be between centers point distance between the cell region CellA and cell candidate area _{A t} 1.

The index I _{S (A} t ₁₎ relating to the shape of the cell candidate area A _{t 1,} is an index representing the cell likelihood of a cell candidate area A _{t 1,} for example, along the boundary of the cell candidate area A _{t 1} integration An index using E (A _t 1) indicating the strength of the edge is given, and is represented by Expression (1). E (A _t 1) in equation (1) is derived by equation (2).

A larger value of E (A _t 1) indicates that the cell candidate region A _t 1 is more likely to be a cell. In order to keep the range of E (A _t 1) within the range of 0 to 1, Equation (1) is used. The value of the index I _S (A _t 1) relating to the shape of the cell candidate region A _t 1 approaches 0 when the value of E (A _t 1) approaches 0, and 1 when the value of E (A _t 1) increases. Get closer to. The parameter σ in Expression (1) can be set by the user so that, for example, the average value of I _S (A _t 1) to I _S (A _t 7) is 0.5.

As a cell candidate area _{A t} 1 of the shapes for the index _I S _(A t 1), without being limited thereto, for example, for each pixel in the cell candidate area _{A t} 1 the luminance value distribution and, in the cell candidate area _{A t} 1 Contour curvature distribution or the like may be used.

Using the index I _D (CellA-A _t 1) relating to the movement of the cell area A and the cell candidate area A _t 1 derived as described above and the index I _S (A _t 1) relating to the shape of the cell candidate area A _t 1. Thus, the likelihood P (1) of hypothesis 1 is calculated. Likelihood P (1) of hypothesis 1 in which cell area CellA of frame ID “t−1” is associated with cell candidate area A _t 1 of frame ID “t” is P (1) = I _D (CellA−A _t 1) × I _S (A _t 1).

Each index used in the above-described likelihood calculation process is an example, and other indexes may be used in the present invention. For example, an index related to the shape similarity between the cell region and the cell candidate region or the luminance histogram similarity between the cell region and the cell candidate region may be used. As an index related to the shape similarity, for example, SIFT (Scale Invariant Feature Transform) feature value, Fourier descriptor (Fourier Descriptors), HOG (Histograms of
Oriented Gradients) features. As the brightness histogram similarity, for example, the distance of the brightness histogram in the cell candidate region and in the cell region can be mentioned. Also, the index I _D of the similarity of the cell area and cell candidate region, may be used any one of the index I _S on the shape of the cell candidate region.

  Returning to the description of FIG. The control unit 11 of the cell behavior analysis apparatus 1 calculates each component of the constraint function C of the hypothesis 1 (step S5). Calculation of the constraint function C will be described with reference to FIG.

FIG. 11 is a diagram illustrating an example of the hypothesis likelihood P, the constraint function C, and the answer vector x. Here, for the sake of explanation, the total number of hypotheses extracted in step S2 is N, the number of cell region IDs related to the frame “t−1” is N ₁ , and the cell candidate region IDs related to the frame “t”. Is N ₂ . The hypothesis likelihood P and the answer vector x are N-dimensional column vectors. The constraint function C is a function in a matrix format with N rows (N ₁ + N ₂ ) columns. The hypothesis likelihood P, the constraint function C, and the row of the answer vector x correspond to the hypothesis numbers extracted in step S2. In the example shown in FIG. 11, it corresponds to Hypothesis 1 to Hypothesis 8. The row component of the hypothesis likelihood P corresponds to the likelihood values (P (1) to P (8)) of hypothesis 1 to hypothesis 8. The columns of the constraint function C correspond to _{1 to} N ₁ , N ₁ +1 to N ₁ + N ₂ . In the example shown in FIG. 11, corresponding to the cell region CellA~ cell region CellB and cell candidate region _A t. 1 to the cell candidate area _{A t} 7, the constraint function C is 8 × 9 matrix.

Next, a method for calculating each component of the constraint function C will be described. Here, for the sake of explanation, m _y εS (A _t Y) is set. S (A _t Y) is a cell candidate in a cell candidate region (including cell candidate region A _t Y) that shares pixels belonging to cell candidate region A _t Y among the cell candidate regions related to frame “t”. It means a set of area IDs. For example, in S (A _t 1), since the cell candidate region A _t 1 has an overlap with all cell candidate regions, S (A _t 1) = {A _t 1, A _t 2, A _t 3, A _t 4, _At5 , _At6 , _At7 }. If S (A _t 2), the cell candidate region A _t 2 has an overlap with the cell candidate region A _t 1 and the cell candidate region A _t 3, so S (A _t 2) = {A _t 1, A _t2 , _At3 }.

Here, the M rows and i columns of the constraint function C corresponding to the hypothesis M in which the cell region CellX of the frame ID “t−1” is associated with the cell candidate region A _t Y of the frame ID “t” are included in the cell region CellX. and the corresponding component, and 1 components corresponding to S (a t _Y) cell candidate area m _y contained in the other component is 0. The constraint function C can be expressed by the following equation (3).

For example, the components of the 1st row and ith column of the constraint function C corresponding to Hypothesis 1 are i = 1 (cell region CellA) and i = 2 + 1 to 2 + 7, {A _t 1, A _t 2, A _t 3, A _t 4 , A _t 5, A _t 6, A _t 7} = S (A _t 1) is 1, and i = 2 is 0. In addition, the components in the 2nd row and the ith column of the constraint function C corresponding to the hypothesis 2 are i = 1 (cell region CellA) and i = 2 + 1 to 2 + 3, {A _t 1, A _t 2, A _t 3} = S ( a _t 2) was used as a 1, and i = 2, 2 + 4 to 7 0.

That is, the constraint function C is a function that eliminates all contradictory hypotheses when a hypothesis is established. For example, when Hypothesis 1 is established, the cell region CellA moves to the cell candidate region A _t 1 in the next frame. In this case, Hypothesis 2 to Hypothesis 8 are not consistent and are not allowed. Further, when the hypothesis 2 is satisfied, it becomes that the cell region CellA moves to the cell candidate region A _{t 2} in the next frame, in this case, although the hypothesis 1 and hypothesis 3 conflicting hypotheses 4 hypothesis 8 is consistent in particular .

  Returning to the description of FIG. The control unit 11 of the cell behavior analysis apparatus 1 repeats the processes of steps S4 to S5 and executes the processes for the number of hypotheses (N = 8). The control unit 11 of the cell behavior analysis apparatus 1 generates a hypothesis likelihood vector P and a constraint function C based on the calculated likelihoods P (1) to P (8) and each component of the constraint function C. . The control unit 11 of the cell behavior analysis apparatus 1 identifies an optimal hypothesis from the hypotheses extracted in step S2 (step S8). The extraction of the optimal hypothesis will be described with reference to FIG.

  The control unit 11 of the cell behavior analysis apparatus 1 specifies an optimal hypothesis by deriving an answer vector x using integer programming. Each component of the answer vector x corresponds to each hypothesis, 1 is assigned to the component corresponding to the adopted hypothesis, and 0 is assigned to the component corresponding to the hypothesis that has not been adopted. The example shown in FIG. 11 shows the answer vector x when hypothesis 2 and hypothesis 5 are adopted. The derivation of the answer vector x is represented by the following equation (4) by integer programming.

Equation (4) is obtained by satisfying the constraint condition that each element of the vector generated by the product of the transposed matrix of the constraint function (constraint matrix) C and the answer vector x is 1 or less. It represents deriving an answer vector x that maximizes the product of the answer vectors x.
This can be solved using a branch-and-bound algorithm based on linear programming (LP). This algorithm searches for the optimal solution for the 0-1 integer programming problem by solving a series of LP relaxation problems that replace the 0-1 integer requirements for variables with weaker constraints 0 ≦ x ≦ 1. The algorithm is as follows (see Non-Patent Document 3 and Non-Patent Document 4).
Search for a feasible solution of 0-1 integers.
As the search tree grows, update the optimal 0-1 integer possible points found so far.
• Confirm that there are no more feasible integer feasible solutions by solving a series of linear programming.
<Non-Patent Document 3> “bintprog”, [online], MathWorks, [searched on September 28, 2012], Internet
<http://www.mathworks.co.jp/help/optim/ug/bintprog.html>
<Non-Patent Document 4> Papadimitriou, CH, Steiglitz, K., Combinatorial Optimization:
Algorithms and Complexity, Dover Publications, 1998.

  In Equation (4), the product of the transposed vector of the likelihood vector P and the answer vector x means the sum of the likelihood values of the adopted hypothesis, and the sum of the likelihood values of the adopted hypothesis is the maximum. An answer vector x such that In the example shown in FIG. 11, the sum of the likelihood P (2) of hypothesis 2 and the likelihood P (5) of hypothesis 5 is the maximum among the constraint conditions. The number of cell region IDs related to the frame “t−1” and the number of hypotheses adopted by the equation (4) are the same.

  Returning to the description of FIG. The control unit 11 of the cell behavior analysis apparatus 1 updates the tracking result data stored in the storage unit 12 based on the optimal hypothesis specified in step S8 (step S9).

The control unit 11 identifies the cell candidate region ID included in the adopted hypothesis as the cell region related to the frame ID “t”, and is the same as the cell region ID of the frame ID “t−1” associated with the hypothesis. Give an ID. In the example shown in FIG. 11, the control unit 11, the cell candidate region ID = _{A t} 2 contained in Hypothesis 2 Grant cell region ID = CellA, cells the cell candidate region ID = _{A t} 4 contained hypothesis 5 The area ID = CellB is assigned and stored in the tracking result data stored in the storage unit 12. Also, the value of the area number ID is incremented by one. As a result, the cell detection related to the frame ID “t” and the association of each cell of the frame ID “t−1” and the frame ID “t” are executed simultaneously. The storage unit 12 of the cell behavior analysis apparatus 1 stores a threshold value of the region number ID in advance, and the control unit 11 determines the corresponding cell region ID if the region number ID related to the frame “t” is equal to or greater than the threshold value.

  In the above description, only “movement” is assumed as the simplest condition. However, the present invention can be similarly applied even when more complicated conditions are assumed. For example, if the values of the hypothesis likelihoods P (2) and P (5) specified in S8 are equal to or less than the threshold value stored in advance in the storage unit 12, the control unit 11 updates the tracking result data. Without moving, the cell region CellA and the cell region CellB are temporarily out of frame (= “disappeared”) while moving from the frame “t−1” to the frame “t”, and the tracking is finished.

However, under conditions where cells move under high density, the outline of each cell may become temporarily unclear, and the above operation may not be tracked between frames despite the presence of cells. Occur. Therefore, a method for restarting automatic tracking for a cell candidate region that is not determined as a cell region by the above-described processing and whose cell outline is clear will be described. First, the cell behavior analysis apparatus 1 assigns a cell candidate region that is not determined as a cell region by the above-described processing and has a clear cell outline to the frame ID “t−1” and the frame ID “t”. Each is extracted from such a frame. Here, a cell candidate region extracted from the frame ID “t−1” is A ′ _t−1, and a cell candidate region extracted from the frame ID “t” is A ′ _t . The cell behavior analysis apparatus 1 exhaustively extracts hypotheses in which the cell candidate region A ′ _t−1 is associated with the cell candidate region A ′ _t , and specifies the optimal hypothesis. The optimum combination of hypotheses is specified by the same method as the derivation of the answer vector x by the integer programming described above, but the likelihood P ′ is used as the likelihood of the hypothesis. Likelihood P'hypothesis M cell candidate region X is associated with the cell candidate region Y (M) is the index I _s on the shape of the cell candidate region described above, the index I _D about moving cellular region and cell candidate region , P ′ (M) = I _D (A ′ _t Y−A ′ _t−1 X) × I _s (A ′ _t Y) × I _S (A ′ _t−1 X) it can. If P ′ (M) is equal to or greater than the threshold, the cell behavior analysis apparatus 1 assigns the same cell region ID to the cell candidate region, and sets “1” to the region number ID related to the frame “t−1”. “2” is stored in the area number ID related to the frame “t”, the tracking result data in the storage unit 12 is updated, and automatic tracking is started.

  Further, based on the confirmed tracking result of the cell region ID, for example, from one cell region in the frame “t−1” to corresponding to two different cell regions in the frame “t”, “cell division” May be allowed. Optimal tracking result data can be obtained by a known technique based on integer programming.

  Further, the cell behavior index can be calculated based on the tracking result data of the cell region ID optimized by the present invention. As a result, the number of cells, cell movement speed, cell size (number of pixels in the cell region), number of cell divisions, amount of pigment in the cell region (color information), and circularity of the cell region (shape) Information) and the like can be calculated, and can be used for evaluation of cell quality and culture conditions with high accuracy.

  Next, an example in which automatic tracking processing is performed by the cell behavior analysis apparatus 1 according to the embodiment of the present invention will be described with reference to FIGS.

Using the cell behavior analysis apparatus 1 according to the present invention, a RPE (retinal pigment cell) is cultured for 30 days, and a time-lapse image obtained by imaging a cultured cell with a CCD camera having a resolution of 320 × 320 pixels (1.03 μm / pixel). An automatic tracking process was executed. FIG. 12 is a diagram showing (a) distribution of cell size and cell migration speed of cultured cells in the initial state, and (b) distribution of cell size and cell migration speed of cultured cells 21 days after the start. The horizontal axis indicates the cell movement speed, and the vertical axis indicates the cell region size. The number of plots is the same as the number of cells measured. It can be seen that the cultured cells 21 days after the start have a small cell size and a slow cell migration speed. This indicates that the cells are present under high density conditions.
In this example, the cell size and the cell movement speed are used as the cell behavior index. However, the cell behavior analysis apparatus 1 of the present invention allows the time transition of the cell number, the time transition of the cell division number, and the dye in the cell region. It is also possible to calculate a cell behavior index such as the amount and the circularity of the cell region.

  FIG. 13 shows one frame as a result of visual cell detection performed on a precondition image obtained by imaging cultured cells in the initial state. An identifier is assigned to the detected cell. FIG. 14 shows one frame as a result of performing cell detection on the same precondition image as in FIG. 13 by the cell behavior analysis apparatus 1 according to the present invention. The outline of the detected cell is indicated by a bold line, and the numerical value represents the cell region ID. The precondition image to be detected has a clear cell outline, and the result of cell detection by visual observation and the result of cell detection by the cell behavior analysis apparatus 1 according to the present invention are substantially the same.

  FIG. 15 shows one frame of a result of performing cell detection by visual observation on a precondition image obtained by imaging cultured cells 21 days after the start. An identifier is assigned to the detected cell. FIG. 16 shows one frame of the result of cell detection performed by the cell behavior analysis apparatus 1 according to the present invention on the same precondition image as FIG. The outline of the detected cell is indicated by a bold line, and the numerical value represents the cell region ID. In the precondition image to be detected, the outline of the cell is unclear, but the cell detection result by visual observation and the cell detection result by the cell behavior analysis apparatus 1 according to the present invention substantially coincide. By using the cell behavior analysis apparatus 1 according to the present invention, the coincidence rate of the automatic tracking data with the correct answer data (result of manual detection by visual observation) is improved by 35% or more compared to the automatic tracking result data by the conventional method.

  The preferred embodiments of the cell behavior analysis apparatus and the like according to the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to such examples. It will be apparent to those skilled in the art that various changes or modifications can be conceived within the scope of the technical idea disclosed in the present application, and these naturally belong to the technical scope of the present invention. Understood

DESCRIPTION OF SYMBOLS 1 ......... Cell behavior analyzer 11 ......... Control part 12 ......... Storage part 13 ......... Media input / output part 14 ...... Communication control part 15 ......... Input part 16 ......... Display part 17 ... ... Peripheral equipment I / F section

Claims (7)

Cell candidate region detection means for detecting cell candidate regions of individual cells in the time lapse image for each frame of the time lapse image so that many overdetections are included but almost no detection is included,
Hypothesis extraction means for extracting a hypothesis of correspondence between the cell region of the previous frame and the cell candidate region of the detection target frame with respect to the two frames that are continuous in time series,
Likelihood calculation means for calculating the likelihood of the hypothesis extracted by the hypothesis extraction means;
A hypothesis specifying means for specifying the optimal hypothesis based on the likelihood calculated by the likelihood calculating means;
A cell region specifying unit that uses the cell candidate region included in the hypothesis specified by the hypothesis specifying unit as the cell region;
Storage means for storing tracking result data in which cell area information including position information of the cell area specified by the cell area specifying means is stored in chronological order;
A cell behavior analysis device comprising: The cell candidate region detection means includes
A plurality of threshold values are set in a range of luminance values of the time-lapse image, a binarized image is generated for each threshold value for each frame, and a connected component of the generated binarized image is extracted and extracted. The cell behavior analysis apparatus according to claim 1, wherein the set of connected components is used as the cell candidate region. The likelihood calculating means calculates the likelihood using an index related to movement between the cell region of the previous frame and the cell candidate region of the detection target frame and an index related to the shape of the cell candidate region. The cell behavior analysis device according to claim 1 or 2. The hypothesis specifying means includes
The number of the extracted hypotheses is the number of rows, and a constraint matrix having the number of columns of the number of the cell regions and the number of the cell candidate regions related to the hypothesis, and for each hypothesis, the hypothesis And the cell region, and the constraint matrix having the mutual relationship of the cell candidate region related to the hypothesis and the answer vector satisfy the constraint condition, the likelihood of the hypothesis 4. The optimal hypothesis is identified by solving an integer programming problem that maximizes the product of a transposed vector of likelihood vectors and the answer vector. The cell behavior analysis device according to Item. The cell behavior index calculation means for calculating a cell behavior index based on the tracking result data stored by the storage means is further provided. Cell behavior analyzer. A cell candidate region detection step for detecting, for each frame of the time lapse image, a cell candidate region of an individual cell in the time lapse image so that many overdetections are included but almost no detection is included;
A hypothesis extraction step for exhaustively extracting hypotheses of correspondence between the cell region of the previous frame and the cell candidate region of the detection target frame for the two frames that are continuous in time series;
A likelihood calculating step of calculating the likelihood of the hypothesis extracted by the hypothesis extracting step;
A hypothesis specifying step for specifying the optimal hypothesis based on the likelihood calculated by the likelihood calculating step;
A cell region specifying step in which the cell candidate region included in the hypothesis specified by the hypothesis specifying step is the cell region; and
A storage step of storing tracking result data in which cell region information including position information of the cell region specified by the cell region specifying step is stored in chronological order;
A cell behavior analysis method comprising: Computer
Cell candidate region detection means for detecting cell candidate regions of individual cells in the time lapse image for each frame of the time lapse image so that many overdetections are included but almost no detection is included,
Hypothesis extraction means for exhaustively extracting hypotheses of correspondence between the cell region of the previous frame and the cell candidate region of the detection target frame with respect to the two frames that are continuous in time series,
Likelihood calculation means for calculating the likelihood of the hypothesis extracted by the hypothesis extraction means;
A hypothesis specifying means for specifying the optimal hypothesis based on the likelihood calculated by the likelihood calculating means;
A cell region specifying unit that uses the cell candidate region included in the hypothesis specified by the hypothesis specifying unit as the cell region;
Storage means for storing tracking result data in which cell area information including position information of the cell area specified by the cell area specifying means is stored in chronological order;
Program to function as.