WO2010098211A1

WO2010098211A1 - Contour definition device and contour definition method, and program

Info

Publication number: WO2010098211A1
Application number: PCT/JP2010/051999
Authority: WO
Inventors: 敦史宮脇; 裕黒川; 久順野田
Original assignee: 独立行政法人理化学研究所
Priority date: 2009-02-24
Filing date: 2010-02-10
Publication date: 2010-09-02
Also published as: JPWO2010098211A1

Abstract

The disclosed method sets the initial contour of an object within the image data for an object that has been imaged, produces a convergent contour by convergence of the initial contour that has been set, acquires the brightness on the resulting convergent contour, calculates the brightness distribution, compares the calculated brightness distribution with a predetermined threshold value, and determines whether the dispersion is greater than the threshold value or if the dispersion is less than the threshold value. If the distribution is determined to be greater than the threshold value, the resulting convergent contour is expanded and said expanded contour is set as the initial contour, and if the distribution is determined to be less than the threshold value, the resulting convergent contour is set as the contour of the object.

Description

Outline extraction apparatus, outline extraction method, and program

The present invention relates to a contour extraction device, a contour extraction method, and a program, and more particularly, to a contour extraction device, a contour extraction method, and a program for a cell.

2. Description of the Related Art Conventionally, an image analysis technique has been developed that extracts the contour of an object using image contouring (Snakes) or the like for image data including the object.

For example, in Patent Document 1, when a contour model is contracted or enlarged using Snakes and the distance between two nodes that are not adjacent to each other is smaller than a threshold value, the contour model is split at the two nodes. It is disclosed.

In Patent Document 2, it is disclosed that the contour model is contracted and deformed using Snakes, and the contact model is divided into a plurality when the contact or intersection in the contour model is detected.

Further, Non-Patent Document 1, unlike the cell contour extraction method by the Snakes method, classifies a cell region and a background from a phase contrast microscope image by creating a binary map based on a luminance histogram, A technique for tracking cells so that the energy function related to cell dynamics is minimized in time order is disclosed.

As described above, the cell extraction method and the cell tracking method in the image analysis software for the conventional fluorescence microscope image set a threshold value for a certain luminance value, extract a cell region from the analysis target image, and position information, etc. The cell movement was tracked from the statistical analysis results.

JP 2002-92622 A JP-A-8-329254

However, the conventional cell extraction method using threshold processing has a problem in that it may be difficult to perform good cell extraction due to a decrease in luminance due to a change in fluorescence luminance or a background luminance gradient.

In addition, when a cell contour is extracted by the Snakes method for efficiently extracting the contour of the region of interest in the image and applied to a time-lapse image, cell tracking is performed using a technique of repeatedly expanding and contracting the Snakes contour. Can be considered. However, in the algorithm in which the contour range calculated by Snakes is simply expanded once and re-reduced in the next frame, the cell moves beyond the expanded range of the contour, and the initial contour before the recontraction calculation When the cell region intersects, there is a problem that the contour energy is minimized by the false contour generated by the luminance change inside the cell, and the true contour of the cell cannot be traced.

The present invention has been made in view of the above problems, and an object of the present invention is to provide a contour extraction device, a contour extraction method, and a program that can accurately extract the true contour of an object. .

In order to achieve such an object, the contour extraction apparatus of the present invention includes at least a storage unit and a control unit, and extracts a contour of an object. The storage unit images the object. Image data storage means for storing the image data, and the control unit, in the image data stored in the image data storage means, initial contour setting means for setting an initial contour of the object, and the initial contour Contour convergence means for converging the initial contour set by the setting means to generate a convergent contour, and luminance dispersion for obtaining the luminance on the convergent contour generated by the contour convergence means and calculating the variance of the luminance The calculation means and the variance of the luminance calculated by the luminance variance calculation means are compared with a predetermined threshold value, and the variance is greater than the threshold value, or A threshold value determining means for determining whether or not the variance is equal to or less than the threshold value, and when the threshold value determining means determines that the variance is greater than the threshold value, the convergence contour generated by the contour convergence means is expanded, The contour expanding means for setting the expanded contour as the initial contour, and the convergence contour generated by the contour converging means when the variance is determined to be less than or equal to the threshold by the threshold determining means. A contour extracting means for extracting the contour of the object.

Further, the contour extracting device of the present invention is characterized in that, in the contour extracting device described above, the contour converging means and the contour expanding means generate and expand the convergent contour by a dynamic contour modeling method.

Further, the contour extracting device of the present invention is the contour extracting device described above, wherein the control unit acquires the center position and / or the in-contour brightness of the contour of the object extracted by the contour extracting means. An acquisition means is further provided.

In the contour extracting device according to the present invention, the image data includes a plurality of frames captured at a plurality of times, and the control unit converts the plurality of frames into the time. Control is performed so that processing is performed in the forward or reverse order.

In the contour extracting device according to the present invention, when the control unit controls the plurality of frames to be processed in the reverse order of the time, the contour of the plurality of objects is obtained. And a cell division point setting means for setting a cell division point when the center position is smaller than a predetermined threshold value.

Further, the contour extracting apparatus of the present invention is the contour extracting apparatus described above, wherein the luminance variance calculating means weights the luminance for each color when the image data includes luminance information corresponding to a plurality of colors. The p-th root of the sum of the powers of information p (where p is a parameter for the luminance information) is acquired as the luminance and the variance of the luminance is calculated.

The present invention also relates to a contour extraction method, and the contour extraction method of the present invention is a contour extraction method that is executed in a contour extraction device that extracts at least a contour of an object, which includes at least a storage unit and a control unit. The storage unit includes image data storage means for storing image data obtained by imaging the object, and the image data stored in the image data storage means is executed in the control unit. An initial contour setting step for setting an initial contour of the object, a contour convergence step for generating a convergent contour by converging the initial contour set in the initial contour setting step, and a contour convergence step. Luminance on the converged contour is obtained, and a luminance variance calculating step for calculating the variance of the luminance and the luminance variance calculating step calculated above The threshold value is compared with a predetermined threshold value to determine whether the variance value is greater than the threshold value or whether the variance value is less than or equal to the threshold value. When it is determined that the variance is larger than the threshold value, the contour extension step for expanding the convergent contour generated in the contour convergence step and setting the expanded contour as the initial contour; and the threshold determination step A contour extracting step of extracting the convergent contour generated in the contour convergence step as the contour of the object when it is determined that the variance is equal to or less than the threshold. .

In addition, the present invention relates to a program, and the program of the present invention is a program for causing an outline extraction apparatus including at least a storage unit and a control unit to execute the program, and the storage unit stores the object. An image data storage means for storing captured image data, and an initial contour setting step for setting an initial contour of the object in the image data stored in the image data storage means in the control unit; A contour convergence step for generating a convergent contour by converging the initial contour set in the initial contour setting step, and obtaining a luminance on the convergent contour generated in the contour convergence step, and distributing the luminance A luminance variance calculating step for calculating the luminance, the variance of the luminance calculated in the luminance variance calculating step, and a predetermined threshold value, In comparison, the threshold determination step for determining whether the variance is greater than the threshold or whether the variance is less than or equal to the threshold, and when the variance is determined to be greater than the threshold in the threshold determination step , Expanding the convergence contour generated in the contour convergence step, setting the expanded contour as the initial contour, and determining that the variance is equal to or less than the threshold in the threshold determination step And a contour extraction step for extracting the convergent contour generated in the contour convergence step as the contour of the object.

According to the present invention, (1) an initial contour of an object is set in image data obtained by imaging the object, (2) a convergent contour is generated by converging the set initial contour, and (3) is generated. The luminance on the convergent contour is acquired, and the luminance variance is calculated. (4) The calculated luminance variance is compared with a predetermined threshold, and the variance is greater than the threshold or the variance is less than or equal to the threshold. (5) When it is determined that the variance is larger than the threshold, the generated convergent contour is expanded, and the expanded contour is set as the initial contour. (6) The variance is equal to or less than the threshold When it determines with it being, it extracts the produced | generated convergence outline as an outline of a target object. Thereby, there exists an effect that the true outline of a target object can be extracted correctly.

In addition, in the above (2) and (5), the present invention generates and expands the convergent contour by the dynamic contour modeling method (Snales), so that it is possible to accurately fit the contour of the object. .

Also, the present invention obtains the center position and / or the luminance within the contour of the extracted object, so that the position and the luminance of the object can be analyzed with high accuracy.

In the present invention, the image data is composed of a plurality of frames captured at a plurality of times, and the plurality of frames are controlled to be processed in the normal order or the reverse order of the times. Even when the object is deformed or moved, the object can be tracked with high accuracy.

Further, the present invention sets the cell division point when the center position of the contours of the plurality of objects is smaller than a predetermined threshold when the plurality of frames are controlled to be processed in reverse order of time. Even when cell division occurs, the cell division point can be analyzed accurately without complicating the algorithm. That is, in the conventional method for analyzing frames in the order of time, cell division is determined by a complicated algorithm called contour model division determination, whereas in the present invention, the frame is analyzed in reverse order of time, and the cells are analyzed. Therefore, the algorithm can be simplified and an accurate cell division point can be analyzed.

In addition, in the above (2), the present invention relates to the p-th root of the p-th power sum of the luminance information weighted for each color when the image data includes luminance information corresponding to a plurality of colors (where p is the luminance). Is obtained as the luminance and the variance of the luminance is calculated, so even if the image is expressed in multiple colors instead of a single color, the luminance corresponding to each color is a single luminance. There is an effect that the calculation for contour extraction can be performed by unifying the indices.

FIG. 1 is a diagram schematically showing how the initial contour is converged from the initial contour by the dynamic contour modeling method and fitted to the contour of the object. FIG. 2 is a block diagram showing an example of the configuration of the contour extraction apparatus 100 to which the present invention is applied. FIG. 3 is a flowchart showing an example of the contour extraction process of the contour extraction apparatus 100 according to the present embodiment. FIG. 4 is a conceptual diagram showing an example in which the contour extraction process according to the present embodiment is performed. FIG. 5 is a diagram illustrating an example in which processing according to a conventional method is performed and an example in which contour extraction processing according to the present embodiment is performed. FIG. 6 is a flowchart illustrating an example of the time order analysis process. FIG. 7 is a diagram schematically showing a state in which the extracted contour extracted in the previous frame is expanded, the initial contour in the next frame is set, and a convergent contour is generated. FIG. 8 is a flowchart illustrating an example of the reverse time order analysis process. FIG. 9 is a diagram schematically illustrating an example of processing for resetting the initial contour by Hough transform. FIG. 10 is a diagram illustrating an example of a contour extraction result according to the present embodiment. FIG. 11 is a diagram for explaining the principle of determining the cell division point. FIG. 12 is a diagram illustrating, as an example, the result of performing contour extraction processing in reverse order on the time axis in time-continuous frames. FIG. 13 is a diagram showing a contour extraction result (left diagram) and a cell division analysis result (right diagram) in a certain frame k. FIG. 14 is a flowchart illustrating the processing of the first embodiment. FIG. 15 is a diagram showing a result of analyzing a fluorescence imaging image of a primary cultured cell of ZebraFish into which a fluorescent protein type indicator Fucci that emits fluorescence specifically in the cell cycle is introduced, based on this experimental example 2. FIG. 16 is a graph showing changes in the intensity of each fluorescence in the contour extracted by the second embodiment, and a phylogenetic tree of cell division created from the cell tracking result and the luminance information in the contour according to the second embodiment. FIG. FIG. 17 is a diagram illustrating an example in which an ROI is set for a colony of HeLa cells and contour extraction of the entire colony is performed.

Embodiments of a contour extraction apparatus, a contour extraction method, and a program according to the present invention will be described below in detail with reference to the drawings. Note that the present invention is not limited to the embodiments.

In particular, in the following embodiment, an example in which the present invention is applied to a cell analysis technique may be described. However, the present invention is not limited to this case, and can be similarly applied to all technical fields such as a security system.

[Outline of the present invention]
Hereinafter, the outline of the present invention will be described with reference to FIG. 1, and then the configuration and processing of the present invention will be described in detail.

The present invention generally has the following basic features. In other words, the contour extracting apparatus of the present invention includes a storage unit and a control unit, and stores image data obtained by capturing an object.

The contour extracting apparatus of the present invention sets the initial contour of the object in the stored image data image. The initial contour is set by manually or automatically using a known technique so as to surround the object to be analyzed.

Then, the contour extraction apparatus of the present invention generates a convergent contour by converging the set initial contour. Here, FIG. 1 is a diagram schematically showing how the initial contour is converged from the initial contour by the dynamic contour modeling method and fitted to the contour of the object. In FIG. 1, the left diagram represents the object and the initial contour, and the right diagram represents the object and the convergent contour. As shown in FIG. 1, the contour extraction apparatus of the present invention generates a convergent contour from a circular initial contour by a dynamic contour modeling method or the like.

Then, the contour extracting apparatus of the present invention acquires the luminance on the convergent contour and calculates the luminance variance. Here, “brightness variance” is a value indicating variation in luminance value, and is represented by, for example, a standard deviation that is a square root of variance.

Then, the contour extraction apparatus of the present invention compares the luminance variance with a predetermined threshold value, and determines whether the variance is greater than the threshold value or whether the variance is less than or equal to the threshold value.

Then, when it is determined that the variance is larger than the threshold value, the contour extracting apparatus of the present invention expands the convergent contour, sets the expanded contour as the initial contour, and repeats the above processing again.

On the other hand, when it is determined that the variance is equal to or less than the threshold, the contour extracting apparatus of the present invention extracts the generated convergent contour as the contour of the object. Here, the contour extracting apparatus of the present invention may acquire the center position of the contour of the object and / or the luminance within the contour.

The above is the outline of the contour extraction processing of the present invention in one frame image. Here, when the image data is composed of a plurality of frames captured at a plurality of times by shooting at regular intervals, etc., the above-described contour extraction processing is performed on the plurality of frames in the normal order or the reverse order of the times. You may control. Further, when the above-described contour extraction process is performed in the reverse order, the cell division point may be set when the center positions of the contours of a plurality of objects are smaller than a predetermined threshold value. This is the end of the description of the outline of the present invention.

[Configuration of the present contour extraction apparatus and system]
Next, the configuration of the contour extracting apparatus will be described with reference to FIG. FIG. 2 is a block diagram showing an example of the configuration of the contour extraction apparatus 100 to which the present invention is applied, and conceptually shows only the portion related to the present invention. As shown in FIG. 2, the contour extracting apparatus 100 is generally configured to include a control unit 102, a communication control interface unit 104, a storage unit 106, and an input / output control interface unit 108. Are communicably connected via an arbitrary communication path.

In FIG. 2, the control unit 102 is a CPU or the like that comprehensively controls the entire contour extraction apparatus 100. The input / output control interface unit 108 is an interface connected to the input unit 112 and the output unit 114. The storage unit 106 is a device that stores various databases and tables.

Various databases and tables (image data file 106a to cell division point file 106d) stored in the storage unit 106 are storage means such as a fixed disk device. For example, the storage unit 106 stores various programs, tables, files, databases, and the like used for various processes.

Among these components of the storage unit 106, the image data file 106a is an image data storage unit that stores image data obtained by capturing an image of an object. Here, the image data file 106a may store image data composed of a plurality of frames 1 to n imaged at a plurality of times by regular interval shooting (time-lapse shooting) or the like. In the following embodiment, frame 1 represents the first frame on the time axis, frame n represents the last frame on the time axis, and frame k represents an arbitrary frame. The image data file 106a may store image data including luminance information corresponding to a plurality of colors (for example, luminance values corresponding to the three primary colors of RGB), and the image data may be stored in the original image. As data, image data obtained by converting the p-th root of the p-th power sum of luminance information weighted for each color (where p is a parameter for luminance information) as luminance may be stored.

The position information file 106b is position information storage means for storing the center position of the contour of the object. Here, in the present embodiment, each contour is assigned a contour number (ID), and one object is associated with one initial contour. Identified by contour number. That is, in this position information file 106b, coordinates (x, y) are stored in association with an outline number (ID) or a frame number as an example.

Also, the luminance information file 106c is luminance information storage means for storing the luminance within the contour of the target object. For example, the luminance information file 106c stores luminance values in association with contour numbers and frame numbers.

The cell division point file 106d is a cell division point storage means for storing cell division points. For example, the cell division point file 106d stores the contour numbers of the contours of two objects in association with the frame numbers.

In FIG. 2, the input / output control interface unit 108 controls the input unit 112 and the output unit 114. Here, the output unit 114 is a monitor (including a home television), a speaker, or the like (hereinafter, the output unit 114 may be described as a monitor). As the input unit 112, a keyboard, a mouse, a microphone, and the like can be used in addition to an image input device such as a microscopic photographing device for photographing at regular intervals.

In FIG. 2, the control unit 102 has an internal memory for storing a control program such as an OS (Operating System), a program defining various processing procedures, and necessary data. And the control part 102 performs the information processing for performing various processes by these programs. The control unit 102 includes a frame setting unit 102a, an initial contour setting unit 102d, a contour convergence unit 102e, a luminance variance calculation unit 102f, a threshold determination unit 102g, a contour expansion unit 102h, a contour extraction unit 102i, and a contour acquisition unit in terms of functional concept. 102j and a cell division point setting unit 102k.

Among these, the frame setting unit 102a controls the plurality of frames stored in the image data file 106a so as to perform contour extraction processing in the normal or reverse order of time. Here, as shown in FIG. 2, the frame setting unit 102a includes a time reverse order frame setting unit 102b and a time normal order frame setting unit 102c. The time reverse order frame setting unit 102b is a time reverse order frame setting unit that sets the final frame n as a start point (key frame) and sets the frame k so that the contour extraction processing is performed in the reverse time order until the first frame 1 is reached. is there. The time normal order frame setting unit 102c sets the first frame 1 as the start point (key frame) and sets the frame k so that the contour extraction processing is performed in the time normal order until the last frame n is reached. Means.

The initial contour setting unit 102d is initial contour setting means for setting the initial contour of the object in the image data stored in the image data file 106a. That is, the initial contour is set so as to surround the object by the initial contour setting unit 102d, and is converged (and expanded if necessary) so as to match the true contour of the object in the subsequent processing. The contour and the convergent contour (that is, the dynamic contour model) represent the ROI (Region of Interest) for each object (in the following embodiment, to distinguish it from the true contour of the object). Sometimes called “ROI”). The initial contour setting unit 102d manages each ROI with a uniform contour number (ID) for the same object across frames.

In the initial contour setting process by the initial contour setting unit 102d, for example, the X coordinate / Y coordinate indicating the initial contour setting position, the number of initial divisions indicating how many lines the initial contour is represented by, An initial radius indicating the size of the contour is set. Some or all of these setting parameters may be set automatically by a known initial contour setting means, or may be set manually. In addition, when the luminance (signal) disappears in the previous frame and reappears in the next frame, the initial contour setting unit 102d performs feature extraction by the Hough transform, and near the ROI in the frame in which the signal is present. An initial contour may be set using feature points.

The contour convergence unit 102e is a contour convergence unit that converges the initial contour set by the initial contour setting unit 102d or the contour expansion unit 102h and generates a convergent contour. Here, the contour convergence unit 102e may generate a convergent contour by a dynamic contour modeling method. More specifically, the contour convergence unit 102e generates a convergent contour by converging the contour (ROI) so that the value calculated by the energy function E _snacks below is minimized.
E _snakes = ∫ {E _in (v (s)) + E _img (v (s)) + E _con (v (s))} ds
(Here, E _in represents the internal energy term of the contour line, and is a function in which the energy decreases as the contour length of the ROI is shorter and the smoothness of the line segment set is higher. E _img is This represents a minimum energy term at the edge, and is a function in which the energy becomes smaller as the image is differentiated and closer to 0. E _con represents an energy term due to external binding force.

For example, in order to generate a contraction contour by the contour convergence unit 102e, when the ROI is contracted, a “contracting force” indicating how much the ROI contracts or a smooth curve of the ROI may be defined. “Smoothness” representing the degree of “ROI”, “adsorption force” representing how much the ROI is to be retained in the vicinity of the object, and “minimum number of vertices” representing how many ROIs are defined as a set of straight lines The parameters such as “the number of contractions” indicating how many times the contraction calculation is performed at the time of ROI calculation can be set in accordance with the type of object, the purpose of analysis, and the like.

Also, the luminance dispersion calculation unit 102f is a luminance dispersion calculation unit that acquires the luminance on the convergence contour generated by the contour convergence unit 102e and calculates the luminance dispersion. Here, the luminance variance calculation unit 102f performs color processing when the image data stored in the image data file 106a includes luminance information corresponding to a plurality of colors (for example, luminance values corresponding to the three primary colors of RGB). The luminance variance may be calculated by acquiring the p-th root of the p-th power sum of the luminance information weighted for each (where p is a parameter for the luminance information) as the luminance.

The threshold determination unit 102g compares the luminance variance calculated by the luminance variance calculation unit 102f with a predetermined threshold, and determines whether the variance is greater than the threshold or whether the variance is equal to or less than the threshold. It is a threshold value determination means. For example, the luminance variance calculation unit 102f determines whether or not the luminance variance on the convergence contour is a variance larger than a certain standard deviation as a threshold value.

Further, the contour expanding unit 102h is a contour expanding unit that expands the contour. Here, when the initial contour of a certain frame k is initially set, the contour expanding unit 102h may expand the convergent contour of the previous frame k-1 and set the expanded contour as the initial contour. In addition, when the threshold determining unit 102g determines that the variance is greater than the threshold, the contour extending unit 102h extends the convergent contour generated by the contour converging unit 102e, and sets the expanded contour as an initial contour. Here, the contour expanding unit 102h may expand the convergent contour by the dynamic contour modeling method.

The contour extraction unit 102i is a contour extraction unit that extracts the convergence contour generated by the contour convergence unit 102e as the contour of the object when the threshold determination unit 102g determines that the variance is equal to or less than the threshold. .

The contour acquisition unit 102j is a contour acquisition unit that acquires the center position and / or in-contour brightness of the contour of the object extracted by the contour extraction unit 102i. The contour acquisition unit 102j stores the center position of the contour of the target object in the position information file 106b, and stores the in-contour brightness of the target object in the luminance information file 106c.

In addition, the cell division point setting unit 102k is configured such that when the target object is a cell and the time reverse order frame setting unit 102b is controlled to perform contour extraction processing of the frame k in reverse time order, the contours of a plurality of target objects This is a cell division point setting means for setting a cell division point and storing it in the cell division point file 106d when the center position is smaller than a predetermined threshold value. Here, as an example, the threshold value is a parameter of “same point determination distance” for defining how close two contraction contours are to be tracked as the same object.

The above is an example of the configuration of the contour extraction apparatus 100. Here, the contour extraction device 100 may be communicably connected to the network 300 via a communication device such as a router and a wired or wireless communication line such as a dedicated line. That is, as shown in FIG. 2, this system roughly provides an outline database 100, an external database that stores setting parameters and the like related to the dynamic outline modeling method, an external program that provides an external program such as an outline extraction program, and the like. The system 200 may be configured to be communicably connected via the network 300.

In this case, the communication control interface unit 104 of the contour extraction device 100 is an interface connected to a communication device (not shown) such as a router connected to a communication line or the like, and the contour extraction device 100 and the network 300 (or router). Communication control between the communication device and the like. That is, the communication control interface unit 104 has a function of communicating data with other terminals via a communication line.

The network 300 has a function of interconnecting the contour extraction device 100 and the external system 200, such as the Internet.

The external system 200 is connected to the contour extraction apparatus 100 via the network 300, and stores an external database for storing setting parameters and the like regarding the dynamic contour modeling method for the user, and an external such as a contour extraction program. It has a function to provide programs and the like. Here, the external system 200 may be configured as a WEB server, an ASP server, or the like. Further, the hardware configuration of the external system 200 may be configured by an information processing apparatus such as a commercially available workstation or a personal computer and its attached devices. Each function of the external system 200 is realized by a CPU, a disk device, a memory device, an input device, an output device, a communication control device, and the like in the hardware configuration of the external system 200 and a program for controlling them.

[Processing of Outline Extraction Device 100]
Next, an example of processing of the contour extraction apparatus 100 according to the present embodiment configured as described above will be described in detail below with reference to FIGS.

[Outline extraction processing]
First, the details of the contour extraction processing will be described with reference to FIGS. FIG. 3 is a flowchart showing an example of the contour extraction process of the contour extraction apparatus 100 according to the present embodiment.

As shown in FIG. 3, first, the initial contour setting unit 102d sets an initial contour in association with an ID for each object in the image data stored in the image data file 106a (step SA-1).

Next, the contour converging unit 102e generates a convergent contour by converging the set initial contour with Snakes (step SA-2). More specifically, the contour convergence unit 102e repeats the generation of the convergence contour and the calculation of the energy value based on the energy function E _sneaks for the set number of convergences, and generates the convergence contour so that the energy value is minimized. Optimize.
E _snakes = ∫ {E _in (v (s)) + E _img (v (s)) + E _con (v (s))} ds
(Here, E _in represents an internal energy term of the contour line, E _img represents an energy term that is minimal at the edge portion of the image, and E _con represents an energy term due to a binding force from the outside. )

Then, the luminance variance calculation unit 102f acquires the luminance on the convergence contour generated by the contour convergence unit 102e (step SA-3). More specifically, the luminance dispersion calculation unit 102f acquires the luminance value of the image pixel corresponding to the convergent contour. Here, the luminance variance calculation unit 102f performs color processing when the image data stored in the image data file 106a includes luminance information corresponding to a plurality of colors (for example, luminance values corresponding to the three primary colors of RGB). The p-th root of the p-th power sum of the luminance information weighted for each (where p is a parameter for the luminance information) may be acquired as the luminance.

Then, the luminance variance calculation unit 102f calculates the obtained luminance variance (step SA-4).

Then, the threshold determination unit 102g compares the luminance variance calculated by the luminance variance calculation unit 102f with a predetermined threshold (for example, a predetermined standard deviation value), and the variance is larger than the threshold, or Then, it is determined whether the variance is equal to or less than the threshold value (step SA-5).

When it is determined by the threshold determination unit 102g that the variance is larger than the threshold (No at Step SA-5), the contour expansion unit 102h determines that the convergent contour has not been correctly set in the true contour of the object (Step S5). SA-6) The convergence contour is expanded, the expanded convergence contour is set as a new initial contour, and the process returns to step SA-2 (step SA-7).

That is, when the variance of luminance on the convergent contour is large, it is considered that the convergent contour is taken inside the object. Arrangement is performed, and the above-described contour convergence process is repeated a plurality of times until the variance becomes equal to or less than the threshold value, thereby controlling the convergence contour to match the true contour of the object.

The transition of the contour by the above contour extraction processing will be described with reference to FIG. Here, FIG. 4 is a conceptual diagram showing an example in which the contour extraction processing according to the present embodiment is performed. In addition, the shaded area in FIG. 4 represents an object (cell). Also, the step numbers in FIG. 4 correspond to the step numbers in FIG.

First, the initial contour setting unit 102d sets an initial contour for the next frame k (<SA-1> in FIG. 4). In this example, the initial contour setting unit 102d sets the extracted contour obtained by extending the contour extracted for the previous frame as the initial contour for the next frame. Then, the contour convergence unit 102e performs contour fitting by Snakes to converge the initial contour and generate a convergent contour (<SA-2> on the left in FIG. 4). In this case, since the object has moved / deformed from the previous frame and moved beyond the expansion range of the Snakes contour, it intersects the true contour of the object, and the luminance gradient inside the object and the luminance gradient around the object Energy minimization occurs, and the Snakes contour cannot be set correctly to the true contour of the object. Therefore, since the variance of the luminance on the convergence contour calculated by the luminance variance calculation unit 102f is determined to be larger than the threshold by the threshold determination unit 102g, the contour expansion unit 102h expanded and expanded the convergence contour The convergence contour is set as a new initial contour (<SA-7> in FIG. 4). Again, the contour convergence unit 102e performs contour fitting by Snakes to converge the initial contour to generate a convergent contour (<SA-2> on the right in FIG. 4). Here, FIG. 5 is a diagram illustrating an example in which processing by the conventional method is performed and an example in which the contour extraction processing of the present embodiment is performed. In FIG. 5, the right diagram and the left diagram represent the same image obtained by imaging two cells as objects, and the convergent contours of contour numbers 0 to 3 are represented by polygons.

As shown in the left diagram of FIG. 5 (conventional example), in the conventional algorithm using Snakes, the cell moves beyond the expansion range of the contour (ROI), and the initial contour before the recontraction calculation and the cell region are In the case of crossing, the contour energy is minimized by the false contour generated by the luminance change inside the cell, and the ROI cannot be matched with the true contour of the cell (see contour number 2 in the left figure). Therefore, in the present embodiment, as shown in the right diagram of FIG. 5, the ROI is arranged in the cell internal region by repeating the contraction and expansion of the ROI using the luminance dispersion as an index in the process of the contour extraction process. Even in the case where it has occurred, the ROI can be expanded from the cell internal region to the outside, and can match the true contour of the object (see contour number 2 in the right figure).

Subsequently, when it is determined by the threshold determination unit 102g that the variance is equal to or less than the threshold (step SA-5, Yes), the contour extraction unit 102i has correctly set the convergence contour to the true contour of the target object. Judgment is made and the convergence contour is extracted as the contour of the object (step SA-8).

Then, the contour acquisition unit 102j stores the center position (X, Y) of the contour of the target object extracted by the contour extraction unit 102i in the position information file 106b in association with the ID, and extracts the contour of the target object. The internal luminance is stored in the luminance information file 106c in association with the ID (step SA-9).

This completes the description of an example of the contour extraction process.

[Time order analysis processing]
Next, the details of time order analysis processing for performing contour extraction processing in the normal order on the time axis for a plurality of frames 1 to n imaged at a plurality of times by time lapse photography or the like will be described below with reference to FIGS. This will be described with reference to FIG. Here, FIG. 6 is a flowchart illustrating an example of the time order analysis process.

As shown in FIG. 6, first, the time-ordered frame setting unit 102c sets the first frame 1 on the time axis in the image data stored in the image data file 106a as the start frame, and sets the initial contour setting unit 102d. Sets an initial contour for each object (step SB-1).

Contour enlargement processing (step SB-2) is not performed for the start frame, and then the contour converging unit 102e converges the initial contour set by the initial contour setting unit 102d by Snakes to generate a converging contour. (Step SB-3). The subsequent processing of steps SB-3 to SB-8 is performed in the same manner as the processing of steps SA-2 to SA-7 described above, and the processing is repeated until the luminance distribution on the convergence contour becomes equal to or less than the threshold value.

If it is determined by the threshold value determination unit 102g that the variance is equal to or less than the threshold value (step SB-6, Yes), the contour extraction unit 102i determines that the convergence contour has been correctly set to the true contour of the object. (Step SB-9), the convergence contour is extracted as the contour of the object, and the setting of the convergence contour is completed (Step SB-10).

Then, the contour acquisition unit 102j stores the center position (X, Y) of the contour of the target object extracted by the contour extraction unit 102i in the position information file 106b in association with the ID, and extracts the contour of the target object. The internal luminance is stored in the luminance information file 106c in association with the ID (step SB-11).

Then, the time-ordered frame setting unit 102c determines whether or not the analysis frame k that has undergone the contour extraction processing has reached the frame n that is the end frame (step SB-12). In other words, the time order frame setting unit 102c determines whether or not k = n.

When the time-ordered frame setting unit 102c determines that the analysis frame k has not reached the frame n (No in step SB-12), k is incremented (k → k + 1), and the analysis frame k is set to 1. The process returns to step SB-2 (step SB-13).

Then, the contour expanding unit 102h extends the contour extracted by the contour extracting unit 102i in the previous frame k-1, and sets the expanded contour as the initial contour of the analysis frame k (step SB-2). Here, FIG. 7 is a diagram schematically illustrating a state in which the extracted contour extracted in the previous frame is expanded, the initial contour in the next frame is set, and a convergent contour is generated. In FIG. 7, the left figure shows the object and the extracted contour in the previous frame, the center figure shows the object and the initial outline in the next frame, and the right figure shows the object and the convergence outline in the next frame. Represents.

As shown in FIG. 7, when the analysis frame k is shifted to the next frame, the contour expanding unit 102h expands the convergent contour extracted by the contour extracting unit 102i in the previous frame once and expands Since the contour is set as the initial contour in the next frame, even if the object moves or deforms, the convergent contour that matches the true contour of the object by setting the initial contour to surround the object Can be set. The process of the contour expanding unit 102h is the same in the following time reverse order analysis process.

Here, the initial contour setting unit 102d may perform feature extraction by performing Hough transform after the signal disappears, and reset the initial contour to the feature extraction portion in the vicinity of the contour. That is, when the signal (luminance) has disappeared in the previous frame k-1, in step SB-2 for the next frame k, the initial contour setting unit 102d performs feature extraction by Hough transform, and there is a signal. An initial contour may be set with feature points near the ROI in the frame. As a result, the Hough transform is performed to reset the initial contour near the signal disappearance point, so that the same object can be continuously tracked before and after the signal disappearance. The initial contour resetting process by the Hough transform will be described in detail with reference to FIGS. 9 and 10 in the following time reverse order analysis process.

Then, the time normal order frame setting unit 102c controls the processing from the above steps SB-2 to SB-13 to be executed in the normal order on the time axis for all frames (k = 1 to n), If it is determined that the final frame, frame n, has been reached (step SB-12, Yes), the process ends. This is the end of the description of an example of the time order analysis process.

[Reverse time analysis process]
Next, details of the time reverse order analysis processing for performing contour extraction processing in reverse order on the time axis for a plurality of frames 1 to n captured at a plurality of times by time lapse photography or the like will be described below with reference to FIGS. The description will be given with reference. Here, FIG. 8 is a flowchart showing an example of the time reverse order analysis processing.

As shown in FIG. 8, first, the time-reverse order frame setting unit 102b sets the last frame n on the time axis among the image data stored in the image data file 106a as a start frame (step SC-1).

Then, the initial contour setting unit 102d sets an initial contour for each object for the start frame (step SC-2).

Contour enlargement processing (step SC-3) is not performed for the start frame, and the contour converging unit 102e then converges the initial contour set by the initial contour setting unit 102d with Snakes to generate a converged contour. (Step SC-4). The subsequent steps SC-4 to SC-9 are performed in the same manner as the above-described steps SA-2 to SA-7, and the processing is repeated until the luminance dispersion on the convergence contour becomes equal to or less than the threshold value.

If it is determined by the threshold determination unit 102g that the variance is equal to or less than the threshold (step SC-7, Yes), the contour extraction unit 102i determines that the convergence contour has been correctly set to the true contour of the object. Then, the convergence contour is extracted as the contour of the object, and the setting of the convergence contour is completed (step SC-11).

Then, the contour acquisition unit 102j stores the center coordinates (X, Y) of the contour of the target object extracted by the contour extraction unit 102i in the position information file 106b in association with the ID, and extracts the contour of the target object. The internal luminance is stored in the luminance information file 106c in association with the ID (step SC-12).

Then, the cell division point setting unit 102k determines whether or not the distance between the center coordinates (X, Y) of the contours of any two objects is sufficiently close (ie, the same point determination). The determination is made based on the distance) (step SC-13).

When the cell division point setting unit 102k determines that the distance between the center coordinates is smaller than a predetermined threshold value (step SC-13, Yes), the time (or simply the frame number of the analysis frame) and the contour number are set. It is stored in the cell division point file 106d (step SC-14).

Then, the time-reverse order frame setting unit 102b determines whether or not the analysis frame k that has undergone the contour extraction processing has reached the frame 1 that is the end frame (step SC-15). That is, the temporal reverse order frame setting unit 102b determines whether or not k = 1.

When the time-reverse order frame setting unit 102b determines that the analysis frame k has not reached the frame 1 (No in step SC-15), it decrements k (k → k−1), and sets the analysis frame k to One is reversed and the process returns to Step SC-3 (Step SC-16).

Then, as described with reference to FIG. 7, the contour expanding unit 102h expands the contour extracted by the contour extracting unit 102i in the previous frame k + 1 as in step SB-2 described above, and the expanded contour Is set as the initial contour of the analysis frame k (step SC-3). Here, when the signal (luminance) disappears in the previous frame k + 1 and reappears in the next frame k, the initial contour setting unit 102d extracts the feature by the Hough transform, and in the frame where the signal is present An initial contour may be set with feature points in the vicinity of the ROI. Here, FIG. 9 is a diagram schematically illustrating an example of processing for resetting the initial contour by Hough transform.

As shown in FIG. 9, in the previous frame k + 1, when the signal disappears within the initial contour set by the contour extension unit 102h (when the luminance is equal to or lower than the threshold value), the initial contour setting unit 102d , A feature extraction may be performed by performing a Hough transform, and a new initial contour may be reset in a feature extraction portion in the vicinity of the contour. Here, FIG. 10 is a diagram illustrating an example of a contour extraction result according to the present embodiment. In the left graph of FIG. 10, the horizontal axis represents the number of frames, and the vertical axis represents the luminance. Moreover, the right figure of FIG. 10 is the figure which represented the analysis result by the phylogenetic tree.

As shown in FIG. 10, for example, when a cell that emits fluorescence according to the expression level is an analysis target, the luminance may change in the middle of the frame and the signal may disappear as shown in the graph. . According to the present embodiment, the initial contour is reset near the signal vanishing point by performing the Hough transform, so that the same object can be identified before and after the signal disappearing, as shown in the right figure. Even if the signal disappears, the same object can be continuously tracked.

In this way, the time reverse order frame setting unit 102b performs control so that the processes from the above steps SC-3 to SC-16 are executed in reverse order on the time axis for all frames (k = n to 1).

When the time-reversed order frame setting unit 102b determines that the frame 1, which is the final frame, has been reached (step SC-15, Yes), the cell division point setting unit 102k reads the center coordinates from the cell division point file 106d. Is read out and the contour number is read out to determine the cell division point (step SC-17). Here, FIG. 11 is a diagram for explaining the principle of determining the cell division point. In FIG. 11, the left figure, the center figure, and the right figure correspond to three frames arranged in order on the time axis, respectively. In the left figure, one object, in the central figure and the right figure, two objects Objects are shown schematically.

As shown in FIG. 11, in the case of performing the contour extraction process of the analysis frame k in the time axis reverse direction, it is determined that the merge (cell fusion) has occurred when the distance between the centers of the contours is less than the threshold value. In this case, the division (cell division or nuclear division) is considered to have occurred if viewed from the forward direction of the time axis. Therefore, the cell division point setting unit 102k uses this time (the time in the central diagram) as the cell division point. Set. Note that the cell division point setting unit 102k determines that the cell division is not performed when the two contours do not overlap in the next frame k-1, even if the distance between the centers of the contours is less than the threshold in a certain frame k. Even if the distance between the centers of the contours is greater than or equal to the threshold value in a certain frame k, if the two contours overlap in the next frame k-1, they are set as cell division points. May be. Here, FIG. 12 is a diagram illustrating, as an example, the result of contour extraction processing performed in reverse order on the time axis in time-continuous frames. In FIG. 12, numbers “282” to “285” indicate frame numbers, and numbers “0” and “1” indicate contour numbers (ID) for the respective contours. Each frame image shows HeLa cells expressing a nuclear-labeled probe in which GFP is linked to Histone H2B, which is a nuclear-related protein responsible for DNA structure stabilization, using Olympus (company name) -LCV100 system (trade name). This is a part (282nd to 285th frames) of image data acquired by long-time time-lapse imaging using 299 (299 frames) fluorescence every minute.

As shown in the analysis result of FIG. 12, since there are two cells as the object in the 285th frame, the contours of ID: 0 and ID: 1 exist independently of each other. In the 284th frame, cell nucleus division is confirmed, and in this frame, contour fusion occurs. In the 282nd frame, the contours of ID: 0 and ID: 1 are completely overlapped because the same cells before cell division are targeted. As described above, in the time-reverse order analysis processing, the cell division point setting unit 102k, as an example, displays the frame number (in this case, “285”) when the distance between the centers of the contours is less than the threshold, or two contours. The frame number (in this case, “284”) in the case of overlapping is set as the cell division point.

Here, the cell division point setting unit 102k may create a phylogenetic tree of cell division based on the time (or frame number) and the contour number stored in the cell division point file 106d. Here, FIG. 13 is a diagram showing a contour extraction result (left diagram) and a cell division analysis result (right diagram) in a certain frame k.

As shown in the left diagram of FIG. 13, when a plurality of contours coincide with the same object, a cell division point is set by the cell division point setting unit 102k, and the contour ID and frame number of the cell division point are registered. Is done. As a result, as shown in the right figure, it is possible to recognize the division of the analysis target and create a system diagram. Above, description of an example of a time reverse order analysis process is completed.

The above is the details of the processing of the contour extracting apparatus 100 according to the present embodiment.

[Example 1]
Example 1 of the present embodiment will be described below with reference to FIG. Here, first, the principle of the first embodiment will be described while comparing with the prior art.

With recent advances in microscopic recording technology and visualization of intracellular phenomena using fluorescent proteins, etc., time-lapse imaging (time-lapse imaging) of cells and analysis of luminance information make it possible to analyze the time and space of intracellular signals. Information can be obtained.

It is required to efficiently acquire information from the time-lapse moving image of this cell and extract scientifically meaningful information.

However, with conventional cell analysis software, it is difficult to efficiently track cells that move with time and change their shape and brightness, and cannot cope with cell division phenomena.

For example, in the conventional cell analysis software, a certain luminance value is set as a threshold for each frame, and a luminance set is extracted by performing binarization by luminance to be a cell region. Then, by performing statistical processing, the luminance sets having the minimum relative movement distance between frames are identified as corresponding to the same cell, and cell tracking is performed. In this conventional method, because of the characteristic of performing binarization by luminance, the analysis target is limited to a time-lapse image of cells having luminance (that is, a time-lapse image with a fluorescence microscope), and binarization is performed. Images that can be implemented were only those in which the background and the luminance of the cells had no unevenness or gradient, and the cell group of the photographed cells had no significant luminance change.

Also, since cell tracking is performed by statistical processing after obtaining luminance information for each frame, there are cases where the identity of the cell is not guaranteed and the cell cannot be accurately tracked, or a cell-specific phenomenon There were problems such as failure to follow division.

Therefore, in the first embodiment, the cell tracking ability is improved by providing a more efficient cell extraction method, and the above problem is solved by the cell division identification algorithm. That is, in the first embodiment, the cell region extraction is not performed by the binarization based on the luminance used in the conventional method, but the dynamic contour modeling method (Snakes) for extracting the region by paying attention to the cell contour. In the subsequent frame, the cell is tracked by once again performing contraction after expanding the range of the contour model.

This improves the tracking ability for cells that move while changing their shape over time and causes a change in brightness, and also performs cell analysis for transmission images that have been very difficult to binarize in the past. It has become possible.

In addition, by analyzing the time-series information in the opposite direction, an algorithm that determines that cell division has occurred at the time when the two contour information matches and the cells have fused can be identified. It was. Here, FIG. 14 is a flowchart showing the processing of the first embodiment.

That is, as shown in FIG. 14, the contour extracting apparatus 100 of the first embodiment opens the image data file 106a by the processing of the time reverse order frame setting unit 102b, and reads the microscopic time-lapse image data of the cells (step SD-1 ).

Then, the contour extracting apparatus 100 acquires necessary information from the image data file 106a by the processing of the time reverse order frame setting unit 102b, and then moves the read control k to the last frame n recorded (step SD-2). . In the first embodiment, the analysis proceeds sequentially from the reverse direction with respect to the time axis.

Then, the contour extracting apparatus 100 performs initial parameters (initial contour position, initial contour radius, contraction strength, number of contraction calculations) to be given to the dynamic contour model (Snakes) on the final frame n by the processing of the initial contour setting unit 102d. The enlargement range, the number of control points on the contour, etc.) are set (step SD-3). This initial contour is individually set for a plurality of cells to be analyzed, that is, initial contours having IDs of 1 to m are set for m cells.

Then, the contour extracting apparatus 100 performs energy model calculation so that the given initial radius is contracted to a position that best matches the contour of the cell by the processing of the contour converging unit 102e, and generates a convergent contour (step SD). -4).

Then, the contour extracting apparatus 100 acquires the luminance value of the image pixel on the convergent contour by the processing of the luminance variance calculating unit 102f (step SD-5), and calculates the acquired luminance variance (step SD-6). .

Then, when it is determined by the threshold determination unit 102g that the luminance variance is larger than the predetermined standard deviation (No in step SD-7), the contour extracting apparatus 100 expands the convergent contour as a new initial contour. Set and return to step SD-4.

Then, the contour extracting apparatus 100 repeats the contour convergence process until the variance is equal to or smaller than the predetermined standard deviation (steps SD-4 to 7), and when the variance is equal to or smaller than the predetermined standard deviation (step SD-7, Yes). Then, the outline of the cell is extracted by the processing of the outline extraction unit 102i, and the region of interest (ROI) is set (adapted to the dynamic outline model).

Then, the contour extracting apparatus 100 acquires luminance information from the obtained ROI by the processing of the contour acquiring unit 102j (step SD-8), and the luminance information in the ROI with ID: 1 to m is stored in the luminance information file 106c. Writing out (step SD-9).

When the contour extraction apparatus 100 determines that two or more ROIs indicate the same range and the ROIs are fused (that is, the cells are fused) by the processing of the cell division point setting unit 102k. (Step SD-10, Yes), when viewed from the forward direction of the time axis, it is determined that cell division has occurred, and the cell division point is recorded in the cell division point file 106d (step SD-11).

Then, the contour extracting apparatus 100 determines whether or not the reading control k has reached the first frame 1 by the processing of the time reverse order frame setting unit 102b (step SD-12), and has not reached the first frame 1 (Step SD-12, No), the control is shifted to the previous frame k-1 with respect to the time axis by the processing of the time reverse order frame setting unit 102b, and then the ROI range is expanded by the processing of the contour extension unit 102h (Step SD-13).

Then, the contour extracting apparatus 100 contracts the contour again in the expanded ROI range for the next frame, and extracts the cell range (steps SD-4 to SD-13).

The contour extracting apparatus 100 performs the above processing in order from the reverse direction of the time axis for all the frames n to 1.

After the analysis of all the frames, that is, when the first frame 1 is reached (step SD-12, Yes), the contour extracting apparatus 100 determines whether the information is recorded in the luminance information file 106c or the cell division point file 106d. Information such as luminance change information and a transition diagram (phylogenetic tree) of cell division is output to the output unit 114.

This completes the process of the first embodiment.

[Example 2]
Example 2 in the present embodiment will be described below with reference to FIGS. 15 and 16.

In the second embodiment, the brightness for contour calculation by the dynamic contour modeling method (Snakes) is appropriate so that the contour can be accurately extracted even when the image is expressed in multiple colors instead of a single color. The cell contour extraction and the cell tracking were performed after performing various processing. The flow of processing other than the calculation of the brightness for contour calculation is basically the same as that in the first embodiment.

That is, in the second embodiment, in order to cope with the case where the image data includes luminance information corresponding to a plurality of colors, the p-th root of the p-th power sum of the luminance information weighted for each color is used as the luminance for contour calculation. Set as Intensity. More specifically, when the fluorescence brightness of each channel obtained by the fluorescence microscope is Ch1, Ch2, and Ch3, the brightness intensity for contour calculation is obtained by the following equation.
Intensity = ((a) * (Ch1) ^ (p) + (b) * (Ch2) ^ (p) + (c) * (Ch3) ^ (p)) ^ (1 / p)
(Here, a, b, and c are arbitrary weighting parameters for the fluorescent luminances Ch1, Ch2, and Ch3, respectively, and p is a parameter for the overall luminance.)

Each parameter in the above formula (weighting parameters a, b, c, and parameter p for the overall luminance) is a real number specified by, for example, a double type floating point number, and more preferably 0. These are real numbers.

The brightness intensity for Snakes contour calculation reflecting the brightness of a plurality of fluorescent images can be obtained by this calculation formula. Then, based on the calculated luminance intensity value, the coordinates where the Snakes contour exists are calculated by the same method (time reverse order analysis processing) as in the first embodiment described above, and exist within the obtained Snakes contour. By acquiring the luminance information (Ch1, Ch2, Ch3) of each original fluorescent image, analysis from an imaging image made up of a plurality of fluorescent image groups is made possible.

Here, FIG. 15 is a diagram showing a result of analyzing a fluorescence imaging image of a primary cultured cell of ZebraFish into which a fluorescent protein type indicator Fucci that emits fluorescence specifically in the cell cycle is introduced, based on this experimental example 2. It is. Here, this indicator Fucci (Fluorescent Ubiquitation-based Cell Cycle Indicator) emits green fluorescence in the S / G2 / M phase of the cell cycle and emits red fluorescence in the G1 phase ( Each of the time required until the green fluorescent protein (mAG) is completely decomposed and the time required until the red fluorescent protein (mKO2) is synthesized and accumulated to emit sufficient fluorescence requires a certain amount of time. In this sense, the fluorescence does not change from green to red immediately after cell division). In addition, the fluorescence imaging image used for the analysis is an image group of about 1000 frames obtained by time-lapse photographing of the primary culture cells of ZebraFish into which Fucci was introduced for 48 hours. Of these, in FIG. The representative image is shown together with the extracted contours (

contour numbers

2, 3, and 5). In this analysis, since the luminance adjustment is performed for each luminance information (Ch1, Ch2, Ch3) of the multicolor fluorescence imaging image in the preprocessing stage for the analysis, each calculation formula in the above calculation formula on the analysis algorithm The parameter was 1.

As shown in FIG. 15, the cells exhibit S / G2 / M phase green fluorescence around 0h-12h, G1 phase red fluorescence around 15h-27h, and S / G2 / M around 30h-45h. The green fluorescence of the period is exhibited, and then the red fluorescence of the G1 period is exhibited again in the vicinity of 48 h. Regardless of the difference between the two fluorescent colors, the outline of the cell can be accurately extracted in any image. It was confirmed that Also, as shown in FIG. 15, three contours (

contour numbers

2, 3, and 5) that were common at 0h are contours of contour number 2 and contour number 3 in a frame (* 1) near 12h. , 5 and the two common contours (contour numbers 3 and 5) are the contours of contour number 3 and contour 5 in the frame (* 2) near 45h. The cell division point was set in the vicinity of these two time points (* 1, * 2) according to Example 2. As shown in FIG. 15, both of these two cell division points are points in time when the green fluorescence before the red fluorescence is weakened, which almost coincides with the M phase indicated by the indicator Fucci. It was confirmed that the cell division point was correctly analyzed.

Here, FIG. 16 is a graph showing the change in the intensity of each fluorescence in the contour extracted in the second embodiment, and the cell division created from the cell tracking result and the luminance information in the contour in the second embodiment. It is a figure which shows no phylogenetic tree.

As shown in the graph of FIG. 16 (upper figure), the intensity of green fluorescence (mAG) in the S / G2 / M phase and the red fluorescence in the G1 phase by Fucci in the contour extracted by the second embodiment. It was shown that the intensity of (mKO2) changed depending on the cell cycle. Further, as shown in the phylogenetic tree of FIG. 16 (below), the length of each period of the cell cycle accompanying cell division is visualized by coloring the phylogenetic tree of cell division reflecting the luminance information of each fluorescent color. Succeeded in doing.

This is the end of the description of the second embodiment.

[Other embodiments]
Although the embodiments of the present invention have been described so far, the present invention is not limited to the above-described embodiments, but can be applied to various different embodiments within the scope of the technical idea described in the claims. May be implemented.

For example, in the above-described embodiment, the example in which one cell in the fluorescence microscope image is analyzed as an object has been described, but the present invention is not limited to this. Here, FIG. 17 is a diagram illustrating an example in which an ROI is set to a HeLa cell colony in a differential interference image and contour extraction of the entire colony is performed. Thus, the present invention is not only an analysis from a fluorescence image in which a cell region and a background region are expressed as a luminance difference, but also an image in which an object and a background such as a differential interference image are not expressed as a luminance difference. It is also possible to extract a target object from the above, and it is also possible to apply a cell group composed of a plurality of cells as a target object. That is, in the above-described embodiment, one initial contour is set for one cell in the fluorescence microscope image, and the contour is extracted. However, the object is extracted from other than the fluorescence microscope observation image. Alternatively, one initial contour may be set for a group of a plurality of cells or a plurality of objects, and the contours of these groups may be extracted.

In addition, for example, the case where the contour extracting apparatus 100 performs processing in a stand-alone form has been described as an example, but processing is performed in response to a request from a client terminal configured with a separate housing from the contour extracting apparatus 100, You may comprise so that the process result may be returned to the said client terminal.

In addition, among the processes described in the embodiment, all or part of the processes described as being automatically performed can be performed manually, or the processes described as being performed manually can be performed. All or a part can be automatically performed by a known method.

In addition, unless otherwise specified, the processing procedures, control procedures, specific names, information including registration data for each processing, parameters such as search conditions, screen examples, and database configurations shown in the above documents and drawings Can be changed arbitrarily.

Further, regarding the contour extracting apparatus 100, each illustrated component is functionally conceptual and does not necessarily need to be physically configured as illustrated.

For example, all or some of the processing functions provided in each device of the contour extraction device 100, particularly the processing functions performed by the control unit 102, are interpreted and executed by a CPU (Central Processing Unit) and the CPU. It may be realized by a program to be executed, or may be realized as hardware by wired logic. The program is recorded on a recording medium to be described later, and is mechanically read by the contour extracting apparatus 100 as necessary. That is, the storage unit 106 such as ROM or HD stores a computer program for performing various processes by giving instructions to the CPU in cooperation with an OS (Operating System). This computer program is executed by being loaded into the RAM, and constitutes a control unit in cooperation with the CPU.

The computer program may be stored in an application program server connected to the contour extraction apparatus 100 via an arbitrary network 300, and may be downloaded in whole or in part as necessary. It is.

The program according to the present invention can also be stored in a computer-readable recording medium. Here, the “recording medium” is an arbitrary “portable physical medium” such as a flexible disk, a magneto-optical disk, a ROM, an EPROM, an EEPROM, a CD-ROM, an MO, a DVD, a Blu-ray Disc, or the like. It includes a “communication medium” that holds a program in a short period of time, such as a communication line or a carrier wave when a program is transmitted via a network represented by a LAN, WAN, or the Internet.

Also, “program” is a data processing method described in an arbitrary language or description method, and may be in any form such as source code or binary code. Note that the “program” is not necessarily limited to a single configuration, but is distributed in the form of a plurality of modules and libraries, or in cooperation with a separate program represented by an OS (Operating System). Including those that achieve the function. Note that a well-known configuration and procedure can be used for a specific configuration for reading a recording medium, a reading procedure, an installation procedure after reading, and the like in each device described in the embodiment.

Various databases and the like (image data file 106a to cell division point file 106d) stored in the storage unit 106 are storage means such as a memory device such as RAM and ROM, a fixed disk device such as a hard disk, a flexible disk, and an optical disk. Various programs, tables, databases, web page files, etc. used for various processing and website provision are stored.

The contour extraction apparatus 100 is connected to an information processing apparatus such as a known personal computer or workstation, and the software (including programs, data, etc.) for realizing the method of the present invention is installed in the information processing apparatus. It may be realized.

Furthermore, the specific form of distribution / integration of the devices is not limited to that shown in the figure, and all or a part of them may be functional or physical in arbitrary units according to various additions or according to functional loads. Can be distributed and integrated.

As described above in detail, according to the contour extracting apparatus, the contour extracting method, the program, and the recording medium according to the present invention, the true contour of the target object can be accurately extracted, and medical and pharmaceutical In addition to fields such as drug discovery, biological research and clinical testing, it is extremely useful in various fields such as crime prevention systems.

DESCRIPTION OF SYMBOLS 100 Contour extraction apparatus 102 Control part 102a Frame setting part 102b Time reverse order frame setting part 102c Time normal order frame setting part 102d Initial contour setting part 102e Contour convergence part 102f Luminance dispersion | distribution calculation part 102g Threshold determination part 102h Contour expansion part 102i Contour extraction part 102j Outline acquisition unit 102k Cell division point setting unit 104 Communication control interface unit 106 Storage unit 106a Image data file 106b Location information file 106c Luminance information file 106d Cell division point file 108 Input / output control interface unit 112 Input unit 114 Output unit 200 External system 300 network

Claims

In the contour extraction apparatus for extracting the contour of the object, comprising at least a storage unit and a control unit,
The storage unit
Image data storage means for storing image data obtained by imaging the object,
The control unit
In the image data stored in the image data storage means, initial contour setting means for setting an initial contour of the object;
Contour convergence means for generating a convergent contour by converging the initial contour set by the initial contour setting means;
A luminance variance calculating means for obtaining the luminance on the convergent contour generated by the contour converging means, and calculating the luminance variance;
Threshold determination for comparing the variance of the brightness calculated by the brightness variance calculating means with a predetermined threshold and determining whether the variance is greater than the threshold or whether the variance is less than or equal to the threshold Means,
Contour expansion means for expanding the convergent contour generated by the contour convergence means and setting the expanded contour as the initial contour when the threshold determination means determines that the variance is greater than the threshold; ,
Contour extraction means for extracting the convergence contour generated by the contour convergence means as the contour of the object when the threshold determination means determines that the variance is equal to or less than the threshold;
An outline extraction apparatus comprising:
The contour extracting apparatus according to claim 1,
The contour converging means and the contour expanding means generate and expand the convergent contour by a dynamic contour modeling method;
Contour extraction device characterized by
In the outline extraction device according to claim 1 or 2,
The control unit
Contour acquisition means for acquiring a center position and / or luminance within the contour of the object extracted by the contour extraction means;
An outline extraction apparatus further comprising:
In the outline extraction device according to any one of claims 1 to 3,
The image data is composed of a plurality of frames captured at a plurality of times,
The contour extraction apparatus, wherein the control unit controls the plurality of frames to be processed in the normal order or the reverse order of the time.
The contour extracting apparatus according to claim 4,
The control unit
A cell division point that sets a cell division point when the plurality of frames are controlled to be processed in the reverse order of the time and the center position of the outline of the plurality of objects is smaller than a predetermined threshold value. Setting means,
An outline extraction apparatus further comprising:
In the contour extracting device according to any one of claims 1 to 5,
The luminance variance calculating means is:
When the image data includes luminance information corresponding to a plurality of colors, a p-th root of the p-th power sum of the luminance information weighted for each color (where p is a parameter for the luminance information). Obtaining the luminance and calculating the variance of the luminance,
Contour extraction device characterized by
A contour extraction method executed in a contour extraction apparatus that extracts a contour of an object, comprising at least a storage unit and a control unit,
The storage unit
Image data storage means for storing image data obtained by imaging the object,
Executed in the control unit,
An initial contour setting step for setting an initial contour of the object in the image data stored in the image data storage means;
A contour convergence step for generating a convergent contour by converging the initial contour set in the initial contour setting step;
A luminance dispersion calculating step for obtaining the luminance on the convergent contour generated in the contour convergence step and calculating the variance of the luminance;
A threshold for comparing the variance of the luminance calculated in the luminance variance calculating step with a predetermined threshold and determining whether the variance is greater than the threshold or whether the variance is less than or equal to the threshold. A determination step;
Contour expansion that expands the convergent contour generated in the contour convergence step and sets the expanded contour as the initial contour when the variance is determined to be greater than the threshold in the threshold determination step Steps,
A contour extraction step for extracting the convergence contour generated in the contour convergence step as the contour of the object when the variance is determined to be equal to or less than the threshold in the threshold determination step;
Contour extraction method characterized by including.
A program for executing in a contour extraction device having at least a storage unit and a control unit,
The storage unit
Image data storage means for storing image data obtained by imaging the object,
In the control unit,
An initial contour setting step for setting an initial contour of the object in the image data stored in the image data storage means;
A contour convergence step for generating a convergent contour by converging the initial contour set in the initial contour setting step;
A luminance dispersion calculating step for obtaining the luminance on the convergent contour generated in the contour convergence step and calculating the variance of the luminance;
A threshold for comparing the variance of the luminance calculated in the luminance variance calculating step with a predetermined threshold and determining whether the variance is greater than the threshold or whether the variance is less than or equal to the threshold. A determination step;
Contour expansion that expands the convergent contour generated in the contour convergence step and sets the expanded contour as the initial contour when the variance is determined to be greater than the threshold in the threshold determination step Steps,
A contour extraction step for extracting the convergence contour generated in the contour convergence step as the contour of the object when the variance is determined to be equal to or less than the threshold in the threshold determination step;
A program for running